Large Model Pre-training Algorithm Researcher

Tencent • Palo Alto, CA

3d

About The Position

1. Core Model Development: Lead the pre-training of generative large language models (LLMs), encompassing large-scale pre-training, long-context extension, and the exploration of efficient linear attention variants. 2. Technical Innovation: Investigate and track cutting-edge research to drive breakthroughs in AIGC (AI Generated Content) capabilities. 3. Knowledge Embedding: Research efficient methods for knowledge injection and explore mechanisms for continual/online model updating. 4. Capability Optimization: Conduct research on in-context learning and advanced prompting strategies to fully elicit knowledge from large models.

Requirements

• Technical Proficiency: Proficient in Python and Linux environments. Expertise in deep learning frameworks such as PyTorch or TensorFlow.
• Theoretical Foundation: Deep understanding of deep learning principles with a solid grasp of Transformer-based architectures.
• Problem Solving: Strong analytical and problem-solving skills. Highly self-motivated with the ability to thrive in a challenging research environment.

Nice To Haves

• Hands-on experience in large-scale model pre-training is highly preferred.
• Candidates with strong research output, evidenced by publications in top-tier conferences (e.g., NeurIPS, ICML, ICLR, ACL), are strongly encouraged to apply.

Responsibilities

• Lead the pre-training of generative large language models (LLMs)
• Investigate and track cutting-edge research to drive breakthroughs in AIGC (AI Generated Content) capabilities.
• Research efficient methods for knowledge injection and explore mechanisms for continual/online model updating.
• Conduct research on in-context learning and advanced prompting strategies to fully elicit knowledge from large models.

Benefits

• Employees hired for this position may be eligible for a sign on payment, relocation package, and restricted stock units, which will be evaluated on a case-by-case basis.
• Subject to the terms and conditions of the plans in effect, hired applicants are also eligible for medical, dental, vision, life and disability benefits, and participation in the Company’s 401(k) plan.
• The Employee is also eligible for up to 15 to 25 days of vacation per year (depending on the employee’s tenure), up to 13 days of holidays throughout the calendar year, and up to 10 days of paid sick leave per year.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Education Level: No Education Listed