Global LLM Data - LLM Training Operation Coding Analyst

ByteDance • Los Angeles, CA

85d

About The Position

As a key member of our LLM Global Data Team, the LLM Training Operations Analyst will play a pivotal role in managing the intricate processes involved in training large language models (LLMs) with diverse coding datasets. This role focuses on overseeing and improving operational workflows, primarily for code-related projects, ensuring they are delivered with high quality and efficiency.

Requirements

• Experience in project management, particularly in technology or data-related fields.
• Strong understanding of large language models and coding datasets.
• Proficiency in Python (Pandas, NumPy, Matplotlib) and SQL for data analysis.
• Experience in workflow design and operational improvement initiatives.
• Ability to analyze data quality and model performance using statistical methods.

Nice To Haves

• Experience working with cross-functional teams in a global environment.
• Familiarity with data annotation processes and quality assurance methodologies.
• Previous experience in mentoring or leading teams.

Responsibilities

• Lead and manage multiple coding-focused LLM training projects, ensuring timelines, quality standards, and objectives are met.
• Track project progress, identify risks, and implement corrective actions as necessary to keep projects on course.
• Build and maintain strong relationships with product managers, engineers, researchers, data annotators, and other cross-functional team members.
• Communicate project updates, address concerns, and align expectations to ensure successful project outcomes.
• Coordinate meetings and discussions with global teams to ensure seamless project execution and work with external vendors and trainers per project demands.
• Design, manage, and optimize workflows for coding-focused LLM training projects, including training design, QA processes, and performance tracking to meet project needs.
• Collaborate closely with product managers, engineers, and cross-functional teams to ensure alignment on quality metrics and project expectations.
• Conduct quality and productivity improvement experiments to enhance operational processes for code-related training data.
• Lead and support general annotation operation improvement initiatives across various data domains.
• Develop and maintain technical guidelines and casebooks to support consistent, high-quality data production.
• Design and implement data analysis strategies for LLM coding projects.
• Analyze annotation quality, model performance, and dataset coverage using statistical and programmatic methods.
• Identify data gaps and failure patterns through slice-based evaluations and error analysis.
• Use Python (Pandas, NumPy, Matplotlib) and SQL to generate insights and support model training operations.
• Collaborate with researchers to inform training strategies and data improvements.
• Provide mentorship and guidance to team members, helping to develop their skills and ensuring the delivery of high-quality outputs.
• Foster a collaborative environment where team members can share knowledge and best practices to improve overall performance.

What This Job Offers

• Career Level: Mid Level
• Industry: Publishing Industries
• Number of Employees: 5,001-10,000 employees