Engineer Jobs

About The Position

Requirements

• Strong programming skills in one or more compiled languages (Swift, C++, or Objective-C)
• Strong Python skills
• Solid computer science fundamentals, including data structures, algorithms, and clean, testable code
• Ability to quickly learn and adapt to evolving technologies and tools, such as GenAI-assisted coding, new ML frameworks, and emerging LLM/agent tooling
• Experience with backend/API development and production debugging
• Excellent communication and cross-team collaboration skills, with experience working effectively within large, cross-functional organizations
• M.S. or B.S. in Computer Science, Machine Learning, or a related field (or equivalent experience)

Nice To Haves

• Experience evaluating ML, LLM, or agent-based systems, including familiarity with metrics, scoring methodology, trajectory and outcome analysis, and techniques like prompting, RAG, or LLM as judge
• Understanding of reinforcement learning and the underlying techniques behind modern LLMs (e.g. transformer architectures, RLHF/RLAIF, fine-tuning, reward modeling ) and frameworks such as PyTorch or Hugging Face, as applied to evaluation and reward signal design
• Familiarity with eval-driven development — defining success criteria and test cases from product goals and real user workflows rather than abstract benchmarks
• Experience with data science methods applied to quality measurement — defining ground truth, measuring inter-rater agreement (e.g. Cohen's/Fleiss' kappa), and validating automated scorers using basic statistical techniques (e.g. correlation, confidence intervals, hypothesis testing)
• Experience with MLOps, deployment, and test/eval environment management — containerization, CI/CD, model versioning, monitoring, cloud platforms (AWS, GCP, or similar), and staging or provisioning environments to produce repeatable, deterministic conditions
• Comfort communicating and collaborating effectively across multicultural teams and time zones

Responsibilities

• Build systems and tooling for evaluation in Siri development.
• Integrate human evaluation, real user feedback, reward and alignment signals, and data science rigor.
• Apply machine learning engineering techniques for model evaluation and reward modeling.
• Develop infrastructure for trustworthy quality signals.
• Contribute to work streams spanning evaluation quality, reward/alignment signals, and data science.
• Implement "evals first" thinking by building tooling and harnesses.
• Design for observability and reproducibility in quality measurement.
• Define and execute paths in an unexplored space.
• Build and ship AI-facing tooling.