Software Engineer: Agentic Evaluation

Apple • Cupertino, CA

11h

About The Position

At Apple, great ideas have a way of becoming great products, services, and customer experiences very quickly. Bring passion and dedication to your job and there's no telling what you could accomplish. Do you want to help measure and improve the quality of Siri across the devices, features, and experiences people rely on every day? Apple's Agentic Evaluation Engineering organization builds the infrastructure that determines how Siri's quality is measured, trusted, and improved. You'll join a team focused on expanding what that platform can reach: the devices and environments we evaluate on, the features and interaction modalities we exercise, and the realistic, repeatable conditions we stage to ground each evaluation. The surface area is large and growing. You'll have real autonomy in how you tackle it, and you'll build infrastructure the team can rely on as priorities shift.

In this role you'll contribute to the infrastructure, tooling, and pipelines that let us evaluate Siri reliably and at scale. You'll have meaningful autonomy in how you get there, and the work will move across several areas of expansion as priorities evolve. The specific platforms, frameworks, and components will change over time, so we're looking for someone who can transition smoothly across them and bring strong evaluation and systems engineering fundamentals to whatever the team needs next.

Requirements

• Strong programming skills in one or more compiled languages (Swift, C++ or Objective-C).
• Python scripting skills for tooling and automation.
• Solid understanding of computer science fundamentals.
• Ability to quickly learn new technologies and adapt to evolving requirements.
• Excellent communication skills and ability to collaborate across teams.
• M.S. or B.S. in Computer Science, Machine Learning, or related field (or equivalent experience).

Nice To Haves

• Experience staging, provisioning, or controlling test or evaluation environments to produce repeatable, deterministic conditions.
• Experience evaluating ML, LLM or agent-based systems, including familiarity with metrics, scoring methodology, or trajectory and outcome analysis.
• Experience designing or operating test infrastructure at scale, such as device provisioning, environment restore, warm pools, or continuous integration systems.
• Proficiency with Python and Swift in a production setting.
• A track record of approaching problems flexibly and cutting through ambiguity, adapting your approach to reach the right outcome and setting a clear path when requirements are not yet defined.
• A talent for focusing and simplifying, stripping away what is not essential and distilling complex decisions down to the factors that matter.
• A history of collaborating across teams and communicating effectively with both technical and program audiences.

Responsibilities

• Contribute to the infrastructure, tooling, and pipelines that let us evaluate Siri reliably and at scale.
• Expand the platform's reach in terms of devices, environments, features, and interaction modalities.
• Stage realistic, repeatable conditions for evaluation.
• Build infrastructure that the team can rely on as priorities shift.
• Transition smoothly across different platforms, frameworks, and components.
• Bring strong evaluation and systems engineering fundamentals to team needs.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Education Level: Associate degree