Machine Learning System Software Engineer

Apple • Sunnyvale, CA

2d

About The Position

Join Apple’s Apple Neural Engine (ANE) team to develop system software that enables high-performance, low-power AI on Apple products, including Vision Pro, iPhone, iPad, and Mac. The role focuses on defining and implementing APIs that allow clients to load, compile, schedule, and execute machine-learning models on device. The APIs will support model lifecycle management, memory allocation strategies, multi-model scheduling, and concurrent execution while scaling to new hardware capabilities.

You will work closely with driver, firmware, and compiler teams to design the runtime-to-silicon interface and expose hardware features through appropriate abstractions. You will profile and optimize end-to-end inference paths, address bottlenecks in scheduling, compilation, and execution, and support client teams integrating the APIs. As the role evolves, opportunities may include technical leadership, influencing design decisions, supporting other engineers, and helping guide Apple’s AI capabilities across its ecosystem.

Requirements

• Bachelor’s degree and at least 3 years of relevant industry experience.
• Experience defining interfaces used by other teams or external developers, with attention to lifecycle, error handling, and forward compatibility.
• Deep proficiency in C and C++ in large production system software.
• Understanding of runtime systems, including process and thread models, memory management, IPC/RPC, and resource lifecycle.
• Understanding of software-hardware interfaces, such as registers, DMA, command queues, or similar accelerator interaction patterns.
• Experience shipping production system software.
• Strong communication skills and experience working across team boundaries, including framework, compiler, and hardware teams.

Nice To Haves

• Experience building or extending ML runtimes, inference engines, or accelerator driver stacks, such as TensorRT, ONNX Runtime, XLA, Metal, or Vulkan compute.
• Familiarity with Swift, Rust, or another memory-safe systems language.
• Familiarity with ML model compilation pipelines and how runtime APIs interact with compiler outputs such as graph IR and compiled binaries.
• Experience with multi-client runtime scenarios, including arbitrating hardware access, managing priority/QoS, and handling client lifecycle events such as connect, disconnect, and crash recovery.
• Knowledge of neural-network inference, including operator execution, tensor memory layout, pipelining, and batching strategies.

Responsibilities

• Design and implement APIs for Apple Neural Engine clients to load, compile, schedule, and execute on-device models.
• Architect APIs for model lifecycle management, memory allocation strategies, multi-model scheduling, and concurrent execution.
• Ensure the runtime scales with new hardware capabilities.
• Collaborate with driver, firmware, and compiler teams to co-design the runtime-to-silicon interface and expose hardware features through appropriate abstractions.
• Profile and optimize end-to-end inference paths, identifying bottlenecks in scheduling, compilation, and inference execution.
• Support clients adopting the APIs and help ensure efficient integration.
• As the role evolves, demonstrate technical leadership, influence design decisions, collaborate with and support other engineers, and help guide Apple’s AI-driven capabilities.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level