ML Deployment Engineer, Edge AI - Q

Apple

2d

About The Position

The ML Deployment Engineer will join the on-device AI team and make state-of-the-art machine learning models run efficiently on Apple silicon. The role focuses on compressing, converting, compiling, and optimizing models to deliver intelligent experiences entirely on-device while meeting performance, power, and privacy expectations.

The engineer owns the end-to-end deployment pipeline, taking trained models from research and applied teams through compression and compilation to runtime on real hardware. This includes profiling where compute and energy are spent, diagnosing operations that fall off accelerators, and tuning models to meet strict latency, memory, and energy budgets. The role contributes throughout the product development lifecycle, from concept through deployment, and collaborates with ML research and applied ML teams to shape deployable models and inform architecture decisions.

Requirements

• Bachelor’s or Master’s degree in Computer Science, Electrical Engineering, Computer Engineering, or a related field, or equivalent practical experience.
• At least 5 years of professional experience deploying machine learning models to production, with significant focus on on-device, edge, or other resource-constrained environments.
• Strong programming skills in Python and C/C++.
• Solid understanding of deep learning fundamentals and hands-on experience with a modern ML framework such as PyTorch.
• Practical experience with model compression and optimization, such as quantization, pruning, palettization, or mixed precision, and working knowledge of floating-point and fixed-point/integer numerics and their quality trade-offs.
• Experience taking models through a conversion and compilation toolchain to a hardware target, such as Core ML, TensorRT, TFLite, ONNX Runtime, or similar.
• Demonstrated ability to profile and optimize code and models against strict latency, memory, and power constraints.
• Strong debugging skills and comfort working across ML models, compilers, and hardware.
• Hands-on experience deploying to hardware accelerators or NPUs, and familiarity with Apple Neural Engine, GPU, and CPU compute paths.
• Excellent problem-solving skills, strong communication and teamwork, and a passion for building innovative products.

Nice To Haves

• Experience with the Core ML toolchain, inspecting or transforming model IRs such as MLIR, and writing custom graph passes.
• Familiarity with compiler analytics, including diagnosing operator fallback, layout, and accelerator residency issues.
• Experience with calibration or data-based compression methods such as GPTQ, activation quantization, or sparsity, and layer-wise sensitivity analysis.
• Background in on-device LLMs or generative models, including tokenization, KV-cache, and quantized inference.
• Experience building benchmarking, numerical-validation, and CI tooling for model deployment.
• Familiarity with low-level compute kernels, SoC memory hierarchies, and hardware-software co-design.

Responsibilities

• Own the model deployment pipeline from trained checkpoint to on-device execution, ensuring product-quality accuracy, latency, memory, and power.
• Apply model compression techniques, including quantization, palettization, pruning, and mixed precision, and validate their accuracy, size, and latency trade-offs.
• Convert and compile models to the Core ML stack and target the Apple Neural Engine, GPU, and CPU.
• Analyze and maximize accelerator residency; diagnose operations rejected by the accelerator or dispatched to fallback compute units, and resolve issues through graph changes.
• Profile end-to-end inference on real hardware and optimize compute-unit placement, memory footprint, bottlenecks, and energy consumption.
• Collaborate with ML research and applied ML teams to co-design models for efficient deployment and feed hardware constraints into model architecture decisions.
• Build and maintain tooling and automation for reproducible compression, compilation, benchmarking, and numerical validation.
• Stay current with advances in efficient ML inference, model compression, and accelerator architectures.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior