Senior ML Engineer, Optimization

Neurophos • Austin, TX

2d • $160,000 - $215,000 • Onsite

About The Position

We are seeking an experienced machine learning engineer to develop advanced post-training quantization methods for large language models (LLMs), diffusion models, and other ML applications for our revolutionary optical inference engines. This role is critical to demonstrating the full potential of our metamaterial-based optical processing units (OPUs) by adapting state-of-the-art AI models to leverage our ultra-high-throughput, low-precision compute architecture. The ideal candidate will bridge the gap between cutting-edge ML research and novel hardware capabilities, ensuring customers can seamlessly deploy their AI workloads on Neurophos hardware.

Requirements

• PhD, or equivalent research experience, in machine learning, applied mathematics, optimization, numerical analysis, computer science, or a closely related field
• 5+ years of experience in machine learning engineering, with at least 3 years focused on model optimization and deployment.
• Research or advanced engineering experience in neural network quantization, model compression, numerical optimization, or efficient inference.
• Strong knowledge of numerical linear algebra, including matrix factorizations, conditioning, covariance estimation, and iterative methods.
• Experience with one or more of non-convex optimization, discrete optimization, manifold optimization, second-order methods, or constrained optimization.
• Strong proficiency in PyTorch and familiarity with other ML frameworks, including JAX, Triton, and TensorFlow.
• Hands-on experience with transformer architectures, LLMs, and diffusion models.
• Experience designing controlled numerical experiments and distinguishing algorithmic improvements from calibration or benchmark artifacts.
• Strong written communication and research collaboration skills.

Nice To Haves

• Experience with low-precision inference optimization (INT8, FP8, or lower).
• Background in analog or optical computing architectures.
• Knowledge of in-memory computing paradigms and matrix-vector multiplication acceleration.
• Knowledge of randomized numerical linear algebra, sketching, or structured transforms.
• Publications in quantization, optimization, numerical linear algebra, model compression, or efficient ML.
• Experience with vector quantization, lattice methods, learned codebooks, or rate-distortion ideas.
• Experience with large-scale batch inference optimization.
• Familiarity with prefill versus decode optimization strategies in LLM inference.
• Experience conducting experiments on models large enough to expose scaling and generalization problems.

Responsibilities

• Develop and execute hardware-aware post-training methods for full model quantization.
• Investigate preconditioning and formulate quantization as non-convex, discrete, constrained, or second-order optimization and develop practical solutions.
• Contribute to refining Neurophos's quantization strategy.
• Design controlled numerical experiments to understand potential improvements and secondary effects due to analog processing hardware.
• Build research-quality implementations and reproducible experiment harnesses for testing candidate methods.
• Adapt models from open-source repositories and customer private models.
• Work with models in various formats, including PyTorch, Triton, JAX, and emerging frameworks.
• Design and execute re-quantization, retraining, and other model adaptation techniques to minimize accuracy loss during precision reduction.
• Optimize GEMM operations for high-throughput execution.
• Collaborate with hardware, software, and architecture teams to co-optimize model architectures for optical compute characteristics.
• Publish research papers on novel optimization techniques and methodologies, with appropriate IP protection.

Benefits

• 100% coverage of base health plan premiums for you and your dependents, plus HSA contributions.
• Unlimited PTO.
• 401(k) matching and stock option opportunities.
• Full suite of voluntary benefits, including Dental, Vision, Life, Hospital, Critical Illness, and Accident insurance.
• Personalized Benefits. Choose the plans that fit your life and take the cash back for those that don’t.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: Ph.D. or professional degree