Member of Technical Staff, ML Inference Engineering
About The Position
Sanas is bringing real-time speech and language models on-premise — deployed at scale directly inside sovereign data centers, not served from behind a hosted cloud endpoint. It's one of the most demanding environments in the industry: strict latency budgets, massive concurrency, and infrastructure that needs to be private and reliable. We're looking for a deeply hands-on, senior engineer to help lead that build. This is someone who shapes core infrastructure and architecture decisions rather than just executing against a specification, and who naturally raises the level of the engineers working alongside them.
Requirements
- 5+ years of experience writing high-quality, high-performance code
- Familiarity with NVIDIA GPU architecture and CUDA
- Fluency in the LLM serving stack, from kernels and quantization up to schedulers and autoscaling
- A research-leaning or systems background in LLM, Speech-to-Text, Text-to-Speech, or Speech-to-Speech inference, with work you can point to
- A record of shipping research or systems that other people build on, whether in a lab or in industry
Nice To Haves
- Experience serving low-precision (FP4/FP8) models, multiple LoRA adapters within one model instance (Multi-LoRA), or models distributed across several GPU nodes
- Experience developing large-scale, high-load production systems
- Experience maintaining or contributing to open-source ML projects
- Experience managing machine learning workloads on Kubernetes clusters
- Experience with InfiniBand or RoCE networking
- Experience with bare-metal provisioning and lifecycle management
- Experience operating large-scale AI training or inference clusters
- Experience with hardware health monitoring and predictive failure detection
- Experience with distributed storage systems
Responsibilities
- Optimize system and GPU performance for high-throughput AI workloads across multi-node training and inference
- Analyze and improve latency, throughput, memory usage, and compute efficiency
- Profile system performance to detect and resolve GPU- and kernel-level bottlenecks
- Implement low-level optimizations using CUDA, Triton, and other performance tooling
- Improve support for mixed precision, quantization, and model graph optimization
- Build and maintain performance benchmarking and monitoring infrastructure
- Scale inference and training systems across multi-GPU, multi-node environments
- Own and evolve our inference engine, enabling reliability and performance at scale
- Develop and optimize runtime inference services for large-scale AI applications
- Implement robust, fault-tolerant systems for data ingestion and processing
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
Education Level
No Education Listed
