Senior Staff / Principal Machine Learning Scientist, AI Inference & Optimization
About The Position
As a Senior Staff Machine Learning Scientist, you will own the inference and optimization layer that makes AI in agentic workflows fast, efficient, and production-grade. You will fine-tune and evaluate models, push latency and throughput on real hardware, and build the runtime that executes bounded AI tasks, validated against usage from Netskope’s large customer base. This role offers high-impact ownership of the model layer of a net-new product that changes the performance and economics of agentic AI, utilizing a cutting-edge and unusual stack involving quantization, KV-cache and memory management, sparsity, fine-tuning, and hardware acceleration under real-world resource constraints. You will leverage Netskope’s customer footprint for production signals to deploy, validate, and iterate fast.
Requirements
- 10+ years of overall industry experience, with 4+ years hands-on in ML/AI (model development, fine-tuning, and inference optimization).
- Hands-on with fine-tuning (e.g. LoRA/QLoRA), quantization (GGUF/AWQ/GPTQ), and inference runtimes (vLLM/SGLang, TensorRT-LLM, ONNX Runtime, llama.cpp, or MLX/CoreML).
- Strong Python; comfort reaching into C++ for low-level interop is a plus.
- Solid grasp of transformer internals and the levers that move real inference performance and cost: KV cache, attention, batching, memory footprint.
- Fluency with agentic coding systems and genuine curiosity about agent harnesses like Claude Code, Pi, and Codex.
- Clear communication: able to distill a model or infra bottleneck into an actionable concept for cross-functional teammates.
Nice To Haves
- On-device or edge inference experience is a strong plus.
- PhD in a related field strongly preferred.
Responsibilities
- Build and optimize the model inference path: quantization, KV-cache optimization, batching, and latency/memory/throughput tuning on constrained, commodity hardware.
- Fine-tune and evaluate models for bounded tasks; build eval harnesses that gate a capability to release on real accuracy, latency, and security relevance.
- Design and grow the task execution runtime (bounded sub-agents), pushing toward dynamic task generation and context compaction.
- Drive hardware acceleration / sparsity and support for larger models as the platform matures.
- Partner with the systems and backend engineers to ship capabilities end-to-end and iterate on real production signals.
Benefits
- Comprehensive health plan
- Bonus plan (for non-sales roles)
- Stock award program
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
