Machine Learning System Engineer

Atlassian•Seattle, WA

20h•$147,906 - $232,650•Hybrid

About The Position

Working at Atlassian Atlassians can choose where they work – whether in an office, from home, or a combination of the two. That way, Atlassians have more control over supporting their family, personal goals, and other priorities. We can hire people in any country where we have a legal entity. As a ML System Engineer on the AI & ML Platform’s Inference team, you will design and optimize large-scale model serving systems end-to-end. You will have the chance to own everything from distributed infrastructure (global KV cache, continuous batching, load balancing, auto-scaling) to deep low-level optimizations (GPU kernels, quantization, speculative decoding).

Requirements

3+ years of software engineering experience
Deep low-level systems programming (C/C++ or Rust)
Experience with large-scale, high-concurrent production serving.
Experience with GPU inference engines (vLLM, SGLang, Triton, TensorRT-LLM, etc.).

Nice To Haves

1+ years of system performance optimization experience
Low-level inference optimizations: GPU kernels
Algorithmic inference optimizations: quantization, speculative decoding, distillation
Experience with testing, benchmarking, and reliability of inference services.
Experience designing and implementing CI/CD infrastructure for inference.
Strong background in system optimizations: batching, caching, load balancing, parallelism.

Responsibilities

Architect and implement scalable distributed infrastructure for model serving (load balancing, auto-scaling, batch scheduling, global KV cache).
Optimize latency and throughput of model inference under real production workloads.
Build reliable, high-concurrency serving systems that serve billions of requests reliably
Benchmark, fine-tune, and accelerate inference engines.
Create robust CI/CD infrastructure for seamless model deployment and inference engine updates.
Partner with senior ML engineers to fine‑tune and deploy open-source LLMs