Software Engineer (Backend-Focused)
About The Position
AZX is a rapidly growing public benefit corporation founded in 2024, focused on accelerating positive impact in critical industries through AI transformation. We work with leaders in real estate, energy, logistics, and utilities on challenges in clean energy, decarbonization, climate risk, energy systems, and global economics. We are building a company for long-term success and aim to be the ultimate workplace for those passionate about AI and positive impact. We are seeking a Staff or Senior ML Engineer to take ownership of the technical backbone for serving and evaluating models at scale. This high-leverage individual contributor role spans our inference platform (GPU scheduling, autoscaling, serving infrastructure for vLLM/SGLang across cloud and customer-managed clusters) and the evaluation systems that assess model, prompt, and agent changes. The ideal candidate will create technical direction for reliable model serving, possess architectural ownership of complex ML infrastructure problems, and have the judgment to build guardrails for safe and rapid team progress.
Requirements
- 4+ years of experience in backend engineering fundamentals: distributed systems, API design, and production experience in Go, Rust, or async Python.
- Comfort working across a range of platform concerns rather than one narrow specialty.
- Eagerness to grow into deeper specialization in gateway, sandbox, or inference infrastructure over time.
- High emotional intelligence and a learning mindset.
- Strong collaboration skills.
- Enjoy others' success and a fun, positive environment.
- Comfortable making decisions in the face of ambiguity and course correcting as needed.
- Currently authorized to work in the United States on a full-time basis.
Nice To Haves
- Familiarity with LLM-specific backend concerns (rate limiting, caching, token accounting).
- Exposure to Kubernetes and containerization; interest in sandboxing or security.
- Experience in both startup and enterprise environments.
- Past work in energy, real estate, utilities, climate or related fields.
- Experience and passion in one or more of: Additional web frameworks (e.g. Svelte, Vue, Angular), Lower-level languages e.g. C++, Rust, Networking paradigms e.g. GraphQL, Websockets, ML capabilities e.g. Sk-learn, xgboost, Pytorch/Tensorflow/JAX, Onnx, Additional database types such as graph or vector databases, DevOps e.g. CI/CD pipelines, Docker, Kubernetes, Terraform, Pulumi and/or Bicep, Generative AI e.g. prompt engineering, RAG, fine-tuning, tooling ecosystem.
Responsibilities
- Build and maintain backend services for our LLM gateway, including routing, rate limiting, key management, and observability in front of the inference fleet.
- Contribute to sandboxing and isolation infrastructure for safe execution of agent-generated code, collaborating with security engineers.
- Support Kubernetes-based platform services, such as operators and autoscaling logic related to the inference platform.
- Write high-performance backend code in Go, Rust, or async Python (FastAPI/Starlette), utilizing infrastructure like Envoy and gRPC.
- Instrument services with OpenTelemetry to ensure behavior, latency, and cost remain observable as the platform scales.
- Collaborate across gateway, sandbox, and inference platform teams, adapting to shifting priorities.
Benefits
- Competitive early-stage startup compensation
- Bonus eligibility
- Health insurance with meaningful coverage for dependents
- Flexible paid time off
- Equity
- Fully remote culture
- Training and learning opportunities
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
Education Level
No Education Listed
