Software Engineer, Inference

Thinking Machines Lab•San Francisco, CA

18h

About The Position

Thinking Machines is hiring a Software Engineer, Inference, to own the reliability, scale, and efficiency of the systems serving its models to real users. The role supports Tinker's live, multi-tenant serving platform and products built on the company's models, bridging inference techniques with the operational needs of production traffic. It is a production-facing systems role covering rollouts, capacity, incidents, and the reliability of a growing platform.

Requirements

Experience operating large-scale, latency-sensitive production systems.
Proficiency in Python and Go or another systems language.
Experience with observability, monitoring, and incident response for production services.
Strong understanding of distributed systems and how they fail at scale.

Nice To Haves

Experience running production inference for large language models or other large-scale ML systems.
Experience with deployment and rollout systems, such as canarying, blue/green deploys, or feature flags.
Experience with capacity planning and cost optimization for GPU or TPU infrastructure.
Familiarity with inference-specific techniques, such as batching, caching, or quantization, and their operational implications.
Comfort with being on-call and leading incident response for critical production systems.
Comfort working with high autonomy in a fast-changing, early-stage environment.

Responsibilities

Operate and scale production inference systems serving live traffic, including Tinker's multi-tenant serving platform.
Own rollouts for new models, model versions, and inference optimizations, ensuring safe, incremental production deployments.
Build and improve observability, alerting, and capacity planning to help detect, diagnose, and resolve production issues quickly.
Partner with inference and research teams to productionize new serving techniques without compromising reliability.
Lead incident response for production inference issues, including root cause analysis and durable fixes.
Design for graceful degradation, failover, and redundancy to keep serving resilient as usage grows.
Manage capacity and cost tradeoffs for serving infrastructure as traffic and model sizes scale.