Software Engineer - Hosted Model Infrastructure

Palantir Technologies•Washington, DC

1d•$145,000 - $200,000•Hybrid

About The Position

We are a software engineering team with expertise in enabling ML models in production. We deploy AI models to run in variety of environments: air-gapped government networks, forward-deployed defense environments, edge nodes, and enterprises with strict data sovereignty requirements. Our customers rely on us for frontier AI capabilities running on hardware they control, often with constrained GPU resources and limited direct access. Rising to that challenge and meeting those expectations is what Palantir's excels at. We treat models like any other software: continuously tested, continually delivered, packaged for reproducible deployment, and built for long-term maintainability. You will own services end-to-end, and work across the full stack, from inference engines, GPU scheduling to deployment pipelines, observability, and integration with Palantir's platform. The goal is to deliver new models and capabilities quickly and continuously. Join us if you want to solve problems at the intersection of infrastructure and machine learning that directly enable critical customers.

Requirements

4+ years of professional software engineering experience building and operating production systems
Engineering background in Computer Science, Mathematics, Software Engineering, Physics, or similar field
Strong coding skills with demonstrated proficiency in programming languages, such as Java, C++, Python, Rust, or similar languages. Familiarity with the Python ML ecosystem is valuable.
Experience with containers, Kubernetes, and deploying backend services in production environments
Strong written and verbal communication skills and ability to iterate quickly with teammates, incorporating feedback and holding a high bar for quality

Nice To Haves

Active US Security clearance, or eligibility and willingness to obtain a US Security clearance is beneficial, but not necessary

Responsibilities

Building high-performance model serving infrastructure that integrates with security models, hardware constraints, and different inference engines
Designing intelligent request handling including authentication, rate limiting, concurrency control, and audit logging for multi-tenant model access
Building and maintaining packaging and deployment pipelines enabling fast, secure, and reliable model rollouts across on-premises and air-gapped environments
Developing observability for production AI systems to enable easy service monitoring and fast incident triage and resolution
Debugging complex issues and performance problems throughout the stack, including open source inference engines, container runtimes, and GPU drivers, in environments you cannot always access directly
Designing and running testing and benchmarking infrastructure that validates model deployments across varying GPU hardware before they reach production
Working with product teams and customers to understand requirements, debug production issues, and deliver the models and capabilities they need
Integrating hosted model infrastructure with Palantir's deployment, configuration, and identity systems

Benefits

Employees (and their eligible dependents) can enroll in medical, dental, and vision insurance as well as voluntary life insurance
Employees are automatically covered by Palantir’s basic life, AD&D and disability insurance
Commuter benefits
Relocation assistance
Take what you need paid time off, not accrual based
2 weeks paid time off built into the end of each year (subject to team and business needs)
10 paid holidays throughout the calendar year
Supportive leave of absence program including time off for military service and medical events
Paid leave for new parents and subsidized back-up care for all parents
Fertility and family building benefits including but not limited to adoption, surrogacy, and preservation
Stipend to help with expenses that come with a new child
Employees can enroll in Palantir’s 401k plan