Software Engineer – Inference

Bitwise • Laurel, MD

2d • Remote

About The Position

We're building next-generation AI infrastructure, and we want you on the team doing it. As a Software Engineer on our AI infrastructure team, you'll play a hands-on role in delivering and maintaining the foundation that powers AI capabilities for our customer — making sure users across the organization have reliable access to the highest-quality large language models (LLMs) available. This is not a slow-moving role. Requirements shift as mission needs evolve and new technologies emerge, and you'll be expected to keep pace — turning loosely defined problems into working solutions. If you thrive on learning fast and building things that matter, you'll fit right in.

Requirements

• 3 years of experience in a relevant technical role, with a B.S. in a technical discipline — or 7 years of experience in lieu of a degree
• Experience with Python and/or other modern programming languages
• Familiarity with Argo CD and/or other CI/CD frameworks
• Experience with Kubernetes and Helm
• Familiarity with AWS or other cloud service providers
• Ability to learn new technologies quickly and independently
• Strong communication skills and a genuine willingness to ask questions and collaborate
• An active TS/SCI clearance with polygraph is required for this role

Nice To Haves

• Experience with vLLM, LiteLLM, or similar inference-serving frameworks
• Experience with other LLM hosting frameworks and practices
• Experience supporting production software using Site Reliability Engineering (SRE) best practices
• Experience with Elastic, Grafana/Prometheus, or other observability frameworks
• Experience with Docker and containerization
• Experience in traffic shaping and quality-of-service engineering
• A genuine interest in hosting and scaling AI capabilities

Responsibilities

• Procuring, configuring, and testing new inference models and preparing them for release to our user base
• Developing in-house services and techniques to ensure continual, high-quality inference for our customer
• Working with model vendor teams and representatives to build reliable pipelines for closed-source model usage
• Jumping in on surge efforts to support short-term, high-priority inference needs as they arise
• Collaborating across teams to establish solid infrastructure for our services and integrate LLM-powered tools for users

Benefits

• Pick your PTO — Everyone values time and money differently, so we give you the flexibility to choose between 3 and 5 weeks of PTO with a corresponding adjustment to your pay.
• All 11 federal holidays, paid!
• Up to 2 snow days, paid!
• We'll quadruple (4x!) the first 6% you contribute to your 401(k), giving you up to a 24% company match. Contributing less than 6%? Unclaimed matches come right back to you as extra income, giving you a guaranteed 24% that goes to your retirement, to your paycheck, or both.
• 100% employer-paid medical, dental, vision, life, and disability insurances.
• $5,250 annual education assistance for training, certifications, tuition, and even student loan repayments.
• Spot bonuses for obtained certifications, customer recognition, and just about anything else that makes us go "Hot damn!".
• Top salaries

What This Job Offers

• Job Type: Full-time
• Career Level: Senior