Senior Software Engineer – Inference

Bitwise • Laurel, MD

3d • Onsite

About The Position

We're building the next generation of AI infrastructure — and we need a Senior Software Engineer who can help lead the charge. In this role, you'll be a core member of an AI infrastructure team focused on making the highest-quality large language models (LLMs) accessible to users across a complex, mission-driven organization. This isn't just about keeping the lights on — it's about shaping how AI capabilities are delivered, scaled, and improved in an environment where mission needs evolve quickly and the stakes are real. You'll wear a few hats here. Some days you'll be deep in the stack, building and refining inference services. Other days you'll be translating big-picture stakeholder goals into concrete engineering outcomes, mentoring your teammates, or collaborating across teams to integrate LLM-powered tools into user workflows. If you thrive at the intersection of hands-on engineering and technical leadership — and you're genuinely excited about where AI infrastructure is headed — this role was built for you.

Requirements

• Experience with Python and/or other modern programming languages
• Experience with Argo CD and/or other CI/CD frameworks
• Experience with Kubernetes and Helm
• Experience with AWS or other cloud service providers
• Strong communication skills and demonstrated ability to mentor other engineers
• Excellent leadership and stakeholder management skills
• Ability to balance hands-on engineering responsibilities with leadership and coordination duties
• 12 years of experience in a relevant technical discipline, or a B.S. in a technical discipline with 12 years of experience; 16 years of experience may be substituted in lieu of a B.S.

Nice To Haves

• Experience with vLLM, LiteLLM, or similar inference-serving frameworks
• Experience with other LLM hosting frameworks and practices
• Experience supporting production software using Site Reliability Engineering (SRE) best practices
• Experience with Elastic, Grafana/Prometheus, or other observability frameworks and practices
• Experience with Docker and containerization
• Experience in traffic shaping and quality-of-service engineering
• Knowledge of and genuine interest in hosting and scaling AI capabilities

Responsibilities

• Lead the evaluation, configuration, and deployment of inference models for end users
• Translate high-level stakeholder goals into actionable engineering outcomes
• Mentor engineers and champion development best practices and standards across the team
• Develop in-house services and techniques that ensure continual, high-quality inference for customers
• Engage with partner teams to establish solid infrastructure foundations and integrate LLM-powered tools into user workflows
• Collaborate on surge efforts to address short-term, high-priority inference needs as they arise

Benefits

• Top salaries
• Pick your PTO — 3 to 5 weeks of PTO with a corresponding adjustment to your pay
• All 11 federal holidays, paid
• Up to 2 snow days, paid
• 4x match on the first 6% contributed to 401(k)
• 100% employer-paid medical, dental, vision, life, and disability insurances
• $5,250 annual education assistance for training, certifications, tuition, and student loan repayments
• Spot bonuses for obtained certifications, customer recognition, and other achievements

What This Job Offers

• Job Type: Full-time
• Career Level: Senior