Senior Software Engineer – Inference

Bitwise • Laurel, MD

3d • Onsite

About The Position

We're building the next generation of AI infrastructure — and we want a senior engineer who's excited to help lead that effort. As a Senior Software Engineer on our AI infrastructure team, you'll play a central role in establishing and maintaining the foundation that powers AI capabilities across the customer organization. Your primary focus will be ensuring that users throughout the inference software stack have access to the highest-quality large language models (LLMs) available. This is not a quiet, steady-state role. Mission needs shift quickly, new technologies emerge constantly, and you'll be expected to stay sharp, adapt, and turn loosely defined problems into working solutions. You'll lead technically, mentor your teammates, and engage directly with stakeholders to make sure engineering outcomes match mission goals.

Requirements

• 12 years of experience in software engineering, or a B.S. in a technical discipline with 12 years of experience (16 years of experience accepted in lieu of a B.S.)
• Experience with Python and/or other modern programming languages
• Experience with Argo CD and/or other CI/CD frameworks
• Experience with Kubernetes and Helm
• Experience with AWS or other cloud service providers
• Strong communication skills with the ability to mentor other engineers
• Excellent leadership and stakeholder management skills
• Demonstrated ability to balance hands-on engineering with leadership and coordination responsibilities
• Active TS/SCI clearance with polygraph

Nice To Haves

• Experience with vLLM, LiteLLM, or similar inference-serving frameworks
• Experience with other LLM hosting frameworks and practices
• Experience supporting production software using Site Reliability Engineering (SRE) best practices
• Experience with Elastic, Grafana/Prometheus, or other observability frameworks and practices
• Experience with Docker and containerization
• Experience in traffic shaping and quality-of-service engineering
• Knowledge of and genuine interest in hosting AI capabilities

Responsibilities

• Lead the evaluation, configuration, and deployment of inference models for users across the organization
• Translate high-level stakeholder goals into concrete engineering outcomes
• Mentor engineers and drive development best practices and standards across the team
• Develop in-house services and techniques to ensure continual, high-quality inference for the customer
• Engage with other teams to build solid infrastructure and integrate LLM-powered tools for user needs
• Collaborate with teammates on surge efforts to support short-term, high-priority inference requirements

Benefits

• Top salaries
• Pick your PTO (3 to 5 weeks with corresponding pay adjustment)
• All 11 federal holidays, paid
• Up to 2 snow days, paid
• 4x match on the first 6% of 401(k) contributions (up to 24% company match)
• 100% employer-paid medical, dental, vision, life, and disability insurances (or salary boost if already covered)
• $5,250 annual education assistance
• Spot bonuses

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: Associate degree