Senior Software Engineer – Inference

Bitwise • Laurel, MD

7d

About The Position

We're building the next generation of AI infrastructure to power innovation across our customer organization, and we need a senior software engineer to help lead the charge. In this role, you'll be instrumental in building and maintaining the foundation for customer AI capabilities while supporting a broader ecosystem of AI-enabled applications. Your focus will be ensuring access to the highest quality large language models across our inference software stack. As a Senior Software Engineer – Inference, you'll take on significant technical and leadership responsibilities. You'll lead the evaluation, configuration, and deployment of inference models for our users, translating high-level stakeholder goals into engineering outcomes. You'll mentor engineers on our team, helping to grow development best practices and standards that elevate everyone's work. You'll develop in-house services and techniques to guarantee continual high-quality inference service for our customer, and you'll engage across teams to establish solid infrastructure for our services and integrate LLM-powered tools for user needs. We move fast in this space — requirements shift as mission needs evolve and new technologies emerge. You'll be expected to keep sharpening your skills and learning to turn loosely defined problems into working solutions. You'll also collaborate with teammates on surge efforts to support short-term, high-priority inference needs, balancing hands-on engineering with leadership and coordination responsibilities.

Requirements

• An active TS//SCI clearance with polygraph is required.
• 12+ years of experience in software engineering, or a B.S. in a technical discipline plus 8+ years of experience.
• Proficiency with Python and/or other modern programming languages.
• Experience with Argo CD and/or other CI/CD frameworks.
• Experience with Kubernetes and Helm.
• Experience with AWS or other cloud service providers.
• Strong communication skills and demonstrated ability to mentor other engineers.
• Excellent leadership and stakeholder management skills.
• Ability to balance hands-on engineering with leadership and coordination responsibilities.

Nice To Haves

• Experience with vLLM, LiteLLM, or similar inference-serving frameworks.
• Experience with other LLM hosting frameworks and practices.
• Experience supporting production software using Site Reliability Engineering (SRE) best practices.
• Experience with Elastic, Grafana/Prometheus, or other observability frameworks and practices.
• Experience with Docker and containerization.
• Experience in traffic shaping and quality-of-service engineering.
• Knowledge of and interest in hosting AI capabilities.

Responsibilities

• Lead the evaluation, configuration, and deployment of inference models for users.
• Translate high-level stakeholder goals into engineering outcomes.
• Mentor engineers on the team, helping to grow development best practices and standards.
• Develop in-house services and techniques to guarantee continual high-quality inference service.
• Engage across teams to establish solid infrastructure for services and integrate LLM-powered tools for user needs.
• Keep sharpening skills and learning to turn loosely defined problems into working solutions.
• Collaborate with teammates on surge efforts to support short-term, high-priority inference needs.
• Balance hands-on engineering with leadership and coordination responsibilities.

Benefits

• Top salaries
• Flexible PTO (3 to 5 weeks with corresponding pay adjustment)
• Paid federal holidays (11)
• Paid snow days (up to 2)
• 401(k) with 4x match on the first 6% contributed (up to 24% company match)
• 100% employer-paid medical, dental, vision, life, and disability insurances (or salary boost if already covered)
• $5,250 annual education assistance for training, certifications, tuition, and student loan repayments
• Spot bonuses for obtained certifications, customer recognition, etc.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: Associate degree