Senior Machine Learning Engineer II

HubSpot • Cambridge, MA

69d • $191,000 - $305,600

About The Position

We're seeking a Machine Learning Engineer to join our Developer Experience team and own the operational deployment and performance optimization of our AI coding infrastructure. You'll be the expert who ensures our code generation models run reliably and efficiently at scale, powering the systems that help developers write software. You'll work across the full model lifecycle. This includes fine-tuning open source models for code generation tasks and implementing RLHF pipelines to improve code quality and align with developer workflows, then taking those customized models and deploying them at scale. You'll evaluate and test bleeding-edge code models as they're released, debug distributed inference frameworks like vLLM, SGLang, and Ray, resolve GPU memory allocation issues, manage CUDA dependencies and kernel compatibility, and navigate the ever-shifting landscape of ML library ecosystems. Your primary focus will be maximizing the throughput and capacity we can extract from our GPU infrastructure, turning experimental code models and fine-tuned variants into production-ready systems that generate code at scale for thousands of developers.

Requirements

• Bachelor's degree in Computer Science or related field.
• 5+ years of experience in machine learning or ML infrastructure roles.
• Strong proficiency with LLM inference frameworks (vLLM, SGLang, Ray, or similar).
• Experience with Python and deep learning frameworks (PyTorch, TensorFlow).
• Hands-on experience with GPU optimization, CUDA, and distributed computing.
• Experience with model evaluation, fine-tuning, or RLHF for LLMs.
• Familiarity with MLOps best practices around model deployment, monitoring, and quality assurance.

Nice To Haves

• Master degree in a specialized discipline like Data Science or Machine Learning.
• Experience specifically with code generation models or AI coding tools.
• Background in developer tooling (static analysis, IDEs, development tools, etc.).
• Contributions to open source ML infrastructure projects.

Responsibilities

• Own the operational deployment and performance optimization of AI coding infrastructure.
• Ensure code generation models run reliably and efficiently at scale.
• Fine-tune open source models for code generation tasks.
• Implement RLHF pipelines to improve code quality.
• Deploy customized models at scale.
• Evaluate and test new code models as they are released.
• Debug distributed inference frameworks like vLLM, SGLang, and Ray.
• Resolve GPU memory allocation issues.
• Manage CUDA dependencies and kernel compatibility.
• Maximize throughput and capacity from GPU infrastructure.

Benefits

• Annual cash compensation range: $191,000-$305,600 USD.
• Participation in HubSpot's equity plan to receive restricted stock units (RSUs).
• Potential eligibility for overtime pay.
• Flexible work environment with options for remote or in-office work.
• Commitment to fair compensation practices and transparency.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Industry: Publishing Industries
• Education Level: Bachelor's degree
• Number of Employees: 5,001-10,000 employees