Senior Machine Learning Engineer

Red Hat•Boston, MA

8d•Hybrid

About The Position

Red Hat is hiring a Senior Machine Learning Engineer to lead the design and implementation of high-performance Python, C++, CUDA, and Triton code for vLLM-based inference systems, GPU kernels, and numerical computing components in production-scale large language model (LLM) serving environments. The role focuses on improving inference latency, throughput, scalability, and cost efficiency through algorithm, kernel, runtime, and system-level optimization. The engineer will own end-to-end optimization of vLLM inference pipelines, including distributed and multi-GPU serving, and will evaluate and integrate techniques from current research and open-source communities. The position also contributes to the upstream vLLM project and establishes engineering standards for performance-critical code, testing, benchmarking, reviews, and maintainability. Telecommuting is permitted within normal commuting distance of Red Hat’s Boston, Massachusetts office.

Requirements

Master’s degree (U.S. or foreign equivalent) in Computer Science, Computer Engineering, or a related field and three years of experience in the job offered or a related role; or a bachelor’s degree (U.S. or foreign equivalent) in one of those fields and five years of experience in the job offered or a related role.
At least three years of experience with high-performance Python and modern C++ development.
At least three years of experience with performance analysis, benchmarking, and profiling on NVIDIA GPUs using Nsight or similar tools.
At least three years of experience with GPU memory architecture and memory optimization for deep learning workloads.
At least two years of experience making open-source pull requests to PyTorch or vLLM.
At least two years of experience with CUDA, Triton, or CUTLASS kernel optimization.
At least two years of experience with LLM inference optimization, including quantization, KV cache, or batching.
At least two years of experience with distributed computing, including inter- and intra-GPU communication protocols.
At least two years of experience leading technical design and reviewing performance-critical code.

Responsibilities

Architect, develop, and optimize LLM inference algorithms, including quantization, sparsification, speculative decoding, paged attention, KV cache optimization, batching, scheduling, and memory management.
Profile, benchmark, and model performance on NVIDIA GPUs to identify bottlenecks and deliver measurable system-level improvements.
Own end-to-end optimization of vLLM-based inference pipelines, including kernel tuning, runtime optimization, distributed inference, multi-GPU serving, tensor and pipeline parallelism, and large-scale deployment performance.
Evaluate, prototype, and integrate LLM serving techniques from research and open-source communities, including continuous batching, prefix caching, attention optimization, and model compression.
Set engineering standards for performance-critical code; conduct code reviews; guide testing and benchmarking strategies; and use AI-assisted development tools to support code quality, reliability, and maintainability.
Contribute to and maintain upstream open-source vLLM infrastructure by reviewing pull requests, triaging issues, participating in design discussions, and supporting release readiness.
Track LLM systems research and GPU computing advancements and identify opportunities to improve serving efficiency, hardware utilization, and model acceleration.