Senior Machine Learning Engineer

Red Hat•Boston, MA

17h•$186,160 - $287,410•Hybrid

About The Position

Red Hat is hiring a Senior Machine Learning Engineer to lead the design and implementation of high-performance Python, C++, CUDA, and Triton code for vLLM-based inference systems, GPU kernels, and numerical computing components in production-scale LLM serving environments. The role focuses on improving inference latency, throughput, scalability, and cost efficiency through algorithm and system optimization, profiling, and large-scale deployment work. The engineer will own end-to-end optimization of vLLM inference pipelines, evaluate and integrate emerging LLM serving techniques, contribute to upstream open-source vLLM, and set engineering standards for performance-critical code. Telecommuting is permitted within normal commuting distance of Red Hat’s Boston, Massachusetts office.

Requirements

Master’s degree (U.S. or foreign equivalent) in Computer Science, Computer Engineering, or a related field and three years of experience in the job offered or a related role; OR bachelor’s degree (U.S. or foreign equivalent) in Computer Science, Computer Engineering, or a related field and five years of experience in the job offered or a related role.
At least three years of experience in high-performance Python and modern C++ development.
At least three years of experience in performance analysis, benchmarking, and profiling on NVIDIA GPUs using Nsight or similar tools.
At least three years of experience with GPU memory architecture and memory optimization for deep learning workloads.
At least two years of experience making open-source pull requests to PyTorch or vLLM.
At least two years of experience optimizing CUDA, Triton, or CUTLASS kernels.
At least two years of experience with LLM inference optimization, including quantization, KV cache, or batching.
At least two years of experience with distributed computing, including inter- and intra-GPU communication protocols.
At least two years of experience leading technical design and reviewing performance-critical code.

Responsibilities

Architect, develop, and optimize LLM inference algorithms, including quantization, sparsification, speculative decoding, paged attention, KV cache optimization, batching, scheduling, and memory management.
Profile, benchmark, and model performance on NVIDIA GPUs using tools such as Nsight Systems, Nsight Compute, CUPTI, PyTorch Profiler, CUDA, Triton, and CUTLASS; identify bottlenecks and deliver measurable system-level improvements.
Own end-to-end optimization of vLLM-based inference pipelines, including kernel tuning, runtime optimization, distributed inference, multi-GPU serving, tensor and pipeline parallelism, and large-scale deployment performance.
Evaluate, prototype, and integrate LLM serving techniques from research and open-source communities, including quantization, sparsity, speculative decoding, continuous batching, prefix caching, attention optimization, and model compression.
Set engineering standards for performance-critical code, conduct peer reviews, guide testing and benchmarking strategies, and use AI-assisted development tools to support code quality, reliability, and maintainability.
Contribute to and maintain upstream open-source vLLM by reviewing pull requests, triaging issues, participating in design discussions, and supporting release readiness.
Track LLM systems research and GPU computing advances, identifying opportunities to improve serving efficiency, hardware utilization, and model acceleration.