Senior Machine Learning Engineer

Red River • Toronto, ON

4d • Hybrid

About The Position

At Red Hat, we are dedicated to advancing the future of AI through open-source solutions, specifically focusing on bringing the power of open-source LLMs and vLLM to enterprises. Our AI Inference team is at the forefront of accelerating AI for businesses and simplifying GenAI deployments. As key developers and maintainers of the vLLM project, and innovators in model quantization and sparsification techniques, we provide a robust platform for enterprises to build, optimize, and scale their LLM deployments. This role involves working closely with our product and research teams to develop state-of-the-art deep learning software, with a focus on model optimization algorithms. You will collaborate with our technical and research teams to create LLM training and deployment pipelines, implement model compression algorithms, and bring deep learning research into production. This position is ideal for individuals who thrive on bridging the gap between research and production, excel at optimizing large models, and are passionate about contributing to open-source AI tooling. Join us in shaping the future of AI! Please note that Red Hat will not provide visa sponsorship for this position. Candidates must have the legal right to work in the specified location without requiring future visa sponsorship.

Requirements

• Strong understanding of machine learning and deep learning fundamentals with experience in one or more of LLM Inference Optimizations and NLP.
• Experience with tensor math libraries such as PyTorch and NumPy.
• Strong programming skills with proven experience implementing Python-based machine learning solutions.
• Ability to develop and implement research ideas and algorithms.
• Experience with mathematical software, especially linear algebra.
• Understanding of Linear Algebra, Gradients, Probability, and Graph Theory.
• Strong communication skills with both technical and non-technical team members.
• BS or MS in computer science, computer engineering, or a related field.

Nice To Haves

• A PhD in a Machine Learning related domain is considered a strong plus.

Responsibilities

• Contribute to the design, development, and testing of various inference optimization algorithms within the LLM-compressor, Speculators, and vLLM projects.
• Design, implement, and optimize model compression pipelines utilizing techniques such as quantization and pruning.
• Develop and maintain speculative decoding frameworks to enhance inference speed while preserving model accuracy.
• Collaborate closely with research scientists to translate experimental ideas into robust, production-ready systems.
• Profile and optimize end-to-end LLM performance, focusing on memory usage, latency, and throughput.
• Benchmark, evaluate, and implement strategies for optimal performance on target hardware.
• Build tools to streamline model training, evaluation, and deployment processes.
• Participate in technical design discussions and propose innovative solutions to complex problems.
• Contribute to open-source projects, including code reviews and documentation, and collaborate with internal and external contributors.
• Mentor and guide team members, fostering a culture of continuous learning and innovation.
• Stay current with advancements in LLM architectures, inference optimizations, quantization research, and CPU/GPU hardware.

Benefits

• Red Hat offers a flexible work environment, allowing associates to work from in-office, office-flex, or fully remote settings depending on role requirements.
• Associates are encouraged to bring their best ideas, regardless of title or tenure.
• An open and inclusive environment that fosters innovation.
• Opportunities for professional development and continuous learning.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior