ML Engineer

Noetik

22h•Hybrid

About The Position

At Noetik, you will work alongside a highly technical, interdisciplinary team tackling hard and impactful problems. We are looking for people who are excited to engage deeply and bring a strong sense of urgency to making an impact. In this role, you’ll work with a high-caliber team building systems and conducting research at the forefront of AI. At Noetik, AI is at the heart of everything we do: we generate our own data purpose-built for machine learning, build systems for advanced machine learning research, train large foundation models from scratch, and center our drug discovery work on these models. Here, AI is not a buzzword. It is the core of our company and the key to deciphering complex biology to make a difference for patients.

Requirements

Strong software engineering fundamentals and experience building and operating production machine learning systems.
Hands-on experience with distributed model training in a production or large-scale research environment.
Strong understanding of multi-GPU and distributed training concepts, including the performance and reliability challenges involved in scaling ML workloads across multiple machines.
Experience working with PyTorch or a similar deep learning framework, with the ability to read, profile, debug, and modify model training and inference code.
Experience identifying and resolving performance bottlenecks across ML workloads, including GPU utilization, data pipelines, memory, distributed communication, and infrastructure.
Comfort working directly with cloud and compute infrastructure, including containers, Kubernetes, GPU systems, storage, networking, and distributed compute.
Experience building tooling or platforms that improve the developer experience of machine learning researchers or engineers.
Ability to debug complex problems spanning model code, distributed frameworks, infrastructure, and underlying compute resources.
Ability to balance reliability and standardization with the speed and flexibility required in a fast-moving research environment.
Comfort using AI coding tools and agents as a core part of the software development process while maintaining a high bar for correctness, maintainability, and understanding of the systems being built.
Strong collaboration skills and the ability to work closely with ML researchers to understand their workflows, identify friction, and build systems that make them more effective.
A collaborative, low-ego approach, with a focus on solving problems as a team and finding the best solution rather than being right.
Typically, several years of experience working with production ML systems, distributed training, or ML infrastructure. We care more about demonstrated depth and impact than a specific number of years.

Nice To Haves

Experience with Ray, Kubernetes, Kueue, MLflow, Terraform, or similar ML infrastructure and orchestration technologies.
Experience with high-performance GPU networking and distributed communication technologies such as NCCL or EFA.
Experience optimizing large-scale model training for throughput, memory efficiency, checkpointing, or compute utilization.
Experience building large-scale batch inference systems or online model-serving infrastructure.
Experience operating GPU clusters in AWS or another major cloud environment.
Experience with modern distributed training approaches such as FSDP, DeepSpeed, or similar frameworks.
Experience with profiling tools, CUDA-level performance debugging, or other low-level GPU optimization techniques.

Responsibilities

Design, build, and evolve systems for distributed model training and batch inference as Noetik’s workloads grow in scale and complexity.
Develop workflows for experiment tracking, model evaluation, and model registry management that support reproducibility across the ML lifecycle.
Profile and optimize training and inference performance across PyTorch model code, data loading, GPU utilization, memory usage, distributed communication, and checkpointing.
Build researcher-facing CLIs, SDKs, and job submission tools that make it easy to move from experimentation to large-scale runs with minimal infrastructure knowledge.
Develop and operate reliable, observable, and cost-efficient GPU infrastructure using technologies such as Kubernetes, AWS, Ray, Kueue, Docker, Terraform, and high-performance networking.
Work directly with ML researchers to troubleshoot issues, understand evolving research needs, and translate them into platform improvements that support rapid iteration.
Use modern AI development tools and agents throughout the engineering workflow to accelerate implementation, debugging, profiling, testing, exploration, and maintenance.

Benefits

Generous health, dental, and vision insurance, with premiums fully covered for both employees and their dependents (depending on your plan choice).
Comprehensive and accessible coverage, featuring modern care platforms like OneMedical, Midi Health, Spring Health, and XP Health.
HSA and FSA accounts.
Dependent care savings.
EAP.
Resources for mental health (Rula).
Health care navigation and advocacy (Rightway).
Flexible PTO.
Nine paid holidays.
Synchronized breaks: two full-week company breaks (the first week of July and the last week of December), plus the Friday after Thanksgiving.
Competitive base salary.
Stock options.
401(k) plan with an employer match.
Fully-covered disability, life and AD&D insurance.
Generous parental leave.
SoFi.
Multiply Mortgage.
Optional programs for pet insurance, legal services, and identity theft protection.