Senior Machine Learning Research Engineer I/ II, Open-Endedness

Lila Sciences•San Francisco, CA

4d•$148,000 - $240,000

About The Position

We’re seeking a Machine Learning Research Engineer for the Open-Endedness Team with expertise in large model training and optimizing novel algorithms for best results in distributed ML infrastructure. You’ll design and maintain large-scale training systems, optimize performance for large models, and integrate cutting-edge techniques to improve efficiency and throughput. Open-Endedness is an emerging area of machine learning that aims to automate never-ending innovative processes of discovery and exploration. The Open-Endedness Team, led by Ken Stanley, investigates in particular how a continual chain of deep transformative creativity can be maintained that far exceeds the derivative creativity seen in current models. In effect, the systems developed on this team will go beyond simply solving problems posed by users, to conceiving the future unimagined directions of science itself.

Requirements

Proven experience with distributed ML training frameworks (Megatron-LM, TorchTitan, DeepSpeed, Ray).
Strong software engineering skills (Python, C++ kernel contributions are a plus).
Understanding of large-scale model training techniques.
Experience with cloud or HPC environments.

Nice To Haves

Prior work with scientific datasets or domain-specific modeling.
Contributions to open-source ML frameworks.

Responsibilities

Distributed training infrastructure for LLMs and multi-modal models.
Performance optimizations for large-scale model training including training and optimization workflows (SFT, RL, long-context, etc.).
Orchestrate frontier and open source LLMs along with complex compute-intensive tool use
Scalable pipelines for data preprocessing and experiment orchestration, including tools for efficient data loading, pipeline parallelism, and optimizer tuning.
System-level performance benchmarks and debugging utilities.