Member of Technical Staff, Applied Research

Embedding VC • New York, NY

4d

About The Position

Chakra Labs builds high-fidelity environments, evaluations, and datasets for frontier AI research. This role focuses on post-training, agent environments, data quality, and research infrastructure, working with frontier AI labs to improve models through measurable, useful methods.

The role involves designing and running model-improvement workflows, including supervised fine-tuning, preference optimization, and reinforcement learning approaches such as GRPO. It also includes designing realistic agent environments and tasks, creating high-quality training trajectories, and developing reward functions, rubrics, validators, and evaluation tools. The position works across distributed GPU clusters and uses PyTorch and FSDP to support training, evaluation, and data generation at scale.

The work is research-facing and production-minded, with ownership of whole problems and collaboration with AI researchers and labs to translate ambiguous research goals into concrete environments, datasets, experiments, and deliverables. The role requires comfort with ambiguity, sound engineering judgment, and the ability to balance rapid iteration with rigorous, reliable results.

Requirements

• Master’s- or Ph.D.-level knowledge of machine learning fundamentals, including linear algebra, optimization, and stochastic gradient descent.
• Ability to reason from first principles when a model or training run behaves unexpectedly.
• Experience with or deep understanding of LLM fine-tuning, preference data, reward modeling, or reinforcement learning for language models.
• Strong Python skills and comfort building with PyTorch, FastAPI, and modern ML infrastructure.
• Ability to work with lower-level primitives when higher-level abstractions are insufficient.
• Strong engineering judgment and ability to move between research ambiguity and production constraints.
• Ability to determine when to iterate quickly, when to be rigorous, and when a result is too fragile to trust.
• Comfort working in an ambiguous, rapidly changing area without a fixed playbook.

Nice To Haves

• Experience designing agent environments, multi-turn tool use, sandboxed tasks, or evaluation harnesses.
• Approximately 3–5 years of experience; the posting notes that more or less experience can also be suitable.

Responsibilities

• Design and run model-improvement workflows across supervised fine-tuning, preference optimization, and reinforcement learning approaches such as GRPO.
• Determine which training signals matter, how to collect them, and whether models have actually improved.
• Design agent environments, tasks, tools, validators, reward signals, and evaluation harnesses that test meaningful capabilities.
• Create, inspect, and improve high-fidelity training trajectories, with attention to taste, correctness, edge cases, and learning value.
• Develop reward functions, rubrics, validators, and analysis tools to make model behavior useful training signal and improve measurement robustness.
• Run experiments across distributed GPU clusters and build infrastructure for model training, evaluation, and data generation at scale.
• Translate frontier AI labs’ ambiguous research goals into concrete environments, datasets, experiments, and deliverables.
• Own end-to-end research and engineering problems, including environment design, training-run debugging, reward improvement, and evaluation pipeline scaling.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level