Remote | Machine Learning Research Engineer — $55–$85/hour

24-Mag•New York, NY

4d•Remote

About The Position

We are sharing a specialised full-time consulting opportunity for machine learning engineers and research practitioners with hands-on experience training, evaluating, and experimenting with ML models end to end. This role supports the development of advanced agentic evaluation benchmarks for frontier AI systems. Selected professionals will transform real machine learning research ideas into rigorous multi-step tasks, implement and run experiments, analyse training behaviour, and evaluate where model-generated solutions fall short of technically correct results.

Requirements

At least 1 year of experience in machine learning research, research engineering, or a comparable technical role
Hands-on experience training and evaluating ML models through complete experimental workflows
Strong understanding of experiment setup, execution, analysis, and reproducibility
Familiarity with large language model capabilities, limitations, and evaluation techniques
Working proficiency in Python and Git
Comfort using both scripting and notebook-based environments
Strong technical writing, analytical reasoning, and attention to detail
Ability to work independently through ambiguous, open-ended research problems
Reliable availability for approximately 35 hours per week

Nice To Haves

Understanding of reinforcement learning concepts, including reward functions and policy training
Experience in AI training, model evaluation, or benchmark development
Background authoring technical tasks, reference solutions, or grading rubrics
Familiarity with agentic AI systems and multi-step model evaluations
Experience diagnosing model-training failures or unexpected experimental behaviour
Knowledge of experimental design, ablation studies, and performance comparison
Experience reviewing code, notebooks, or research analyses prepared by other practitioners
Familiarity with reproducible ML environments and collaborative Git workflows

Responsibilities

Turn practical ML research ideas into well-defined, multi-step evaluation tasks
Develop assignments involving model training, experimental modifications, and performance analysis
Define clear technical requirements, expected outputs, and success criteria
Ensure tasks assess genuine implementation and experimental reasoning rather than superficial library usage
Implement reference solutions using Python, scripts, and notebook environments
Configure and run model-training experiments from setup through final evaluation
Modify model components, training procedures, reward functions, or experimental parameters
Validate code, dependencies, datasets, intermediate outputs, and final results
Document complete workflows so experiments can be reproduced independently
Review how frontier AI models approach complex machine learning tasks
Assess implementation quality, experimental methodology, and technical conclusions
Identify coding errors, unsupported assumptions, weak experimental controls, and misleading interpretations
Determine whether reported improvements are observed by the results
Explain clearly where and why a model-generated solution fails
Develop selected tasks involving reinforcement learning fundamentals
Evaluate reward-function changes, policy-training behaviour, and experimental outcomes
Assess whether proposed modifications produce the intended training effect
Identify instability, unintended incentives, or incorrect interpretations of RL results
Work closely with researchers, task authors, and fellow machine learning specialists
Compare evaluation decisions to maintain consistent and rigorous benchmark standards
Refine task instructions, reference solutions, and grading criteria based on testing outcomes
Share recurring model failure patterns and opportunities for stronger benchmark coverage