Machine Learning Engineer, Frontier Data Products

Recruiting From Scratch • San Francisco, NY

1d • Onsite

About The Position

Our client is one of the fastest-growing AI infrastructure companies, building the human intelligence platform powering the next generation of frontier AI models. Their platform enables leading AI labs and enterprises to access expert knowledge at scale, accelerating the development of advanced AI systems through high-quality human feedback and evaluation. As a profitable Series C company with a multi-billion-dollar valuation, they are building foundational infrastructure at the intersection of machine learning, human intelligence, and AI product development.

Requirements

• 3+ years of experience building and deploying production machine learning systems
• Strong software engineering skills with Python and production backend development
• Experience shipping ML systems that measurably improved product or business outcomes
• Strong understanding of evaluation methodologies, error analysis, and production model behavior
• Experience with LLM applications, retrieval systems, model evaluation, or AI-powered products
• Strong engineering fundamentals across the full ML lifecycle, from data through deployment
• Comfortable operating in ambiguous, fast-moving environments with evolving requirements
• Excellent communication and cross-functional collaboration skills

Nice To Haves

• Experience with LLM evaluation, RAG, prompt engineering, fine-tuning, or AI agents
• Experience building human-in-the-loop machine learning systems
• Experience with Temporal, AWS, PostgreSQL, LiteLLM, or distributed backend systems
• Familiarity with production monitoring, drift detection, and model observability
• Startup or high-growth engineering experience
• Strong product instincts and ability to balance ML sophistication with operational simplicity
• Experience working on AI infrastructure, developer platforms, or ML tooling

Responsibilities

• Design and build production machine learning systems that evaluate, validate, and improve AI-generated work
• Develop evaluation frameworks for complex tasks where labels are ambiguous, incomplete, or constantly evolving
• Build feedback loops that incorporate human review, disagreement resolution, and model corrections into continuous learning systems
• Own production model performance, including quality, latency, explainability, drift detection, and cost optimization
• Improve AI system performance using prompting, retrieval, fine-tuning, heuristics, active learning, and human-in-the-loop techniques
• Partner closely with backend engineers to integrate ML inference into scalable production workflows
• Debug and improve live production ML systems where model quality directly impacts customer outcomes
• Help define the architecture and long-term technical direction of AI evaluation infrastructure

Benefits

• Competitive base salary: $130,000 – $500,000
• Generous equity package
• Bi-annual performance bonus
• Relocation assistance
• Housing stipend for employees living near the office
• Monthly meal, wellness, and lifestyle stipends
• Comprehensive medical, dental, and vision coverage
• Opportunity to build core infrastructure used by the world's leading AI companies

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed