AI Engineer - Quality and Post-Training

Glean • Mountain View, CA

6d • $200,000 - $280,000 • Hybrid

About The Position

Glean is hiring an AI Engineer to improve the quality of its enterprise AI assistants, autonomous agents, models, and agent harnesses. The role sits at the intersection of applied research, quality, and product engineering, using evaluation results and production usage to identify and deliver measurable improvements.

The work includes improving the agent stack—including model routing, prompts, retrieval, and the agent harness—while balancing quality, latency, and cost. The engineer will post-train models for specialized enterprise use cases, build reinforcement-learning environments that simulate enterprise workflows, and improve evaluation sets, LLM judges, and agentic verifiers. The role also involves bringing emerging research into the product and collaborating with product, platform, and research teams to ship quality improvements.

The position is hybrid, with four days per week in Glean’s Mountain View or San Francisco office. The standard base salary range is $200,000–$280,000 annually. Compensation may also include variable compensation, equity, and benefits depending on the role and other factors. Glean offers medical, vision, and dental coverage, generous time off, a 401(k) plan, home-office improvement stipend, annual education and wellness stipends, company events, and daily healthy lunches.

Requirements

• At least 5 years of industry experience in AI or Machine Learning Engineering.
• Hands-on experience post-training LLMs, including adapting open-weight models to specialized tasks.
• Experience building evaluations for AI assistant and agent systems, including generating evaluation sets and building LLM-as-a-judge verifiers.
• Ability to calibrate offline and online signals and exercise good judgment about whether a metric reflects a quality improvement.
• Strong coding skills in Python and/or Go; TypeScript is a plus.
• Proficiency in an ML framework.
• A track record of shipping production machine-learning systems.
• Product-minded approach, including the ability to turn an ambiguous problem into a concrete experiment.
• Ability to thrive in a close-knit, cross-functional team.

Nice To Haves

• TypeScript coding skills.

Responsibilities

• Use evaluation results and production traces to improve the agent harness, model routing, prompts, and retrieval while balancing quality, latency, and cost.
• Post-train models using supervised fine-tuning and preference- or reward-based optimization methods such as DPO, PPO, and GRPO, including generating training data.
• Build reinforcement-learning environments for simulating enterprise workflows, including evaluation sets, harness support, and LLM-based verifiers for optimization and benchmarking.
• Improve representative evaluation sets, LLM judges, and agentic verifiers, and calibrate them against human graders so scores can inform release decisions and model training.
• Collaborate with product, platform, and research teams to deliver quality improvements in production.
• Bring emerging research into the product and use evaluation findings and production learnings to improve customer outcomes.

Benefits

• Medical coverage
• Vision coverage
• Dental coverage
• Generous time-off policy
• 401(k) plan
• Home-office improvement stipend
• Annual education stipend
• Annual wellness stipend
• Regular company events
• Daily healthy lunches
• Variable compensation and equity may be available for certain roles.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed