Lead Research Engineer – AI Evaluation & Benchmarking

Thomson Reuters•Eagan, MN

5d•Hybrid

About The Position

Thomson Reuters Labs is seeking a Lead Research Engineer to lead the design and delivery of production-ready software for AI model evaluation and benchmarking. The role builds scalable infrastructure that enables research and product teams to compare large language models, measure quality, cost, and latency, and make informed decisions about model selection and development. The position owns the engineering approach from early prototypes through reliable, maintainable tools. It works closely with research scientists, engineering teams, and product partners, helping shape technical direction and engineering delivery. The role also leads technical planning, translates research methods into software, ensures reproducible comparisons and production reliability, and mentors engineers. This posting is for proactive recruitment and may be used to fill current openings or future vacancies. The role uses a hybrid work model for office-based roles. Eligible locations include New York City, San Francisco, Los Angeles, Irvine, McLean, Washington, DC, and Ontario, Canada. The role may be eligible for an annual bonus based on enterprise and individual performance.

Requirements

8–10+ years of relevant software engineering experience, including building and maintaining production systems involving machine learning or complex data.
Strong hands-on Python skills, including API design, automated testing, and work with structured and unstructured data.
Technical leadership and end-to-end delivery experience, including architecture decisions, engineering planning, staffing and effort estimation, project execution, and adapting priorities as requirements change.
Experience collaborating with research scientists or machine learning practitioners to translate experiments and research methods into working software.
Experience integrating or testing machine learning models, interpreting evaluation results, and applying findings to engineering decisions.
Understanding of language models and ability to recognize unreliable comparisons or test-data leakage.
Cloud engineering experience using AWS, Azure, or a comparable platform, including distributed workflows, automated deployment, and monitoring.
Working knowledge of SQL and experience developing software that processes structured and unstructured data.
Strong collaboration, communication, problem-solving, and mentoring skills, with experience coordinating technical delivery across research, engineering, and product stakeholders.
Bachelor’s degree in Computer Science, Engineering, or a related field, or equivalent practical experience.

Nice To Haves

Experience building LLM evaluation and benchmarking pipelines, including automated scoring methods and human assessment workflows.
Familiarity with experiment tracking, dataset versioning, and reproducible evaluation processes.
Experience in natural language processing, information retrieval, or other applications involving complex text.
Experience reducing model inference or operating costs while maintaining evaluation quality and system performance.
Track record of developing reusable software libraries, frameworks, or tools adopted by other engineering teams.
Research contributions or open-source software contributions.

Responsibilities

Partner with program and research leaders to set priorities, break projects into delivery stages, estimate effort and cost, recommend staffing and skills, identify dependencies, and adapt plans to available resources.
Extend shared tools and services, design new components where needed, and develop production-quality Python software, APIs, and reusable evaluation frameworks.
Deliver initial versions, validate them with users, and improve them iteratively.
Collaborate with scientists to build automated evaluation pipelines connecting model APIs, test datasets, and agreed scoring methods.
Implement reliable workflows to compare LLMs and other AI models using datasets relevant to products and customer needs.
Maintain repeatable experiments, separate evaluation data from model development, and make results traceable to inputs and methods.
Measure model quality, operating cost, and latency to help product teams validate performance.
Develop user interfaces, documentation, evaluation reports, and diagnostic capabilities so teams can run evaluations, understand failures, interpret results, and extend tools.
Integrate automated testing, deployment, and monitoring; address performance, scalability, and operating costs; establish service maintenance and support ownership; and work with data specialists on access controls, permitted-use rules, and retention requirements.
Coordinate with foundational research, data engineering, platform engineering, and product teams to align interfaces, responsibilities, and delivery plans.
Resolve technical dependencies, mentor engineers, communicate trade-offs and delivery risks, and recommend changes to scope or technical approach when needed.

Benefits

Flexible vacation and flexible work arrangements, including work from anywhere for up to 8 weeks per year.
Two company-wide Mental Health Days off annually.
Access to the Headspace app.
Retirement savings plan with company match in the United States.
Tuition reimbursement.
Employee incentive programs and potential annual bonus.
Resources for mental, physical, and financial wellbeing, including an Employee Assistance Program and fitness reimbursement.
Health, dental, vision, disability, and life insurance programs in the United States.
Paid holidays, sick and safe paid time off, parental leave, and sabbatical leave.
Flexible Spending and Health Savings Accounts.
Two paid volunteer days off annually and opportunities for pro-bono consulting and ESG initiatives.
Adoption and surrogacy assistance.
Employee Stock Purchase Plan.
Commuter benefits.
Optional hospital, accident and sickness insurance; optional life and AD&D insurance; group legal and identity theft protection; and access to a 529 Plan.