Principal Machine Learning Engineer, Evaluation

Autodesk • Vancouver, BC

33d • Hybrid

About The Position

The work we do at Autodesk touches nearly every person on the planet. By creating software tools for making buildings, machines, and even the latest movies, we influence and empower some of the most creative people in the world. As a Principal Machine Learning Engineer focused on evaluating ML models for CAD and BIM, you will ensure ML-powered experiences meet high standards for quality, reliability, and customer impact across Autodesk products. You will work hands-on with ML-enabled Autodesk software to design evaluation datasets, metrics, and protocols that reflect real user workflows. You will use those signals to guide release readiness and continuous improvement in production. You will communicate results through clear evaluation summaries and recommendations, partnering with researchers, engineers, and product teams to close the gap between offline performance and real customer outcomes. You will report to a manager in the Model Delivery team within Autodesk Research. This role is based in proximity to our North American Autodesk offices, with a preference for Toronto or Vancouver. We support both in-person, hybrid, and remote work.

Requirements

• BS or MS in Mechanical Engineering, Architecture, Computer Engineering, Computer Science, Applied Math, Statistics, or equivalent industry experience
• 4+ years of professional experience in ML model evaluation, ML enabled QA, or applied ML engineering, including designing evaluation datasets, metrics, and protocols
• Strong software engineering skills for building repeatable evaluation systems, including Python, data pipelines and analysis, testable and maintainable code, version control, and cloud based workflows (for example AWS or Azure)
• Strong written communication skills for documenting evaluation methods, results, and recommendations

Nice To Haves

• Familiarity with design, manufacturing, or AEC workflows, including hands-on experience with CAD/BIM tools such as Fusion, AutoCAD, or Revit
• Experience with geometry or design data representations, including 2D and 3D
• Familiarity with ML frameworks and tooling (for example PyTorch, Ray, or similar)

Responsibilities

• Design and run evaluation protocols (datasets, metrics, statistical analysis) that reflect real CAD/BIM user workflows and production conditions
• Build repeatable evaluation tooling and automation to support model development, regression testing, and release readiness decisions
• Curate, process, and analyze data from multiple sources (including production derived samples) to assess model behavior and customer outcomes
• Validate end-to-end ML powered product experiences and translate product requirements into measurable evaluation criteria
• Communicate findings and recommendations clearly to researchers, engineers, product teams, and leadership
• Document model quality issues discovered in production or during validation (accuracy, robustness, latency, failure modes) and help prioritize follow-up investigations

What This Job Offers

• Job Type: Full-time
• Career Level: Principal
• Number of Employees: 5,001-10,000 employees