ML Scientist (ML & LLM Systems)

Explorance•Montreal, QC

16d

About The Position

Join a Montreal headquartered company that helps organizations around the world create a personalized journey of impact and fulfillment for their people. Explorance offers innovative Feedback Analytics solutions because we believe that each experience matters. We’re looking for an Applied ML Scientist to help us design, evaluate, and productionize machine learning and LLM-based systems that power decision-making for millions of users globally. This role sits at the intersection of research and product, focusing on experimentation, evaluation frameworks, and real-world performance—not just model building. Role Description: Designing and running experiments to evaluate ML and LLM approaches. Defining metrics, datasets, and acceptance criteria for proposed features. Analyzing errors, edge cases, and cost–quality trade-offs. Collaborating with engineers to translate findings into implementation-ready specifications. Reviewing data quality, labeling strategies, and dataset gaps. Documenting conclusions, recommendations, and decision rationales. Advising on whether LLMs, traditional ML, or simpler solutions are most appropriate for a given problem.

Requirements

Strong experience in applied machine learning in production-oriented environments
Hands-on experience with Python and PyTorch or similar ML frameworks
Experience designing and running structured ML and LLM (local and third party) experiments
Ability to define and interpret quantitative evaluation metrics
Strong analytical skills for error analysis and model comparison
Ability to clearly document findings and decision rationales
Experience working with both traditional ML models and LLM-based systems
Familiarity with NLP tasks such as classification, extraction, or summarization
Experience working with labeled datasets and iterative data improvement
Exposure to relational databases (e.g., Postgres, MSSQL) for experiment data

Nice To Haves

Familiarity with monitoring or evaluation dashboards (e.g., Grafana)
Experience collaborating with platform or infrastructure teams

Responsibilities

Design and execute applied research experiments across ML systems, including:
Traditional ML models (e.g., classification, scoring, ranking)
LLM-based approaches where appropriate
Define evaluation methodologies, metrics, datasets, and acceptance criteria for ML-driven features
Assess trade-offs between model performance, interpretability, cost, and operational complexity
Analyze errors, edge cases, and ambiguous inputs; propose concrete mitigation strategies
Contribute to existing ML systems by supporting:
Data strategy and labeling approaches
Model training, validation, and evaluation
Evaluate when LLMs are appropriate versus when standard ML approaches are sufficient
Produce clear research outputs, including:
Written conclusions and recommendations
Implementation-ready specifications for engineering teams
Collaborate closely with ML Software Engineers and the Applied Research Team Lead to ensure smooth handoff from research to production