Principle Machine Learning Systems Engineer
About The Position
Join Atlassian’s Rovo Chat Quality team within its Rovo + AI organization. The team builds evaluation systems, datasets, quality signals, and production ML capabilities to assess and improve Rovo. In this hands-on principal-level role, you will set technical direction across teams and advance models, agents, and evaluation systems to deliver measurable improvements for users. You will define rigorous ways to evaluate qualities such as helpfulness, correctness, judgment, and reliability; build scalable production ML and evaluation systems; improve agentic capabilities; and connect evaluation findings to product interventions and customer outcomes. The role also involves making high-impact technical decisions, leading cross-organizational initiatives, establishing reusable quality standards, and developing other engineers through technical leadership and mentorship. Atlassians can work from an office, from home, or a combination of the two. Atlassian can hire in countries where it has a legal entity. The role may be eligible for benefits, bonuses, commissions, and equity. U.S. base pay ranges for new hires are $236,700–$309,025 in Zone A, $213,030–$278,123 in Zone B, and $196,461–$256,491 in Zone C.
Requirements
- Track record of architecting and shipping large-scale ML or AI systems with measurable customer and business impact.
- Deep experience with LLM-powered products, agentic systems, model evaluation, experimentation, and dataset development.
- Strong production engineering skills and ability to remain hands-on in a modern ML stack.
- Principal-level technical leadership, including setting direction, influencing across teams, and leading complex initiatives from ambiguity to production.
- Excellent product and technical judgment, including balancing quality, speed, reliability, safety, latency, and cost.
- Ability to communicate complex ideas clearly and increase the effectiveness of engineers through mentorship and technical leadership.
Responsibilities
- Set the technical vision for measuring and improving Rovo Chat, translating qualities such as helpfulness, correctness, judgment, and reliability into evaluation strategies.
- Architect and build scalable production ML and evaluation systems, including datasets, simulations, automated judges, regression suites, experimentation platforms, and online quality signals.
- Advance Rovo’s agentic capabilities across reasoning, planning, tool selection and use, action correctness, retrieval, memory, and long-horizon execution.
- Identify systemic failure modes, develop and ship interventions, and determine whether they improve customer outcomes.
- Make technical decisions across model choice, prompting, fine-tuning, data strategy, system architecture, and inference while balancing quality, latency, reliability, safety, and cost.
- Lead cross-organizational initiatives with ML, engineering, product, analytics, research, and design partners.
- Establish reusable methods, systems, and quality standards for Rovo Chat and other AI-powered Atlassian experiences.
- Multiply other engineers’ impact through technical leadership, design reviews, mentorship, and long-term technical vision.
Benefits
- The role may be eligible for benefits, bonuses, commissions, and equity.
- Health and wellbeing resources.
- Paid volunteer days.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Principal
Education Level
No Education Listed
