Senior Machine Learning Engineer - Firefly

Adobe•San Jose, CA

About The Position

Adobe Firefly – Applied Science & Machine Learning Generative AI is redefining creativity. As these systems scale to millions of users across image, video, and audio, the ability to learn from real-world usage becomes a critical differentiator. Building great generative models is only half the challenge. The other half is closing the loop: using production signals and user feedback to continuously measure, calibrate, and improve model behavior. The Adobe Firefly Applied Science & Machine Learning team builds the systems that ensure our generative models perform reliably and responsibly at scale. We are looking for a Senior Machine Learning Engineer to own and build systems that improve our deployed models based on user input over time. This role sits at the intersection of production ML infrastructure, preference modeling, and large-scale data systems. You will be responsible for turning noisy, real-world signals into actionable model and system improvements. What You'll Build Feedback-Driven Model Improvement Design and build production pipelines that ingest user feedback and behavioral signals to systematically identify where deployed models are underperforming. Develop preference models from pairwise data to calibrate model decision boundaries, improving output quality while managing trade-offs between precision and recall. Establish a data-driven iteration loop that connects production behavior to model updates, replacing manual tuning with continuous, measurable improvement. Production ML Systems Architect reliable, scalable data pipelines for feedback collection, processing, and signal extraction across multimodal generative workflows. Build evaluation and monitoring infrastructure that quantifies model performance in production, surfacing regressions and improvement opportunities. Design systems for extensibility as Firefly expands into new modalities and third-party integrations. What You'll Do Own the feedback systems initiative end to end, from problem scoping through system design, implementation, and production deployment. Collaborate closely with Applied Scientists and MLEs working on adjacent modeling initiatives to ensure feedback signals translate into concrete improvements. Work multi-functionally with engineering, product, and other collaborators to define feedback collection strategies and success metrics. Scope and prototype new applications of feedback data, including preference-based steering of generation quality and training data improvement, as the initiative matures and new high-value opportunities emerge . Contribute to the team's broader technical strategy around building adaptive, data-driven systems that improve with scale.

Requirements

MS or PhD in Computer Science, Machine Learning, or a related field.
5+ years of experience building and deploying ML systems in production, with clear end-to-end ownership.
Strong software engineering skills in Python and modern ML frameworks ( PyTorch ), with experience building data pipelines and production services.
Experience designing systems under real-world constraints: latency, throughput, data quality, and reliability.
Strong understanding of multimodal ML, including vision-language models and generative architectures (diffusion, autoregressive, multimodal transformers), with the ability to reason about model behavior across modalities and identify where feedback signals can drive meaningful improvement.
Hands-on experience with preference modeling, reward modeling, or RLHF-adjacent techniques, with an understanding of how to apply these in production settings where data is noisy and labels are imperfect.
Strong experimental design skills with the ability to define metrics, run rigorous evaluations, and draw reliable conclusions from ambiguous, real-world data.
Demonstrated ability to operate independently in ambiguous problem spaces, scoping work and defining technical direction with minimal guidance.
Experience making architectural decisions that balance short-term delivery with long-term system health.

Responsibilities

Design and build production pipelines that ingest user feedback and behavioral signals to systematically identify where deployed models are underperforming.
Develop preference models from pairwise data to calibrate model decision boundaries, improving output quality while managing trade-offs between precision and recall.
Establish a data-driven iteration loop that connects production behavior to model updates, replacing manual tuning with continuous, measurable improvement.
Architect reliable, scalable data pipelines for feedback collection, processing, and signal extraction across multimodal generative workflows.
Build evaluation and monitoring infrastructure that quantifies model performance in production, surfacing regressions and improvement opportunities.
Design systems for extensibility as Firefly expands into new modalities and third-party integrations.
Own the feedback systems initiative end to end, from problem scoping through system design, implementation, and production deployment.
Collaborate closely with Applied Scientists and MLEs working on adjacent modeling initiatives to ensure feedback signals translate into concrete improvements.
Work multi-functionally with engineering, product, and other collaborators to define feedback collection strategies and success metrics.
Scope and prototype new applications of feedback data, including preference-based steering of generation quality and training data improvement, as the initiative matures and new high-value opportunities emerge .
Contribute to the team's broader technical strategy around building adaptive, data-driven systems that improve with scale.