Machine Learning & Data Operations Engineer

Lilly • Indianapolis, IN

2d • $151,500 - $244,200 • Hybrid

About The Position

As a Machine Learning & Data Operations Engineer on the TuneLab team, you will be instrumental in building cutting-edge ML and AI tools to accelerate and enhance Lilly’s drug discovery process. This role involves a hands-on approach across the entire lifecycle of models and their supporting data. Key responsibilities include transitioning trained models from research into reliable production environments, managing large-scale inference, and developing robust data pipelines and readiness checks to ensure the trustworthiness of the underlying data. You will establish validation, monitoring, and model-card review processes to ensure both models and data are production-ready, proactively identifying anomalies, schema drift, and performance regressions before they impact researchers. This position requires close collaboration with teams across Lilly Research Labs, AI, Software Engineering, Data Science, and IT Operations, as well as external collaborators, to integrate ML and computational tooling directly into researchers' daily workflows.

Requirements

• Ph.D. in Computer Science or a related computational field (e.g., Computational Science, Computational Biology, Bioinformatics, or a related quantitative computational discipline)
• Hands-on experience in software engineering and architecture, with a proven track record of delivering complex, cross-functional solutions
• Proficiency in a systems or object-oriented language (Go, Rust, Java, or C++) and a scripting language (Python and/or JavaScript)
• Hands-on experience deploying to containers, serverless, Kubernetes, and other hosting targets
• Experience deploying and serving machine learning models in production, including packaging, versioning, and promotion across environments
• Experience building data pipelines and working with relational and non-relational data stores (e.g., PostgreSQL, MySQL, MongoDB)
• Solid understanding of HTTP and RESTful APIs
• Experience using CI tools to automatically test and CD tools to automatically deploy updates, and applying test-driven development to prevent feature regression
• Experience applying systems-engineering concepts to distributed systems with high throughput and availability requirements

Nice To Haves

• Experience integrating AI/ML models into production with a focus on scalability, performance, and reliability (MLOps)
• Familiarity with MLOps and model-serving tooling (e.g., MLflow, Kubeflow, and model or artifact registries such as JFrog Artifactory)
• Experience with model validation, evaluation, and model-card and documentation practices for model governance
• Experience implementing data-quality, anomaly-detection, or schema-drift monitoring for production datasets
• Familiarity with streaming and CDC tooling (e.g., Kafka, Kafka Streams, Spark Streaming) and big-data processing (Spark)
• Familiarity with LLM application patterns—retrieval-augmented generation, tool-calling, and multi-agent orchestration—and with inference optimization
• Experience with infrastructure-as-code (Terraform), service mesh, and cloud-native monitoring and observability
• Exposure to drug discovery, life sciences, or healthcare data and workflows, including high-dimensional or biological datasets
• Experience contributing to federated or collaborative ML and data initiatives across organizations

Responsibilities

• Move trained models from research and experimentation into production, packaging, versioning, and promoting them across development, staging, and production environments and across cloud targets (AWS, Azure, GCP) and on-prem or hybrid infrastructure
• Build and operate scalable inference services and APIs—batch, real-time, and streaming—delivering low-latency, high-throughput serving that meets researcher and downstream-system needs
• Design and maintain model-serving infrastructure using containers and Kubernetes, with autoscaling, versioned rollouts (e.g., blue-green or canary), and rollback so updates ship without disrupting users
• Integrate models into researcher-facing tools and enterprise systems, ensuring seamless interoperability and data flow across platforms
• Design, build, and maintain scalable, secure data pipelines—batch, change-data-capture (CDC), and streaming—that move and transform data across the platform, including the embedding, vectorization, and feature pipelines that feed downstream ML and LLM applications
• Implement scalable storage and retrieval for large-scale structured and unstructured scientific data across cloud and on-prem or hybrid infrastructure
• Build and operate automated data-readiness and quality-monitoring workflows for high-dimensional scientific and enterprise datasets, including multi-method anomaly and outlier detection across numerical and categorical data
• Validate files for missing values, illegal characters, and structural issues, and build schema-drift detection with historical tracking and automated reporting—catching data-contract changes before they reach models and significantly reducing manual data QA
• Author, review, and validate model cards—verifying documented performance, intended use, limitations, data lineage, and evaluation results before models are promoted
• Run and automate model validation and evaluation—reproducing metrics, checking calibration and performance against acceptance criteria, and gating promotion on the results
• Implement production monitoring for model, data, and service health—latency, throughput, data and prediction drift, and quality—with alerting and proactive remediation
• Define acceptance criteria, audit trails, and reproducible checks; adjudicate flagged data and model issues with data owners and scientists; and track and report operational metrics
• Design and develop robust, scalable, and secure software solutions with a hands-on approach, from architecture through implementation
• Build and maintain microservices architectures and APIs (REST and GraphQL) that support model serving, data access, and tool-calling workflows
• Implement infrastructure-as-code and CI/CD pipelines to automatically test and deploy model, data, and service updates, applying test-driven development to catch regressions early
• Apply systems-engineering practices to distributed systems with high throughput and availability requirements, and troubleshoot complex issues across the model, data, and serving stack
• Collaborate within a team of engineers using best practices such as design reviews, code reviews, testing, and continuous integration and deployment
• Partner with Lilly Research Labs, Data Science, AI/ML, and IT Operations to translate research and business requirements into technical solutions
• Work with external, industry-leading collaborators to integrate models, data, and tooling into shared and federated workflows within Lilly’s controlled cloud environment
• Contribute to platform adoption through clear documentation, data dictionaries, runbooks, and support for internal end users

Benefits

• company bonus (depending, in part, on company and individual performance)
• company-sponsored 401(k)
• pension
• vacation benefits
• medical, dental, vision and prescription drug benefits
• flexible benefits (e.g., healthcare and/or dependent day care flexible spending accounts)
• life insurance and death benefits
• certain time off and leave of absence benefits
• well-being benefits (e.g., employee assistance program, fitness benefits, and employee clubs and activities)

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: Ph.D. or professional degree