Machine Learning Engineer 4 - Manager IC

Capital One • San Jose, CA

3d • $197,300 - $245,600

About The Position

Capital One is seeking a Machine Learning Engineer to help drive AI transformations and scale production models. As part of the Enterprise ML Platform team, this role builds and evolves a high-throughput, multi-tenant training platform that supports workflows ranging from classical machine learning to GenAI fine-tuning and distributed computing with Ray, Spark, and Dask. The team owns the Kubeflow Pipelines v2 application layer on Amazon EKS and develops internal Python and Go SDKs, custom Kubernetes operators, and developer tooling for critical, regulated workloads. The role also integrates agentic AI into engineering lifecycles and contributes to upstream open-source Kubeflow, balancing feature delivery with platform stability and security.

The role spans machine learning engineering, operations, modeling, and data engineering. It involves designing and delivering models and components, informing infrastructure choices with knowledge of modeling methods, writing and testing application code, validating models, automating tests and deployments, and working with product, data science, and cross-functional Agile teams. Engineers retrain, maintain, and monitor production models; build cloud-based architectures and data pipelines; and apply deployment, governance, security, and Responsible and Explainable AI practices.

Requirements

• Bachelor's degree or higher in Computer Science, Machine Learning, or a related quantitative field such as Statistics, Economics, Operations Research, Analytics, Mathematics, or Engineering.
• At least 4 years of programming experience with Python, Java, Golang, or C++.
• At least 4 years of machine learning experience using industry-standard frameworks PyTorch or TensorFlow and libraries such as Pandas, NumPy, and Scikit-learn.
• At least 4 years of experience using and operating large-scale distributed systems such as Spark or Ray to prepare AI or machine learning data.
• At least 2 years of experience deploying and operating machine learning solutions in production, operating production services in a cloud environment (AWS, GCP, or Azure), and using Kubernetes to manage large-scale containerized ML software systems.

Nice To Haves

• Master's or doctoral degree in Computer Science, Electrical Engineering, Mathematics, or a related field.
• At least 3 years designing, training, optimizing, and evaluating modern ML models across architectures such as Transformers, CNNs, and RNNs and paradigms such as supervised, unsupervised, and reinforcement learning, including loss design, hyperparameter tuning, and diagnostic debugging.
• At least 3 years architecting and scaling distributed, end-to-end ML systems, from production-ready training and evaluation data pipelines to algorithm and infrastructure optimization.
• At least 3 years managing ML workloads using Docker, Kubernetes, and pipeline orchestration tools; Kubeflow is a strong plus.
• At least 3 years applying software best practices such as CI/CD, version control, automated testing, and code reviews, and deploying resilient systems with progressive rollout strategies, monitoring, alerting, and incident response planning.
• At least 1 year of experience as a technical lead developing ML solutions using industry best practices, patterns, and automation.
• Authored or co-authored a paper on an ML technique, model, or proof of concept.

Responsibilities

• Design, build, and deliver machine learning models and components that solve business problems in collaboration with Product and Data Science teams.
• Use knowledge of modeling techniques and issues—including model, data, and feature selection; training; hyperparameter tuning; dimensionality; bias and variance; and validation—to inform ML infrastructure decisions.
• Solve complex problems by writing and testing application code, developing and validating ML models, and automating tests and deployment.
• Collaborate on a cross-functional Agile team to create and enhance software for big data and machine learning applications.
• Retrain, maintain, and monitor models in production.
• Leverage or build cloud-based architectures, technologies, and platforms to deliver optimized ML models at scale.
• Construct optimized data pipelines to feed ML models.
• Apply continuous integration and continuous deployment practices, including test automation and monitoring, to deploy ML models and application code successfully.
• Manage code to reduce vulnerabilities, govern models from a risk perspective, and apply Responsible and Explainable AI practices.
• Use programming languages such as Python, Scala, or Java.
• Build and evolve the Enterprise ML Platform's high-throughput, multi-tenant training platform.
• Develop internal Python and Go SDKs, custom Kubernetes operators, and developer tooling for regulated workloads.
• Integrate agentic AI into engineering lifecycles and contribute to upstream open-source Kubeflow.

Benefits

• Comprehensive, competitive, and inclusive health, financial, and other benefits supporting total well-being.
• Eligible for performance-based incentive compensation, which may include cash bonuses and/or long-term incentives.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior