Machine Learning Engineer

University of California, San Francisco•San Francisco, CA

1d•Hybrid

About The Position

The Machine Learning Engineer will lead development, implementation, and maintenance of data pipelines and infrastructure supporting the deployment and continuous monitoring of machine learning and generative AI tools within UCSF’s APeX Enabled Research (AER) team. The position primarily manages and optimizes the data and monitoring pipelines of the Health IT Platform for Advanced Computing (HIPAC), a cloud infrastructure supporting AI/ML tools, including large language models, in the electronic health record. Work includes implementing new data integrations, enhancing HIPAC’s ETL capabilities, productionizing AI/ML tools developed by UCSF data scientists and researchers, and designing and implementing metrics to monitor tools deployed at UCSF Health. The role collaborates with UCSF technical teams, researchers, and data scientists on customized research solutions. The AER team supports the UCSF research community through consultation, grant support, budget estimation, project implementation, and support. Example projects include EHR-based interventions embedded in clinical trials, enabling researchers to use evidence-supported algorithms and digital tools, developing technical approaches and budgets for implementing tools in the electronic medical record, and building scalable, low-cost research infrastructure. This is a grant-funded, 18-month limited appointment. The position is flexible, with two days on-site, and requires availability for on-site lab meetings that may occur weekly or bi-weekly according to departmental needs. Certain employment terms, including pay and benefits, are subject to negotiation with the appropriate union.

Requirements

Bachelor’s degree in Computer Science, Computer Engineering, or a related area, and/or equivalent experience or training.
Six years of experience in positions of increasing responsibility designing, implementing, and maintaining complex AI/ML applications.
Experience with data analysis and machine learning tools such as Jupyter, Pandas, scikit-learn, NumPy/SciPy, and PyTorch.
Advanced knowledge of the full software development lifecycle.
Advanced Python skills, including writing clean, efficient, production-level code.
Advanced SQL experience, such as SQL Server or PostgreSQL.
Experience deploying, monitoring, and maintaining AI/ML models and pipelines.
Advanced experience with database systems, data warehousing solutions, and ETL pipelines.
Advanced experience designing, building, or maintaining data infrastructure for efficient ML model training and inference.
Experience with MLOps, DevOps, and CI/CD pipeline toolsets.
Experience developing complex automated testing.
Experience with cloud-based architecture in platforms such as AWS, GCP, or Azure.
Effective communication and interpersonal skills, including the ability to communicate technical information to technical and non-technical personnel at different organizational levels.
Ability to work independently and as part of a team, learn effectively, and meet deadlines.
Broad problem-solving skills and ability to interface regularly with management.
Excellent project leadership and management skills.
Deep understanding of Epic data models, specifically Clarity and Caboodle; candidates must have or be able to obtain Epic Clinical/Clarity data model certification shortly after onboarding.

Nice To Haves

Master’s degree or PhD in Computer Science, Computer Engineering, or a related area, and/or equivalent experience or training.
Experience with large language models and other generative AI technologies, especially deploying GenAI-based tools in production.
Familiarity with data visualization tools such as Tableau.
Experience with Epic data structures.
Epic Clarity certification.
Cloud development certifications.

Responsibilities

Develop, implement, and maintain data pipelines and infrastructure for deploying and continuously monitoring machine learning and generative AI tools.
Build and maintain data integrations with SQL databases or APIs and data processing and transformation pipelines for AI/ML tools.
Identify and build systems for implementation, monitoring, and maintenance of AI/ML tools; collaborate with researchers and developers to productionize and maintain tools in the health system.
Collaborate with data scientists and researchers to design and implement metrics and processes that automatically monitor AI/ML tools for safety, potential bias or drift, performance, and validity.
Plan, design, develop, modify, debug, deploy, and evaluate complex software; analyze existing software and devise algorithms for new systems.
Perform complex data analysis and software testing and debugging; initiate, analyze, design, and apply complex data sources; apply and enforce programming security practices.
Develop and execute complex test plans, conversion and system implementation plans, data modeling, performance and integration testing, and interfaces.
Determine source-code control techniques and configuration-management design and changes.
Prepare and approve or obtain approval for system and programming documentation; oversee changes to development, maintenance, and system standards; set technical requirements for complex software specifications.
Apply industry practices, community standards, and department policies and procedures; potentially serve as technical lead for projects or lead software development professionals.
Develop and maintain systems and metrics to monitor deployed AI/ML tools.