Senior AI Data and Evaluation Engineer

Stryker•Menlo Park, CA

20h•$116,000 - $193,000•Hybrid

About The Position

We are looking for an experienced and highly skilled Senior AI Data and Validation Engineer. A successful candidate will be responsible for both dry and wet lab experiments for AI functionality acquiring datasets, developing labeling guidelines, and preparing dataset for AI model development. Also, candidate will be responsible for developing methods and processes for scientific evaluation of AI/ML models used in medical devices, ensuring compliance with all applicable FDA guidelines and regulations. This role is crucial for maintaining the safety, effectiveness, and quality of AI-enabled medical devices throughout their product lifecycle from initial training to final validation. The engineer will contribute to the design, execution, and analysis of validation studies, ensuring that the devices perform predictably and reliably for their intended use and across all relevant demographic groups., placing a strong emphasis on comprehensive data management practices, including data collection, processing, annotation, storage, and use. This includes ensuring data integrity, quality, and representativeness to mitigate potential biases and demonstrate the safety, effectiveness, and quality of AI-enabled medical devices throughout their lifecycle.

Requirements

Bachelor's Degree in Computer Science, Machine Learning, Electrical Engineering, Mathematics, Statistics, Bioengineering or related field, 2+ years of work experience required.
OR Master's Degree in the above fields

Nice To Haves

2+ years of experience in medical device R&D, preferably AI-enabled devices.
Fluent in commercial data analysis tools or similar scripting and statistical language for data analysis, Python preferred.
Experience in the development, evaluation and validation of AI models, specifically within the medical device or healthcare industry. This experience including one or more of the following R&D life cycle: Data Acquisition and preprocessing and labeling, model training and tuning, AI Model evaluation and performance, bias detection and mitigation, risk assessment, and QMS documentation
Experience in Statistics and descriptive data analysis to help develop data-driven scientific rigor to AI model validation process, including experience with one or more of the following: confidence intervals development, power analysis, statistical tests, etc.
Entry level knowledge with AI/ML frameworks such as PyTorch, OpenCV, TensorFlow, scikit-learn etc. for model training and development, testing and validation.
Experience with commercial Data Operation tools
General knowledge of the healthcare market and competitors.

Responsibilities

Perform dataset collection, develop, synthesis, generation, preparing datasets for AI development, stress testing and validation.
Execute web lab experiments that interact with human blood samples with good laboratory practice.
Create detailed validation plans, protocols, and reports for AI-enabled medical device software functions encompassing both performance validation and model evaluation.
Perform Data Operation such as management of data used in both the development and validation of AI models, ensuring the quality, diversity, and independence of datasets . This involves assessing data representativeness, addressing potential biases, and ensuring the appropriate separation of training and test datasets
Conduct comprehensive risk assessment for AI models, considering potential and unintended biases, limitations, and cybersecurity vulnerabilities
Conduct rigorous performance validation studies, analyzing performance across different subgroups of the intended user population .
Perform analysis to address transparency and bias throughout the device lifecycle, from design to post-market surveillance . This includes evaluating whether a device benefits all relevant demographic groups similarly and incorporating transparency into device design
Stay abreast of the latest FDA guidelines, regulations, and industry best practices related to AI model validation in medical devices.