Bask Health is seeking a hands-on Machine Learning Engineer, Evaluation & Reliability to help ensure that AI-powered features are accurate, dependable, and suitable for real-world use. The role focuses on developing evaluation infrastructure, testing workflows, and performance measurements for large language models, retrieval systems, AI assistants, and automated workflows. Work includes creating evaluation datasets, implementing benchmarks, measuring response quality, testing retrieval accuracy, analyzing failures, and monitoring production behavior. The engineer will collaborate with applied AI, machine learning, and backend engineers, product managers, and other stakeholders to define successful AI behavior and translate expectations into measurable tests and engineering improvements. Evaluation must account for accuracy, privacy, permissions, and the risks of unreliable responses in healthcare-related environments, including unsupported information, restricted-data exposure, unauthorized actions, and behavior outside approved boundaries. This position emphasizes practical implementation and continuous improvement. The engineer will contribute to evaluation methodologies and engineering standards, work with senior engineers on reliability strategies and complex technical decisions, and own assigned projects from implementation through deployment and ongoing improvement.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
Job Type
Full-time
Career Level
Mid Level
Education Level
No Education Listed