University of Chicago-posted 17 days ago
$100,000 - $140,000/Yr
Full-time • Manager
Hyde Park, IL
5,001-10,000 employees

The TMW Center for Early Learning + Public Health (TMW Center) develops, tests, and implements evidence-based interventions designed to promote very young children’s cognitive and social-emotional development, with emerging technologies that enhance—rather than replace—the pivotal role that caregivers play in building healthy young brains. Although there is a rich body of research demonstrating the importance of the early language environment for maximizing early learning, there are very ways to assess the quality of those environments. As a result, when it comes to nurturing their children’s brains, parents often feel like they are in a maze without a map. The TMW Center has launched a wearable device and an accompanying app that uses machine learning to measure and analyze a child’s language environment and provide real-time information as well as personalized feedback and guidance for enhancing that environment. This groundbreaking piece of technology gives parents and caregivers with information they need to engage in robust brain-building interactions — and helps deepen caregiver-child connection. The TMW Center is looking for an ML Validation & Data Operations Manager to lead and monitor operations for continuous algorithm improvement and validation. This will involve guiding the development of data management systems and establishing data assessment strategies to enable efficient and scalable validation processes. The ML Validation & Data Operations Manager will also contribute to defining the vision for current and longer-term algorithm developments. They will collaborate closely with our data science, research, and data management teams to coordinate workstreams, ensure a smooth integration of the developed systems and protocols within broader operations, and align with research developments. This position will report to the CTO, with guidance from the Scientific Director.

  • Leads data labeling effort to build ground-truth corpus for existing and future algorithms, including identifying data requirements and protocols.
  • Determines validation criteria and metrics across models and settings/partnerships/use cases.
  • Collaborates with data management teams, and application development teams to identify and capture the data necess­ary to perform validation.
  • Recruits, trains, and leads a team of data labelers.
  • Ensures alignment between the validation roadmap and Center’s priorities.
  • Establishes timelines and strategies for the validation of different algorithms.
  • Works with ML and engineering teams to develop and manage pipelines for continuous algorithm validation and optimization.
  • Promotes advances in, and creative ML solutions for validation and data management enhancement (e.g., automation of protocols and training pipelines).
  • Builds quality assurance processes to continuously assess reliability of data.
  • Maintains comprehensive records of data sources, methodologies, and results.
  • Establishes best practice-based processes related to reproducibility, documentation, and version control.
  • Designs new systems, features, and tools.
  • Solves complex problems and identifies opportunities for technical improvement and performance optimization.
  • Reviews and tests code to ensure appropriate standards are met.
  • Utilizes technical knowledge of existing and emerging technologies, including public cloud offerings from Amazon Web Services, Microsoft Azure, and Google Cloud.
  • Acts as a technical consultant and resource for faculty research, teaching, and/or administrative projects.
  • Performs other related work as needed.
  • Minimum requirements include a college or university degree in related field.
  • Minimum requirements include knowledge and skills developed through 5-7 years of work experience in a related job discipline.
  • Degree in economics, data science, data analytics, computer science, software engineering or related fields strongly preferred.
  • At least two years of experience managing people.
  • Prior experience setting up data labeling and validation processes.
  • Demonstrated understanding of data science management, Machine Learning and Data operations.
  • Experience building out ML operations teams and processes.
  • Experience with statistical modeling and programming.
  • Demonstrated ability to work independently with little supervision.
  • Excellent strategic planning and execution skills.
  • Strong problem-solving skills.
  • Ability to balance short-term, long-term, and big picture objectives.
  • The University of Chicago offers a wide range of benefits programs and resources for eligible employees, including health, retirement, and paid time off. Information about the benefit offerings can be found in the Benefits Guidebook .
© 2024 Teal Labs, Inc
Privacy PolicyTerms of Service