Data Scientist, Decision Quality

Sperry Rail•Shelton, CT

About The Position

Sperry's detection pipeline is part machine and part human. A neural network scans ultrasonic and electromagnetic test data collected from track and puts forward candidate defects. Trained analysts then review that output and decide what is real, what is not, and what gets sent to the railroad. Those decisions are the last judgment before a defect either reaches a customer or does not. Your job is to model that decision. Given what the analyst could see at the moment they made the call, was the disposition right, and where the pipeline gets it wrong, what actually caused it. We already capture the decision logs, so the data is there from your first week. This is a quality role rather than an automation role. The point is to make analysts better and to show us where our training, our tooling, and our detection thresholds are letting people down. The first phase is retrospective scoring of decisions already made. Where it goes after that depends in large part on what you find. You are one of the first three seats in a new US data science team, alongside a lead who owns risk analytics and an engineer who puts models into production. The work is internal-facing and it sits close to the operation, so you will spend real time with the people whose decisions you are modeling.

Requirements

Strong proficiency in Python (NumPy, Pandas, Scikit-learn, or similar) and SQL
Applied statistics: classification metrics, sampling, inter-rater agreement, bias, and experimental design
Supervised machine learning on imbalanced and imperfectly labeled data
Experience evaluating decisions or predictions against ground truth that arrives late, partially, or not at all
Ability to explain method and uncertainty to an operational audience and have them act on it
Familiarity with version control (Git), CI/CD pipelines, and agile development practices
Strong problem-solving skills and ability to learn new technologies quickly
Good communication skills - able to explain technical concepts to non-technical stakeholders
A collaborative, team-first mindset aligned with our values of being Humble, Hungry, and Smart
Qualifications and years of experience are indicative guidelines, not mandatory requirements. These criteria may be met through demonstrated competency or equivalent experience.

Nice To Haves

Bachelor's degree in statistics, computer science, engineering, or a related quantitative field; advanced degree welcome
Human-in-the-loop machine learning, expert review systems, or label quality and annotation quality work
Model evaluation and monitoring tooling
Human factors, quality management, or reliability engineering exposure
Signal or sensor data, particularly ultrasonics, induction, or eddy current
Experience in rail testing, NDT, or sensor-based inspection industries (ultrasound, eddy current, electromagnetic, etc.)

Responsibilities

Build and maintain retrospective scoring of analyst dispositions - given the data available at the time of review, how sound was the decision
Quantify agreement and variation across analysts, shifts, territories, and test conditions
Separate analyst-attributable outcomes from detection-threshold, data-quality, and volume effects, so that the organization acts on the right cause
Identify the contributing factors behind missed and misinterpreted defects, and express them as a taxonomy the analysis organization can use rather than as individual scorecards
Work with analysis leadership to turn findings into feedback loops: training content, review guidance, tooling changes, and threshold recommendations
Define and maintain the measures of decision quality that hold up over time, and be clear about their limits
Design sampling and review studies where the passive data cannot answer the question
Work with the detection and platform teams so the signals your models need are captured properly at source
Present findings to analysis leadership and to the wider engineering organization
Write clean, tested, well-documented code following engineering best practices
Participate in code reviews, sprint planning, and technical design discussions
Write and maintain documentation so the methodology is transferable rather than held tacitly