Senior Software Development Engineer in Test, ML

PlayStation•Madison, WI

4d•Hybrid

About The Position

Sony Interactive Entertainment (SIE), the company behind the PlayStation brand, is seeking a Senior Software Development Engineer in Test, ML to join its Madison team. The role leads quality efforts for machine learning-powered products and services, partnering with ML engineers, software engineers, product managers, and data partners to develop scalable test automation and validation for systems serving PlayStation globally. The position focuses on building confidence in model reliability, production readiness, and behavioral consistency. Testing probabilistic systems involves evaluating score distributions, cohort-level behavior, thresholds, and statistical signals, rather than relying only on deterministic pass/fail assertions. The role is located in Madison, Wisconsin, or Waterloo, Ontario, Canada. The estimated base pay range is $152,000–$190,000 USD. The position is eligible for SIE’s benefits package, including medical, dental, vision, matching 401(k), paid time off, a wellness program, Sony product discounts, and potentially a bonus package. Employment is subject to background checks at the offer stage.

Requirements

Bachelor’s degree in Computer Science or a related field.
At least 5 years of experience as an SDET, with strong experience testing backend systems.
Proficiency in Python, Java, Scala, or Go for test automation.
Hands-on experience with API testing tools such as Postman or similar tools for distributed systems.
Familiarity with CI/CD pipelines and test execution in Jenkins or similar environments, such as GitHub Actions or ArgoCD.
Hands-on experience building and maintaining automated test frameworks such as pytest, JUnit, or TestNG.
Experience working with databases for backend validation.
Experience with cloud and container technologies such as AWS, GCP, Kubernetes, or Docker.
Familiarity with monitoring and observability tools such as Prometheus, Grafana, or Datadog for validating service health.
Strong understanding of the software development lifecycle and agile methodologies.
Excellent problem-solving and communication skills.

Nice To Haves

Experience testing machine learning systems, such as recommendation systems, search ranking systems, or other probabilistic ML applications.
Experience creating synthetic or seeded test data to simulate realistic customer or account behaviors.
Experience evaluating model outputs using statistical techniques, distribution analysis, or scenario-based validation rather than deterministic assertions.
Knowledge of data pipelines, feature stores, inference systems, or model-serving infrastructure such as Seldon, KServe, or Ray Serve.
Experience with performance and load testing tools.
Experience with Databricks or similar ML platform tooling.
Experience architecting and optimizing CI/CD pipelines, quality gates, and release validation systems using Jenkins or similar platforms.

Responsibilities

Drive architecture-level automation solutions and scalable quality engineering practices across multiple teams and product areas.
Design, develop, and maintain automated test frameworks and scripts for ML inference services and related ML platform components using Python or Java.
Build and automate realistic, representative test account creation and datasets to evaluate model behavior across scenarios.
Validate score distributions, ranking behavior, output quality, and other model signals for incoming ML models before release.
Perform functional, integration, regression, API, performance, and load testing of RESTful APIs, microservices, databases, and inference services.
Verify inference service scalability, latency requirements, and reliability under production-like conditions.
Work with ML engineering and operations teams to ensure features and bug fixes have automated test coverage and support continuous integration and deployment.
Debug and analyze test failures and model anomalies, and report defects.
Drive high standards of quality for engineering decisions and customer-facing features through effective test and automation strategies.
Develop and implement scalable, maintainable test automation best practices.
Monitor test execution and optimize performance in test environments.
Develop innovative solutions to organizational problems.