Senior Software Development Engineer in Test, ML

PlayStation Global•Madison, WI

5d•$152,000 - $190,000

About The Position

Sony Interactive Entertainment is seeking a Senior Software Development Engineer in Test, ML to join its Madison team and lead quality efforts for machine learning-powered products and services. The role partners with ML engineers, software engineers, product managers, and data partners to build scalable test automation and validation approaches for systems serving PlayStation at global scale. It focuses on confidence in model reliability, production readiness, and behavioral consistency, including validation of probabilistic systems through score distributions, cohort-level behavior, thresholds, and statistical signals rather than only deterministic pass/fail assertions. The position develops architecture-level automation solutions and quality engineering practices across teams and product areas. It builds and maintains test frameworks and test scripts for ML inference services and platform components, creates representative test accounts and datasets, and validates model outputs and signals before release. Testing spans functional, integration, regression, API, performance, and load testing. The role also works with ML engineering and operations teams on automated coverage and CI/CD, investigates failures and model anomalies, improves test practices and environment performance, and contributes innovative solutions to organizational problems. The position is listed for Madison, Wisconsin, and Waterloo, Ontario, Canada. The posting describes SIE’s hybrid working policy but does not specify the exact workplace arrangement for this role. The estimated base pay range is $152,000–$190,000 USD. The role is eligible for SIE’s benefits package and may also be eligible for a bonus package.

Requirements

Bachelor’s degree in Computer Science or a related field.
At least 5 years of experience as an SDET, with strong experience testing backend systems.
Proficiency in Python, Java, Scala, or Go for test automation.
Hands-on experience with API testing tools such as Postman or similar tools for distributed systems.
Familiarity with CI/CD pipelines and test execution in Jenkins or similar environments, such as GitHub Actions or ArgoCD.
Hands-on experience building and maintaining automated test frameworks such as pytest, JUnit, or TestNG.
Experience working with databases for backend validation.
Experience with cloud and container technologies such as AWS, GCP, Kubernetes, or Docker.
Familiarity with monitoring and observability tools such as Prometheus, Grafana, or Datadog for validating service health.
Strong understanding of the software development lifecycle and agile methodologies.
Strong problem-solving and communication skills.

Nice To Haves

Experience testing machine learning systems, such as recommendation systems, search ranking systems, or other probabilistic ML applications.
Experience creating synthetic or seeded test data to simulate realistic customer or account behaviors.
Experience evaluating model outputs using statistical techniques, distribution analysis, or scenario-based validation rather than deterministic assertions.
Knowledge of data pipelines, feature stores, inference systems, or model-serving infrastructure such as Seldon, KServe, or Ray Serve.
Experience with performance and load testing tools.
Experience with Databricks or similar ML platform tooling.
Experience architecting and optimizing CI/CD pipelines, quality gates, and release validation systems using Jenkins or similar platforms.

Responsibilities

Drive architecture-level automation solutions and scalable quality engineering practices across multiple teams and product areas.
Design, develop, and maintain automated test frameworks and test scripts for ML inference services and related ML platform components using Python or Java.
Build and automate realistic, representative test account creations and datasets to evaluate model behavior across different scenarios.
Validate score distributions, ranking behavior, output quality, and other model signals for incoming ML models before release.
Perform functional, integration, regression, and API testing involving RESTful APIs, microservices, and databases.
Perform non-functional performance and load testing to verify inference service scalability, latency requirements, and reliability under production-like conditions.
Work with ML engineers and operations teams to ensure features and bug fixes have automated test coverage and support continuous integration and deployment.
Debug and analyze test failures and model anomalies, and report defects.
Drive high standards of quality for engineering decisions and customer-facing features through an effective test and automation strategy.
Develop and implement highly scalable and maintainable test automation best practices.
Monitor test execution and optimize performance in test environments.
Develop innovative solutions to organizational problems.