(USA) Principal, Software Engineer

Walmart•Sunnyvale, CA

1d•Onsite

About The Position

As a Principal Engineer on the Walmart Advertising technology team, you will define and lead the test engineering strategy for AI-powered advertising systems — spanning ML model validation, intelligent audience pipelines, real-time bidding infrastructure, and programmatic data platforms. You'll architect test frameworks, embed quality into AI/ML development lifecycles, and operate as a senior technical leader across engineering, data science, and product. The mission of Walmart Ads is to connect brand owners and shoppers with relevant ads. We bring brand awareness and rich product information to shoppers and help brand owners grow their business. The Adtech QE team is a horizontal team, with cutting edge technologies, to understand shopper intent and surrounding context, serve relevant ads and optimize campaign performance end-to-end. We are a small, high-performance team that believes quality is a first-class engineering discipline. If you want to work on AI systems that operate at Walmart scale, this is the team.

Requirements

8+ years in quality/test engineering; 3+ years at Staff or Principal SDET level in distributed, large-scale systems
Proven expertise testing AI/ML systems — model validation, feature store integrity, inference pipeline testing, or data quality for ML training datasets
Expert-level experience designing test automation frameworks for APIs, data, and services layers (Pytest, Selenium, Appium, WebDriver, or equivalent)
Deep expertise in data pipeline quality — ETL/ELT validation, schema governance, data completeness, transformation correctness at scale (BigQuery, Hive, Spark)
Strong experience testing event-driven and async systems — Kafka pipelines, REST APIs, webhook integrations
SQL as a first-class testing tool — writing complex queries against large analytical datasets to validate data correctness
Experience enforcing quality gates in CI/CD at organizational scale
Bachelor's degree in computer science, computer engineering, computer information systems, software engineering, or related area and 5 years’ experience in software engineering or related area.
OR 7 years’ experience in software engineering or related area.

Nice To Haves

Experience with LLM/GenAI quality — prompt regression testing, output evaluation frameworks, hallucination detection, or RAG pipeline validation
Expertise in programmatic advertising quality — RTB auction integrity, DSP/SSP integrations, ad serving correctness, measurement/attribution validation
Hands-on with GCP data platform: BigQuery, Dataflow, Pub/Sub, Vertex AI
Experience in privacy and compliance testing — GDPR, CCPA, consent signal propagation, data deletion verification, CMP validation
Proficiency with performance and load testing for high-throughput, low-latency systems
Familiarity with MLOps tooling — experiment tracking (MLflow, Vertex AI), model registries, feature stores, and deployment pipelines
Experience with data clean room and privacy-preserving measurement validation
Background in building and growing SDET teams — hiring, mentoring, technical interview design
Master’s degree in computer science, computer engineering, computer information systems, software engineering, or related area and 3 years' experience in software engineering or related area.
Experience with creating inclusive digital experiences, demonstrating knowledge in implementing Web Content Accessibility Guidelines (WCAG) 2.2 AA standards, assistive technologies, and integrating digital accessibility seamlessly.
Knowledge of accessibility best practices and joining us as we continue to create accessible products and services following Walmart’s accessibility standards and guidelines for supporting an inclusive culture.

Responsibilities

Architect and own enterprise-scale test automation frameworks for APIs, data pipelines, UI, and services layers — built for maintainability, scale, and CI/CD integration
Drive shift-left quality — embedding automated quality checks at every stage of the development lifecycle, from schema design through production monitoring
Define and enforce quality gates in CI/CD pipelines at an organizational level; own release criteria and go/no-go decisions for major platform changes
Lead data pipeline testing at scale — ETL/ELT validation, schema governance, event-level tracking accuracy, audience segment integrity, and aggregation correctness (BigQuery, Spark, Kafka)
Own test strategy for AI/ML systems — including model validation, data drift detection, feature pipeline integrity, inference correctness, and algorithmic fairness testing for advertising models (lookalike, propensity, audience prediction)
Define model quality gates in ML training and deployment pipelines — validating model outputs against golden datasets, statistical thresholds, and business KPIs before promotion to production
Build automated evaluation frameworks for GenAI and LLM-integrated components — including prompt regression testing, output consistency checks, and hallucination detection pipelines
Partner with data scientists to implement shadow testing, A/B experiment integrity validation, and statistical significance guardrails for algorithmic experiments
Implement performance and chaos engineering for latency-sensitive systems — bidding engines, ad decisioning, and audience lookup at millions of QPS
Represent quality in architecture and design reviews — surfacing testability gaps, observability blind spots, and data integrity risks before they are built in
Author quality frameworks, testing playbooks, and SDET standards adopted across WMX and Growth Tech
Define SLOs and production quality posture — integrating quality signals into observability dashboards and driving proactive incident prevention
Mentor and grow Staff and Senior SDETs; raise the engineering bar through design reviews, code reviews, and hiring