Senior Manager, Business Operations eCommerce

Wal-Mart • Bentonville, AR

45d • Onsite

About The Position

Position Summary... What you'll do...

Requirements

• 2-4 years in AI operations, ML evaluation, or eCommerce data quality; experience managing annotation or human-in-the-loop programs preferred.
• Working knowledge of AI evaluation frameworks (LLM-as-Judge, golden datasets, A/B testing).
• Skilled in KPI design, audit processes, and vendor oversight.
• Analytical storyteller able to link model metrics to business outcomes.
• Obsessed with customer trust, transparency, and continuous improvement.
• Follows the law our code of conduct and company policies and sets expectations for others to do the same promotes an environment where associates feel comfortable sharing concerns and reinforces our culture of non retaliation listens to concerns raised by associates takes action and encourages others to do the same holds self and others accountable for achieving results in a way that is consistent with our values
• Acts as an altruistic servant leader and is consistently humble self-aware honest and transparent
• Delivers expected business results while putting the customer member first and consistently applying an omni merchant mindset and acts with an everyday Low-cost mindset to drive value and every day Low Prices for customers members
• Adopts a holistic perspective that considers data analytics customer member insights and different parts of the business when making plans and shaping the teams strategy
• Consistently raises the bar and seeks to improve demonstrates curiosity and a growth mindset seeks feedback asks thoughtful questions fosters an environment that supports learning innovation and learning from mistakes and intelligent risk-taking and exhibits resilience in the face of setbacks
• Seeks and implements continuous improvements and encourages the team to leverage new digital tools and ways of working
• Bachelor's degree in information technology, computer science, or related area and 5 years' experience in eCommerce merchandising, site operations, business management, or related area OR 7 years' experience in eCommerce merchandising, site operations, business management, information technology, computer science or related area.
• 2 years' supervisory experience.

Nice To Haves

• ECommerce Merchandising, Site Operations, Business Management, or related area
• Master's degree in Information Technology, Computer Science, or related area

Responsibilities

• Manage dashboards connecting capability metrics with business metrics to surface what's working and what isn't.
• Maintain golden datasets for benchmarking model relevance, execution accuracy, personalization, and compliance.
• Partner with AI Tech and Tech Ops to calibrate LLM-as-Judge and ensure automated evaluation remains aligned with human judgment.
• Conduct periodic audits of shopper-facing AI experiences (answers, nudges, recommendations, in-shop chat).
• Define and monitor KPIs for intent recognition, execution accuracy, and personalization precision.
• Run cross-journey evaluations to identify friction or divergence patterns.
• Own and evolve SOPs for annotation, labeling, and evaluation to reflect changing features and customer contexts.
• Capture and codify new edge cases into updated labeling guides.
• Ensure all SOPs align with shopper-first outcomes and compliance .
• Track variance between human labels, automated judgments, and live model outputs.
• Create structured escalation paths for high-risk failures (irrelevant, unsafe, or biased responses).
• Quantify divergence trends and drive remediation with Product and Engineering.
• Manage external labeling partners, ensuring throughput, accuracy, and compliance SLAs.
• Deliver training grounded in real Sparky interactions and golden dataset examples.
• Audit vendor output and hold partners accountable for quality metrics.
• Build and manage the continuous feedback loop that connects: Shopper signals Human review & annotation Automated LLM evaluation
• Ensure that every signal from real interactions feeds directly into model training, experience design, and KPI improvement-driving measurable gains in personalization, nudge effectiveness, and conversational trust

Benefits

• At Walmart, we offer competitive pay as well as performance-based bonus awards and other great benefits for a happier mind, body, and wallet. Health benefits include medical, vision and dental coverage. Financial benefits include 401(k), stock purchase and company-paid life insurance. Paid time off benefits include PTO (including sick leave), parental leave, family care leave, bereavement, jury duty, and voting. Other benefits include short-term and long-term disability, company discounts, Military Leave Pay, adoption and surrogacy expense reimbursement, and more.
• You will also receive PTO and/or PPTO that can be used for vacation, sick leave, holidays, or other purposes. The amount you receive depends on your job classification and length of employment. It will meet or exceed the requirements of paid sick leave laws, where applicable.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Industry: General Merchandise Retailers
• Number of Employees: 5,001-10,000 employees