(USA) Principal, Software Engineer

Wal-Mart • Bentonville, AR

48d • $110,000 - $220,000

About The Position

We are looking for a Principal Software Engineer who is an architect-level expert in Python, with a deep mastery of LLM-driven systems, agentic architectures, and scalable intelligent automation. This is a pivotal role where you'll define the technical strategy, influence architecture across domains, and lead the creation of next-gen AI-driven platforms that move beyond prompt engineering into modular reasoning systems. This role isn't about building one-off workflows-it's about inventing and hardening intelligent systems that can reason, act, and adapt. You will shape the core architecture of multi-agent platforms, ensure LLM integrations are secure, efficient, and observable, and build frameworks that others can extend across use cases and orgs. As a Principal Engineer, you set the vision, write the critical path code, guide Staff and Senior engineers, and are the final word on whether a design is ready for scale.

Requirements

• 10+ years of professional software engineering experience, with 7+ years in Python, building distributed systems at scale.
• Deep knowledge of agentic design patterns, including: ReAct, Plan-and-Execute, AutoGen-style coordination
• Tool calling, dynamic agent routing, and recursive agent planning
• Semantic memory, embedding-based context lookup, summarization windows
• Expertise in building LLM-based systems with LangChain, OpenAI, Anthropic, or custom orchestrators.
• Hands-on experience with:
• RAG pipelines using vector stores (FAISS, Pinecone, Weaviate, Qdrant, Azure Cognitive Search)
• LLM evaluation and observability (tracing, token usage, agent state tracking)
• Workflow orchestration using config-first approaches (YAML/JSON definitions, step runners, etc.)
• Proven ability to drive technical vision, resolve ambiguity, and make architectural tradeoffs at scale.
• Strong background in distributed systems, task queues, asynchronous workflows, and backend performance optimization.
• Experience in cloud-native environments (AWS, GCP, or Azure), including containerization, monitoring, and secure API integrations.

Nice To Haves

• Built or contributed to a custom agentic orchestration framework used across multiple product lines.
• Experience with vector search optimization, context ranking, or temporal memory solutions.
• Published talks, blogs, or papers on LLM systems, AI architecture, or applied reasoning frameworks.
• Deep understanding of how to apply LLM systems in regulated or high-compliance environments (PII handling, redaction, observability).
• Exposure to DevEx platforms for developers to build workflows on top of intelligent agents.
• Familiarity with multi-modal agents (text + vision), LLM simulation patterns, or offline evaluation loops.

Responsibilities

• Define and own the agentic architecture strategy across teams, including MasterAgent design, tool orchestration, memory layers, and dynamic router agents.
• Architect modular, testable, and composable Python systems that support multi-agent workflows, tool-chaining, RAG, memory management, and fallback strategies.
• Design LLM-powered execution engines that support both high throughput and adaptive reasoning (via LangChain, AutoGen, or custom frameworks).
• Lead implementation of retrieval-augmented generation (RAG) pipelines, semantic search, and structured knowledge memory systems.
• Build and scale integrations with internal LLMs, including handling signature-based auth, function calling, and context management at scale.
• Drive end-to-end lifecycle: from configuration schema (YAML) to execution trace logging, observability, and self-healing recovery patterns.
• Lead cross-org architecture reviews, influence roadmap prioritization, and set coding and design standards for all agentic platform work.
• Act as a multiplier by mentoring Staff/Senior engineers, building reusable libraries, and leading technical guilds around AI agent infrastructure.

Benefits

• At Walmart, we offer competitive pay as well as performance-based bonus awards and other great benefits for a happier mind, body, and wallet. Health benefits include medical, vision and dental coverage. Financial benefits include 401(k), stock purchase and company-paid life insurance. Paid time off benefits include PTO (including sick leave), parental leave, family care leave, bereavement, jury duty, and voting. Other benefits include short-term and long-term disability, company discounts, Military Leave Pay, adoption and surrogacy expense reimbursement, and more.
• You will also receive PTO and/or PPTO that can be used for vacation, sick leave, holidays, or other purposes. The amount you receive depends on your job classification and length of employment. It will meet or exceed the requirements of paid sick leave laws, where applicable.
• Live Better U is a Walmart-paid education benefit program for full-time and part-time associates in Walmart and Sam's Club facilities. Programs range from high school completion to bachelor's degrees, including English Language Learning and short-form certificates. Tuition, books, and fees are completely paid for by Walmart.

What This Job Offers

• Job Type: Full-time
• Career Level: Principal
• Industry: General Merchandise Retailers
• Number of Employees: 5,001-10,000 employees