Staff Data Engineer

Walmart • Sunnyvale, CA

7d • Onsite

About The Position

The Staff Data Engineer establishes, modifies, and documents data governance projects and recommendations, implementing governance practices with business stakeholders and peers. The role interprets company and regulatory data policies, educates others on governance processes, and recommends updates to governance policies, practices, and guidelines.

The position translates business problems into data-related or mathematical solutions, identifies appropriate methods and tools, and determines suitable, fit-for-purpose data sources. It performs initial data quality checks, considers requirements and service-level agreements, and reviews junior associates’ deliverables while providing guidance on data sources and quality.

The engineer builds infrastructure for transformation and integration across varied data sources, automates repeatable data preparation tasks, and deploys pipelines with scheduling and orchestration frameworks. The role identifies and addresses data risks, reshapes complex multi-source data for modeling, and keeps current with big data analytics and data science trends. It develops complex logical and conceptual models, guides physical modeling, defines methods for exposing data to other systems, and maintains data documentation and metadata.

The position contributes to architectural design and reviews application and solution designs for business, technical, and data requirements. It identifies languages, libraries, tests, deployment environments, and testing methodologies; creates proofs of concept; reviews and troubleshoots code; and recommends changes for differing environments and scalability. It monitors production applications, leads patch development when needed, and ensures code documentation is complete and current. The role applies defined strategies to moderately complex business problems that may span functions or domains.

The position is based at 1375 Crossman Avenue, Sunnyvale, CA 94089. Salary range is $212,326 to $286,000 per year, with additional annual or quarterly performance incentives.

Requirements

• Bachelor’s degree or equivalent in Computer Science, Information Technology, Engineering (any), or a related field and 4 years of experience in software engineering or a related field; OR a master’s degree or equivalent in one of those fields and 2 years of experience in software engineering or a related field.
• Experience with SQL for data retrieval, extraction, transformation, and querying, including SQL/JDBC.
• Experience developing robust, scalable data engineering pipelines using Python, Java, and Scala.
• Experience containerizing data engineering applications with Docker and deploying workloads on Kubernetes.
• Experience architecting and engineering big data solutions using Databricks, Apache Spark, SparkSQL, and Hadoop ecosystems.
• Experience building and deploying solutions on public cloud platforms, including Google Cloud Platform and Databricks.
• Experience developing real-time and batch data systems, including data cleansing, transformation, and loading using Apache Spark, Spark Streaming, and Kafka.
• Experience orchestrating data pipelines using Apache Airflow.
• Experience designing and automating CI/CD pipelines using Jenkins and Ansible.
• Experience troubleshooting business and production data issues, identifying root causes, and maintaining SLAs.
• Experience implementing data quality checks, row-level comparisons, and hash-based validation.
• Experience with data modeling and developing logical and physical data models for transactional systems, analytical systems, and data lakes.
• Experience designing and implementing REST API web services using Java or Python.
• Experience implementing data governance frameworks for metadata management, schema validation, and data asset cataloguing.
• Experience creating unit, integration, and load test cases using PyTest and custom validation frameworks.
• Experience monitoring and alerting production pipelines for failures and implementing custom logging solutions.
• Experience creating proofs of concept to evaluate the feasibility of emerging data and cloud technologies.
• The employer will accept any amount of experience with the required skills.

Responsibilities

• Establish, modify, and document data governance projects and recommendations.
• Implement data governance practices with business stakeholders and peers; interpret policies and educate others on governance processes, policies, and guidelines.
• Recommend updates or inputs to data governance policies, practices, and guidelines.
• Translate business problems into data-related or mathematical solutions; identify suitable methods and tools and demonstrate how they address the problem.
• Understand the priority of requirements and service-level agreements; identify fit-for-purpose data sources and perform initial data quality checks.
• Review junior associates’ deliverables and provide guidance on data sources and quality.
• Build infrastructure for data transformation and integration across a wide variety of sources.
• Automate repeatable data preparation and integration tasks using appropriate tools, techniques, and architecture.
• Deploy pipelines using scheduling and orchestration frameworks.
• Assess data issues and risks, and create methods to fuse and reshape complex multi-source data for modeling.
• Maintain knowledge of emerging big data analytics and data science trends and techniques.
• Build complex logical and conceptual data models and guide physical data modeling.
• Identify techniques for exposing data to other systems; review and guide team data modeling activities.
• Create and maintain data documentation and metadata; contribute to data modeling standards and foundational best practices.
• Provide architectural design input to make effective use of resources in light of goals and expected loads.
• Review solution and application designs to ensure they meet business, technical, and data requirements.
• Identify languages and libraries for development; map test cases to business and functional requirements; create proofs of concept.
• Review and troubleshoot code against final designs and recommend appropriate testing methodologies.
• Identify deployment environments and recommend application modifications for environmental requirements and scalability.
• Monitor production applications and lead development of patches when required.
• Ensure code documentation is complete and periodically updated.
• Apply defined strategy principles to moderately complex business problems spanning one or more functions or domains.

Benefits

• Competitive pay and performance-based incentive awards.
• Medical, vision, and dental coverage.
• 401(k), stock purchase, and company-paid life insurance.
• Paid time off, including sick leave, parental leave, family care leave, bereavement leave, jury duty, and voting leave.
• Short-term and long-term disability.
• Education assistance, including company-paid college degrees.
• Company discounts.
• Military service pay.
• Adoption expense reimbursement.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior