Staff Data Engineer

Walmart • Sunnyvale, CA

1d • $212,326 - $286,000 • Onsite

About The Position

The Staff Data Engineer establishes, modifies, and documents data governance projects and recommendations, partnering with business stakeholders and peers to implement governance practices. The role interprets company and regulatory data policies, educates others on governance processes, and recommends updates to policies, practices, and guidelines.

The role translates business problems into data-related or mathematical solutions, identifies suitable methods and tools, prioritizes requirements and service-level agreements, and finds fit-for-purpose data sources. It performs initial data quality checks, assesses data risks, and reviews junior associates’ deliverables while providing guidance.

The Staff Data Engineer builds infrastructure for transforming and integrating data from varied sources, automates repeatable data preparation tasks, and deploys pipelines through scheduling and orchestration frameworks. The role fuses and reshapes complex, multi-source data for modeling and keeps current with big data analytics and data science trends.

The position builds complex logical and conceptual data models, guides physical modeling, defines data exposure techniques, and maintains data documentation and metadata. It contributes to data modeling standards and architectural design, reviews solution and application designs, and ensures they meet business, technical, and data requirements.

The role identifies languages, libraries, testing methodologies, and deployment environments; creates proof of concepts; reviews and troubleshoots code; and recommends application changes for differing environments and scalability. It monitors production applications, leads development of patches as needed, and ensures code documentation is complete and periodically updated. It applies defined strategy principles to moderately complex business problems that may span functions or domains.

The position is based at 1375 Crossman Avenue, Sunnyvale, CA 94089. The salary range is $212,326 to $286,000 per year, with additional annual or quarterly performance incentives.

Requirements

• Bachelor’s degree or equivalent in Computer Science, Information Technology, Engineering (any), or a related field and 4 years of experience in software engineering or a related field; OR a master’s degree or equivalent in one of these fields and 2 years of experience in software engineering or a related field.
• Experience with SQL for data retrieval, extraction, transformation, and querying, including SQL/JDBC.
• Experience developing robust and scalable data engineering pipelines using Python, Java, and Scala.
• Experience containerizing data engineering applications with Docker and deploying containerized workloads on Kubernetes.
• Experience architecting and engineering big data solutions using Databricks, Apache Spark, SparkSQL, and Hadoop ecosystems.
• Experience building and deploying solutions on public cloud platforms, including Google Cloud Platform and Databricks.
• Experience developing real-time and batch data systems, including cleansing, transformation, and loading with Apache Spark, Spark Streaming, and Kafka.
• Experience orchestrating data pipelines using Apache Airflow.
• Experience designing and automating CI/CD pipelines using Jenkins and Ansible.
• Experience troubleshooting business and production data issues, identifying root causes, and maintaining SLAs.
• Experience implementing data quality checks, row-level comparisons, and hash-based validation.
• Experience with data modeling and developing logical and physical data models for transactional systems, analytical systems, and data lakes.
• Experience designing and implementing REST API web services using Java or Python.
• Experience implementing data governance frameworks for metadata management, schema validation, and data asset cataloguing.
• Experience creating unit, integration, and load test cases using PyTest and custom validation frameworks.
• Experience monitoring and alerting production pipelines on failures and implementing custom logging solutions.
• Experience creating proof of concepts to evaluate cutting-edge data and cloud technologies.
• Employer will accept any amount of experience with the required skills.

Responsibilities

• Establish, modify, and document data governance projects and recommendations, implementing practices with business stakeholders and peers.
• Interpret data policies and educate others about governance processes, practices, policies, and guidelines; recommend updates as needed.
• Translate business problems into data-related or mathematical solutions and identify appropriate methods, tools, and data sources.
• Understand requirements and service-level agreement priorities; perform initial data quality checks and assess data issues and risks.
• Review junior associates’ deliverables and guide them on data sourcing and quality.
• Build infrastructure for data transformation and integration from varied sources, automate repeatable preparation and integration tasks, and deploy pipelines using scheduling and orchestration frameworks.
• Create methods to fuse and reshape complex, multi-source data for modeling and stay informed about emerging big data analytics and data science trends.
• Build complex logical and conceptual models, guide physical data modeling, define techniques for exposing data to other systems, and contribute to modeling standards and best practices.
• Create and maintain data documentation and metadata so data can be understood and used as a shared asset.
• Contribute to architectural design and review solution and application designs for business, technical, and data requirements.
• Identify development languages and libraries, map test cases to requirements, create proof of concepts, review and troubleshoot code, and recommend testing methodologies.
• Identify deployment environments and recommend application modifications for environmental needs and scalability.
• Monitor production applications and lead development of patches when required.
• Review and maintain complete, up-to-date code documentation.
• Apply defined strategy principles to moderately complex business problems spanning one or more functions or domains.

Benefits

• Medical, vision, and dental coverage.
• 401(k), stock purchase, and company-paid life insurance.
• Paid time off, including sick leave, parental leave, family care leave, bereavement leave, jury duty, and voting leave.
• Short-term and long-term disability.
• Education assistance, including 100% company-paid college degrees.
• Company discounts.
• Military service pay.
• Adoption expense reimbursement.
• Annual or quarterly performance incentives.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior