Staff Data Engineer

Walmart • Bentonville, AR

9d

About The Position

The Staff Data Engineer translates and co-owns business problems in the data discipline, identifies suitable methods and tools, and recommends solutions to complex business issues. The role translates business requirements into projects and tasks aligned with business strategy, develops business cases, and connects business needs with practical results.

The position identifies fit-for-purpose data sources, performs initial data quality checks, and guides junior associates. It builds and automates data transformation and integration infrastructure and pipelines, evaluates data risks, and makes complex, multi-source data usable for modeling. The role assembles and models data across platforms, supports data architecture and modeling standards, and maintains documentation and metadata.

The Staff Data Engineer reviews designs and code, develops and tests solutions, monitors production applications, and leads patches and scalability improvements. The role also establishes and documents data governance recommendations, implements governance practices with stakeholders, interprets company and regulatory policies, and educates others on governance processes.

Requirements

• Bachelor's degree or equivalent in Computer Science or a related field and 4 years of experience in software engineering or related experience; alternatively, a master's degree or equivalent in Computer Science or a related field and 2 years of experience in software engineering or related experience.
• Experience designing and implementing scalable, optimized, distributed big-data pipelines and ETLs using frameworks such as Hadoop, Hive, Spark, and Databricks.
• Experience developing ETL pipelines using Scala, Python, Spark, and SQL.
• Experience using SQL and Spark SQL for data analysis and validation, and Spark shell scripting for automated data validation.
• Experience documenting data pipelines and workflows using tools such as Confluence, Jira, and Azure DevOps.
• Experience orchestrating and automating pipeline solutions with tools and frameworks such as Automic, AutoSys, Oozie, and Apache Airflow.
• Experience querying relational databases and data warehouses using SQL, including Apache Hive and Snowflake.
• Experience developing and maintaining modules in cloud ecosystems such as Google Cloud, AWS, and Azure.
• Experience working with Parquet, AVRO, CSV, ORC, and JSON, and file-based storage systems including HDFS, GCS, Azure Blob Storage, and S3.
• Experience creating and automating dashboards using Power BI, Tableau, and Looker.
• Experience with system architecture, high-level design through detailed design, and disaster recovery.
• Experience with resource modeling, optimization, and forecasting.
• Experience applying incident management, change management, defect-reduction, and system-reliability processes to improve operational performance.

Responsibilities

• Translate and co-own business problems into data-related or mathematical solutions; select appropriate methods and tools and demonstrate how they address business needs.
• Recommend solutions to complex business issues, develop project business cases, translate requirements into aligned projects and tasks, and communicate relevant business insights.
• Identify suitable data sources based on requirements and service-level agreements, perform initial data-quality checks, and guide junior associates on data sources and quality.
• Build and automate infrastructure and pipelines for data transformation and integration across varied sources; deploy pipelines with scheduling and orchestration frameworks and assess data issues and risks.
• Create methods to fuse and reshape complex, multi-source data for modeling and keep current with big-data analytics and data-science trends and techniques.
• Assemble data across platforms; build logical and conceptual models, guide physical modeling, define methods for exposing data to other systems, and contribute to modeling standards and architecture.
• Maintain critical data documentation and metadata so data can be understood and shared.
• Review solution and application designs, select development languages and libraries, map tests to requirements, create proofs of concept, troubleshoot code, and recommend testing and deployment approaches.
• Identify and drive application changes needed for different environments and scalability; monitor production applications and lead patch development when needed.
• Ensure code documentation is complete and periodically updated.
• Establish, modify, and document data governance projects and recommendations; implement governance practices with stakeholders and peers.
• Interpret company and regulatory data policies, educate others on governance practices, and recommend policy or guideline updates.
• Lead large-scale projects and provide guidance to team members.

Benefits

• Medical, vision, and dental coverage.
• 401(k), stock purchase plan, and company-paid life insurance.
• Paid time off, including sick leave, parental leave, family care leave, bereavement leave, jury duty leave, and voting leave.
• Short-term and long-term disability.
• Education assistance, including company-paid college degrees.
• Company discounts.
• Military service pay.
• Adoption expense reimbursement.
• Annual or quarterly performance incentives.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior