Senior Data Engineer

Walmart • Bentonville, AR

1d • $90,000 - $180,000

About The Position

The Senior Data Engineer applies data strategy and engineering practices to routine business problems, identifying suitable data sources and performing initial data quality checks. The role extracts data from databases, creates pipelines, transforms data for business and analytical use, and uses appropriate integration, preprocessing, and modeling techniques across cloud, big data, data warehouse, and data lake environments. It also analyzes data systems, flows, dependencies, and integration challenges; contributes to conceptual, logical, and physical data models; develops warehouse and data mart designs; and defines relational tables, keys, and stored procedures. The position evaluates existing models and databases, develops efficient data flows, and creates training documentation and trains end users on data modeling. It oversees less experienced programmers and stipulates system troubleshooting support.

The role uses data management, quality, governance, accessibility, storage, scalability, and data-value practices; SQL and NoSQL technologies; ETL, streaming ingestion, scrapers, APIs, and crawlers; data warehousing and MPP platforms; analytical methods; and cloud and big data environments. It develops knowledge of current data science and analytics trends and applies technical problem formulation grounded in business requirements and insights.

Work location: 811 Excellence Dr., Bentonville, AR 72716. The stated annual pay range is $90,000 to $180,000, with additional annual or quarterly performance incentives.

Requirements

• Master’s degree or equivalent in Computer Science or a related field, plus 1 year of experience in software engineering, data engineering, database engineering, business intelligence, business analytics, or a related field; OR a bachelor’s degree or equivalent in Computer Science or a related field, plus 3 years of experience in one of those fields.
• Experience designing and implementing applications with Apache Spark.
• Experience with Hadoop ecosystem components and frameworks, including Hadoop, HDFS, YARN, TEZ, and MapReduce.
• Experience building scripts using Unix Shell Script.
• Experience scheduling production jobs using Apache Airflow and Oozie.
• Experience using Google Cloud Platform to execute applications with clusters and manage data.
• Experience writing Hive applications using Hive Query Language.
• Experience with Maven and SBT deployment tools.
• Experience implementing CI/CD pipelines using Jenkins and using Git for source control management.
• Experience with Agile methodologies and day-to-day sprint tasks involving story pointers (JIRA/Scrum).
• Experience with project management activities, including initiation, planning, execution, monitoring and control, and closure.
• Experience participating in requirements gathering from multiple teams for project increment planning.

Responsibilities

• Apply data strategy and its principles to routine business problems involving a single function.
• Identify appropriate data sources based on business domains, data requirements, data quality, database technologies, and existing business systems and processes.
• Support prioritization of requirements and service level agreements, identify fit-for-purpose data sources, and perform initial data quality checks on extracted data.
• Extract data from identified databases, create data pipelines, and transform data into structures suited to the problem.
• Use data transformation and integration methods, including ETL batch processing, streaming ingestion, scrapers, APIs, and crawlers.
• Analyze complex data elements, systems, data flows, dependencies, and relationships to contribute to conceptual, physical, and logical data models.
• Develop logical and physical data models, including data warehouse and data mart designs.
• Define relational tables, primary and foreign keys, and stored procedures to create data model structures.
• Evaluate existing data models and physical databases for variances and discrepancies.
• Develop efficient data flows and analyze data-related system integration challenges, proposing appropriate solutions.
• Create training documentation and train end users on data modeling.
• Oversee tasks of less experienced programmers and stipulate system troubleshooting support.
• Develop knowledge of current data science and analytics trends.

Benefits

• Medical, vision, and dental coverage.
• 401(k), stock purchase, and company-paid life insurance.
• Paid time off, including sick leave, parental leave, family care leave, bereavement leave, jury duty leave, and voting leave.
• Short-term and long-term disability.
• Education assistance, including 100% company-paid college degrees.
• Company discounts.
• Military service pay.
• Adoption expense reimbursement.
• Performance-based incentive awards, including annual or quarterly performance incentives.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior