Data Engineer, Central InfraOps Analytics Team

Amazon • Herndon, VA

5h

About The Position

As a Data Engineer you will enable data-driven decision making within the Amazon
Web Services Data Center Infrastructure Operations organization. The
Infrastructure Operations Team is responsible for planning, implementing,
monitoring and continuously improving the global Amazon Data Center
infrastructure. The team supports all aspects of the Data Center based
organizations, including but not limited to : Safety, Security, maintenance,
operations, logistics, engineering and equipment management.

At AWS, the Data Engineer fully embraces the "You Build It, You Own It"
philosophy, taking complete ownership of data solutions from conception through
deployment and ongoing maintenance. You design architectures, implement
pipelines, and remain responsible for their health and evolution as business
needs change.

Each day begins with reviewing pipeline alerts and data quality metrics,
followed by a 15-30 minute team stand-up to align on priorities and discuss
blockers. You'll spend time monitoring infrastructure, reviewing logs for ETL
pipeline health and data lake performance, then dedicate time to address
stakeholder queries and prioritizing incoming requests via email, Slack and
intake forms. The majority of your time is spent developing and maintaining ETL
pipelines that ingest infrastructure operational data from global data centers,
which includes writing code, debugging issues, optimizing queries, and
implementing quality checks. The role requires frequent context switching
between developing new data models, supporting existing infrastructure, and
consulting on data utilization.

Key challenges you'll tackle include unifying and understanding fragmented data
from diverse data center systems, enabling infrastructure monitoring, supporting
analytics for capacity planning, driving optimization through data insights,
automating manual processes, creating self-service access for business users,
maintaining quality across massive datasets, ensuring compliance with strict
security requirements, designing for scale as AWS expands globally, and
modernizing legacy systems to reduce technical debt.

Requirements

• 3+ years of data engineering experience
• 3+ years of developing and operating large-scale data structures for business intelligence analytics using ETL/ELT processes experience
• Experience with data modeling, warehousing and building ETL pipelines
• Experience in at least one modern scripting or programming language, such as Python, Java, Scala, or NodeJS

Nice To Haves

• Experience with AWS technologies like Redshift, S3, AWS Glue, EMR, Kinesis, FireHose, Lambda, and IAM roles and permissions
• Experience with non-relational databases / data stores (object storage, document or key-value stores, graph databases, column-family databases)
• Knowledge of batch and streaming data architectures like Kafka, Kinesis, Flink, Storm, Beam
• Experience working on and delivering end to end projects independently

Responsibilities

• Design, develop, and maintain ETL pipelines to ingest data into the data warehouse and data lake
• Create and optimize logical data models that drive physical design for the Infrastructure Operations organization
• Implement data quality measures and ongoing monitoring to ensure data integrity
• Build scalable, efficient, and maintainable data solutions that support business intelligence needs
• Optimize data storage and query performance across various data platforms
• Develop automated processes to replace manual data operations
• Collaborate with business stakeholders to understand data and reporting requirements
• Translate business questions into data solutions that drive decision-making
• Mentor and develop peers in data engineering best practices
• Participate in code reviews, design discussions, and team planning
• Improve self-service access to data for business users
• Enhance code quality and dependency management
• Automate manual processes to increase efficiency
• Identify and resolve root causes of complex data problems

Benefits

• health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage)
• 401(k) matching
• paid time off
• parental leave

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Education Level: No Education Listed