Senior Data Engineer

ATPCO•Herndon, VA

1d•$110,572 - $145,000•Remote

About The Position

ATPCO is the world's primary source for air fare content, holding over 200 million fares across 160 countries. Every day, the travel industry relies on ATPCO's technology and data solutions to help millions of travelers reach their destinations efficiently. Join us and contribute to the development of innovative applications that power global travel. ATPCO is everywhere people buy flights. We’re not just shaping the future of air travel– we’re redefining how and where great work happens. At ATPCO, we believe in flexibility, trust, and a culture where your wellbeing comes first As a Senior Data Engineer, you will be responsible for building and optimizing data pipelines, managing data storage and processing systems, and ensuring the availability, scalability, and reliability of our data platform. You will collaborate with cross-functional teams, including data scientists, software engineers, and business stakeholders to understand data requirements and deliver efficient high-quality data solutions. We are seeking an accomplished individual to join our esteemed team of ATPCO data engineers, dedicated to tackling intricate data processing challenges within the Airline industry. This role offers an opportunity to work on cutting-edge solutions that involve handling hundreds of terabytes of data daily, leveraging the latest AWS services.

Requirements

Strong programming skills in languages like Python, Java, or Scala, with experience in data manipulation and transformation frameworks
Proven experience as a data engineer, with experience in designing and building large-scale data processing systems
Strong understanding of data modeling concepts and data management principles
In-depth knowledge of SQL and experience working with relational and non-relational databases
Knowledge of Data Mesh principles and experience designing and implementing data architectures following Data Mesh concepts within the AWS ecosystem
Experience with real-time data streaming architectures using AWS services like Amazon Kinesis or Apache Kafka
Familiarity with AWS cloud services, such as AWS Glue, AWS Lambda, AWS EMR, AWS S3, Amazon Redshift, and their data-related features and functionalities
Familiarity with AWS security services and features for data security and privacy
Bachelor's or Master's degree in Computer Science, Information Systems, or a related field

Responsibilities

Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and design data models and schemas that facilitate data analysis and reporting
Design, develop, and maintain scalable and efficient data pipelines and ETL processes to ingest, process, and transform large volumes of data from various sources into usable formats
Build and optimize data storage and processing systems, including data warehouses, data lakes, and big data platforms, using AWS services such as Amazon Redshift, AWS Glue, AWS EMR, AWS S3, and AWS Lambda, to enable efficient data retrieval and analysis
Implement and manage real-time data streaming architectures using AWS services like Amazon Kinesis or Apache Kafka to enable real-time data processing and analytics
Perform data profiling, data cleansing, and data transformation tasks to prepare data for analysis and reporting
Implement data security and privacy measures to protect sensitive and confidential data using AWS security services and features
Design and implement data architectures following Data Mesh principles within the AWS environment, including domain-oriented data ownership, self-serve data infrastructure, and federated data governance
Provide technical guidance and mentorship to junior data engineers, reviewing their work and ensuring adherence to best practices and standards