Senior Data Engineer

The Washington Post•Washington, DC

12d•$119,700 - $199,300•Onsite

About The Position

We are seeking a skilled and experienced Senior Data Engineer for our Data and AI team to contribute to the development of innovative AI-powered projects at The Washington Post. As a key member of our team, you will be responsible for designing, building, and deploying data solutions that power cutting-edge AI technologies, including Generative AI, personalization, and recommendation systems, as well as revenue science applications. Your work will also support advertising insights generation and campaign infrastructure. This role involves gathering requirements, writing code, testing, deploying, and maintaining software using best practices in development. The engineer will work collaboratively on large-scale projects, contribute to the creation of new features, products, and tools, and mentor junior developers.

Requirements

Bachelor's degree in Computer Science, Software Engineering, or a related field is acceptable, but a Master's degree is preferred. Equivalent professional experience will also be considered.
3+ years of overall experience in ETL/ELT, Data modeling, and querying and analyzing large datasets using SQL.
Extensive experience with ETL development using Python and PySpark.
Experience with AWS Data Lakehouse technologies including Redshift, Athena, EMR, and Glue catalog.
Strong understanding of Pub/Sub and event-driven architecture.
Able to design and deploy DAGs using Apache Airflow.
Strong proficiency with cloud infrastructure, including experience with AWS services (e.g., EC2, S3, RDS, Lambda, Batch) for building and deploying scalable, reliable, cloud-based applications.
Experience with software development best practices, including coding standards, code reviews, testing, and CI/CD pipelines.
Experience with performance monitoring and observability tools (e.g., Splunk, Datadog) for maintaining application health and performance.
Proven ability to mentor junior engineers, influence team-wide practices, and communicate technical concepts to both technical and non-technical stakeholders.
Excellent problem-solving skills, with the ability to troubleshoot and debug issues across the stack.
Ability to handle complex projects with limited supervision and manage technical decision-making.

Nice To Haves

Experience with Apache Iceberg, Postgres, MySQL, DynamoDB, Tableau, and Superset also highly preferred.
Experience working with Google Ad Manager data and Ad Tech vendors is a plus.
Strong skills with Docker and ECS are highly preferred.

Responsibilities

Lead data engineering efforts using Python, Spark, Flink and other data processing technologies, delivering scalable, secure, and reliable data pipelines, and contribute to the architecture and decision-making processes of large, complex projects.
Collaborate on gathering and analyzing project requirements to define technical solutions, and work with cross-functional teams to integrate front-end and back-end solutions, ensuring seamless user experiences.
Write clear, maintainable code, following best practices in version control, code reviews, and testing, and ensure that all software solutions are scalable, secure, and meet the needs of our users.
Collaborate with cross-functional teams, including data scientists, product managers, and designers, to identify and prioritize project requirements, and communicate technical solutions to stakeholders.
Mentor and coach junior engineers, offering technical guidance and reviewing code to ensure quality, and contribute to the growth and development of our engineering team.
Collaborate with Product, Design, Newsroom and business stakeholders to manage and optimize project timelines, proactively identifying blockers and risks to ensure timely delivery.
Stay current with industry trends, suggesting improvements to enhance performance, security, and scalability, and apply knowledge of data engineering to drive innovation and improvement in our products and services.
Contribute to the development of technical documentation and design specifications for team members and share knowledge and expertise with the team to promote a culture of innovation and collaboration.