Senior Data Engineer

The Washington Post

22h•Onsite

About The Position

Join the future of news We’re on a mission to deliver riveting storytelling for all of America. At The Washington Post, you’ll help reinvent news. Our work is driven by a deep investigative spirit and enhanced by innovation to bring audiences closer to the stories that matter most. About Our Team The Washington Post is powered by the passion and talent of our people. It takes all of us to reinvent news. Beyond our award-winning Newsroom and Opinions teams, we work across many departments, including Brand & Events, Communications, Customer Care, Engineering & Product, Finance, Human Resources, Legal, Marketing & Advertising, Print Operations, and Sales. Why This Role Matters We are seeking a skilled and experienced Senior Data Engineer for our Data and AI team to contribute to the development of innovative AI-powered projects at The Washington Post. As a key member of our team, you will be responsible for designing, building, and deploying data solutions that power cutting-edge AI technologies, including Generative AI, personalization, and recommendation systems, as well as revenue science applications. Your work will also support advertising insights generation and campaign infrastructure. This role involves gathering requirements, writing code, testing, deploying, and maintaining software using best practices in development. The engineer will work collaboratively on large-scale projects, contribute to the creation of new features, products, and tools, and mentor junior developers. What Motivates You You are passionate about solving complex technical challenges and driving impactful projects that shape the user experience for a news media platform. You'll thrive in our team if you're a natural problem-solver, a strong communicator, and a proactive and curious individual who is always looking to learn and grow. You'll be motivated by the opportunity to tackle complex technical challenges, lead and influence a team, and collaborate with cross-functional teams to drive user-focused solutions. You'll also have the freedom to explore emerging technologies and trends, and to identify new opportunities for growth and innovation. You're passionate about delivering high-quality solutions that meet the needs of our users.

Requirements

Education and Experience: Bachelor's degree in Computer Science, Software Engineering, or a related field is acceptable, but a Master's degree is preferred. Equivalent professional experience will also be considered.
3+ years of overall experience in ETL/ELT, Data modeling, and querying and analyzing large datasets using SQL.
Technical Skills: Extensive experience with ETL development using Python and PySpark.
Experience with AWS Data Lakehouse technologies including Redshift, Athena, EMR, and Glue catalog.
Strong understanding of Pub/Sub and event-driven architecture.
Able to design and deploy DAGs using Apache Airflow.
Strong proficiency with cloud infrastructure, including experience with AWS services (e.g., EC2, S3, RDS, Lambda, Batch) for building and deploying scalable, reliable, cloud-based applications.
Strong skills with Docker and ECS are highly preferred.
Software Development: Experience with software development best practices, including coding standards, code reviews, testing, and CI/CD pipelines.
Performance Monitoring: Experience with performance monitoring and observability tools (e.g., Splunk, Datadog) for maintaining application health and performance.
Leadership and Communication: Proven ability to mentor junior engineers, influence team-wide practices, and communicate technical concepts to both technical and non-technical stakeholders.
Problem-Solving: Excellent problem-solving skills, with the ability to troubleshoot and debug issues across the stack.
Autonomy and Decision-Making: Ability to handle complex projects with limited supervision and manage technical decision-making.

Nice To Haves

Experience with Apache Iceberg, Postgres, MySQL, DynamoDB, Tableau, and Superset also highly preferred.
Experience working with Google Ad Manager data and Ad Tech vendors is a plus.

Responsibilities

Technical Leadership: Lead data engineering efforts using Python, Spark, Flink and other data processing technologies, delivering scalable, secure, and reliable data pipelines, and contribute to the architecture and decision-making processes of large, complex projects.
Project Planning and Execution: Collaborate on gathering and analyzing project requirements to define technical solutions, and work with cross-functional teams to integrate front-end and back-end solutions, ensuring seamless user experiences.
Software Development: Write clear, maintainable code, following best practices in version control, code reviews, and testing, and ensure that all software solutions are scalable, secure, and meet the needs of our users.
Collaboration and Communication: Collaborate with cross-functional teams, including data scientists, product managers, and designers, to identify and prioritize project requirements, and communicate technical solutions to stakeholders.
Mentorship and Coaching: Mentor and coach junior engineers, offering technical guidance and reviewing code to ensure quality, and contribute to the growth and development of our engineering team.
Project Management: Collaborate with Product, Design, Newsroom and business stakeholders to manage and optimize project timelines, proactively identifying blockers and risks to ensure timely delivery.
Technical Expertise: Stay current with industry trends, suggesting improvements to enhance performance, security, and scalability, and apply knowledge of data engineering to drive innovation and improvement in our products and services.
Documentation and Knowledge Sharing: Contribute to the development of technical documentation and design specifications for team members and share knowledge and expertise with the team to promote a culture of innovation and collaboration.