Staff Software Engineer, Data Ingestion - Slack

Slack•Washington, DC

11d•$197,300 - $344,700•Onsite

About The Position

Slack is looking for a Staff Software Engineer to join the Data Ingestion Team within the Data Engineering organization. This team builds and operates the platform that ingests data from various sources into our Lakehouse using both streaming and batch approaches. The current infrastructure handles over 180PB of storage, processes approximately 1 trillion records daily, and manages over 400 data pipelines and 1000+ active Airflow DAGs. The team's goal is to build a highly scalable and resilient ingestion platform leveraging modern Streaming and Lakehouse technologies to accommodate Slack's growing data, customer base, features, and employee count. As a Staff Software Engineer, you will be instrumental in driving our streaming and lakehouse strategies, building scalable services and tools to help customers acquire large volumes of high-quality data efficiently, and ensuring pleasant and productive interactions with ingested data. The ideal candidate will possess deep technical skills, be a self-starter, be detail and quality-oriented, and be passionate about driving data-driven decisions and making a significant impact at Slack.

Requirements

U.S. citizenship and willingness to undergo a background check for GovSlack authorization. Visa sponsorship is not provided.
7+ years of software/data engineering experience.
Experience with building applications using Streaming and Lakehouse technologies (e.g., Kafka/Kafka Connect, Flink/Spark Streaming, Iceberg/Hudi/Delta or equivalent).
Hands-on experience and knowledge in building and maintaining batch data pipelines using Spark, Airflow, EMR, S3 or equivalent.
Proficiency in object-oriented and/or functional programming languages: SQL, Python, Java/Scala, Go or equivalent.
Skilled at crafting and building robust distributed microservices with tools like Kubernetes, Docker, AWS ECS/EKS, Terraform, Grafana, etc.
Familiarity with AI-assisted software development and automation, with hands-on experience of Claude Code/Codex or equivalent.
Excellent written and verbal communication and interpersonal skills; able to effectively collaborate with cross-functional partners and explain sophisticated technical concepts to non-technical stakeholders.

Responsibilities

Design and develop highly scalable and resilient streaming and batch pipelines for ingesting and processing structured and unstructured datasets.
Design and implement Slack’s Lakehouse solutions and enable production use cases for customers.
Improve the reliability and performance of the services/pipelines through AI assisted and/or Agentic solutions and tools.
Automate and handle the lifecycle of datasets (schema evolution, metadata management, change and backfill management, deprecation and migration).
Collaborate with cross-functional partners and lead technical initiatives independently from end to end.
Coach and mentor other engineers across the team and the organization.
Write, review, or provide feedback on technical design proposals.