Staff Software Engineer, Data Ingestion - Slack

Salesforce•Washington DC, VA

About The Position

Salesforce is seeking a Staff Software Engineer to join the Data Ingestion Team within the Data Engineering organization at Slack. This team is responsible for building and operating the platform that ingests data from various sources into their Lakehouse using both streaming and batch approaches. The current infrastructure handles over 180PB of storage, processes approximately 1 trillion records daily, and manages over 400 data pipelines and 1000+ active Airflow DAGs. The team's objective is to construct a highly scalable and resilient ingestion platform utilizing modern Streaming and Lakehouse technologies to accommodate Slack's growing data, customer base, features, and employee count. As a Staff Software Engineer, you will be instrumental in shaping the streaming and lakehouse strategies, developing scalable services and tools to enable customers to acquire large volumes of high-quality data efficiently, and ensuring pleasant and productive interactions with ingested data. The ideal candidate will possess deep technical expertise, be a self-starter, exhibit a strong focus on detail and quality, and be passionate about driving data-driven decisions and making a significant impact at Slack.

Requirements

U.S. citizenship and willingness to undergo a background check for GovSlack authorization. Visa sponsorship is not provided for this role.
7+ years of software/data engineering experience, including experience with building applications using Streaming and Lakehouse technologies, e.g. Kafka/Kafka Connect, Flink/Spark Streaming, Iceberg/Hudi/Delta or equivalent.
Hands-on experience and knowledge in building and maintaining batch data pipelines using Spark, Airflow, EMR, S3 or equivalent.
Proficiency in object-oriented and/or functional programming languages: SQL, Python, Java/Scala, Go or equivalent.
Skilled at crafting and building robust distributed microservices with tools like Kubernetes, Docker, AWS ECS/EKS, Terraform, Grafana, etc.
Familiarity with AI-assisted software development and automation, with hands-on experience of Claude Code/Codex or equivalent.
Excellent written and verbal communication and interpersonal skills; able to effectively collaborate with cross-functional partners and explain sophisticated technical concepts to non-technical stakeholders.

Responsibilities

Design and develop highly scalable and resilient streaming and batch pipelines for ingesting and processing structured and unstructured datasets.
Design and implement Slack’s Lakehouse solutions and enable production use cases for customers.
Improve the reliability and performance of the services/pipelines through AI assisted and/or Agentic solutions and tools.
Automate and handle the lifecycle of datasets (schema evolution, metadata management, change and backfill management, deprecation and migration).
Collaborate with cross-functional partners and lead technical initiatives independently from end to end.
Act as a role model and multiplier, coaching and mentoring other engineers across the team and the organization.
Write, review, or provide feedback on technical design proposals.