Staff Software Engineer, Data Catalog
About The Position
Airbnb is seeking a Staff Software Engineer to join its Data Infrastructure organization and lead development of data catalog infrastructure products across Airbnb’s Data Ecosystem. The role focuses on building infrastructure, tools, standards, and processes to ensure that data assets are cataloged, discoverable, and governed. The engineer will develop products for data discovery, metadata management, data quality, and lineage, as well as infrastructure for governance processes such as ownership management, classification, privacy, access control, and retention. The role includes building end-to-end data lineage across online and offline data, machine learning, metrics, visualizations, and third-party datasets; partnering with cross-functional and data infrastructure teams; and establishing scalable metadata onboarding, integrations, policies, and governance practices. The position is US remote eligible. It may involve occasional work at an Airbnb office or attendance at offsites, as agreed with the manager. Employees must live in a state where Airbnb, Inc. has a registered entity.
Requirements
- BS, MS, or PhD in Computer Science or a related field, or equivalent work experience.
- 9 or more years of software engineering experience, focused on data infrastructure.
- Experience with data storage and distributed processing technologies such as Hive, Spark, Trino, Flink, or SQL databases.
- Strong programming skills in one or more of Java, Python, or Scala.
- Experience with data catalog products, including metadata infrastructure, metadata management, metadata integration frameworks and tooling, and metadata-driven data management.
- Experience with workflow orchestration solutions such as Apache Airflow, Prefect, or Kubeflow.
- Strong knowledge of data governance frameworks, policies, and processes, including data classification, data lineage, data quality, data privacy, and retention.
- Excellent communication skills and ability to collaborate with cross-functional teams to build empathetic tools and platforms.
- Ability to work in areas outside of one's usual comfort zone.
- Excellent analytical and problem-solving skills.
Responsibilities
- Develop and implement data catalog products supporting data discovery, metadata management, data quality, and data lineage.
- Develop and implement metadata infrastructure for data governance, including ownership management, data classification, data privacy, access control, and data retention.
- Build an end-to-end data lineage product across online and offline data, machine learning, metrics, visualizations, and third-party datasets.
- Collaborate with cross-functional teams to ensure datasets are cataloged, governed, and aligned with business goals and regulatory requirements.
- Conduct programs to define and ensure metadata quality and metadata governance requirements.
- Work with data infrastructure framework teams to identify, design, and build metadata integrations for data systems.
- Build infrastructure tools and processes for scalable metadata onboarding and integrations.
- Develop and implement metadata-driven data policies and procedures to support data governance requirements.
Benefits
- The role may be eligible for a bonus, equity, benefits, and Employee Travel Credits.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
