Reddit-posted 4 months ago
$190,800 - $267,100/Yr
Full-time • Mid Level
1,001-5,000 employees

The Machine Learning Platform team at Reddit is a high-impact team that owns the infrastructure that powers recommendations, content discovery, user and content quantification, while directly impacting other teams such as Growth, Ads, Feeds, and Core Machine Learning teams. As a Senior Data Engineer, you will lead development of data pipelines and workflow for large scale ML models at Reddit. You will design and implement scalable and secure data processing pipelines and storage environments that prepare our source of truth datasets for our models.

  • Ensure data is cleansed, mapped, transformed, and otherwise optimized for storage and use according to business and technical requirements.
  • Build effective data pipelines and workflows to streamline data ingestion, processing, and distribution tasks.
  • Setting up and operating data workflow management tools for SQL code versioning, dependency tracing, etc.
  • Load transformed data into storage and reporting structures in destinations including data warehouse, reporting systems and analytics applications.
  • Monitor and troubleshoot issues with the data environment to maintain high availability and performance.
  • Support monitoring and observability across training datasets, model metrics and implement diagnostic tools for metric movements.
  • Maintain effective documentation regarding data procedures, systems, and architectures to maintain clarity and enable easy collaboration.
  • 5+ years of experience in Data Engineering or ML Infrastructure
  • Experience with large scale data transforms to prepare graph data
  • Experience with Graph DB, Spark, Kafka pipelines
  • Experience working with Airflow and MLFlow
  • Experience with storage frameworks like BQ, parquet, iceberg
  • Awareness of ML models and architectures is a huge plus.
  • Strong focus on scalability, reliability, performance, and ease of use.
  • Strong organizational & communication skills
  • Comprehensive Healthcare Benefits and Income Replacement Programs
  • 401k Match
  • Family Planning Support
  • Gender-Affirming Care
  • Mental Health & Coaching Benefits
  • Flexible Vacation & Reddit Global Days off
  • Generous paid Parental Leave
  • Paid Volunteer time off
© 2024 Teal Labs, Inc
Privacy PolicyTerms of Service