Senior Data Engineer
About The Position
Cast & Crew is seeking an experienced Senior Data Engineer to design, build, and operate scalable data platforms that power mission-critical applications, analytics, reporting, and AI-driven solutions across the entertainment production ecosystem. This role focuses on building reliable, cloud-native data pipelines that ingest, transform, secure, and serve operational data for internal engineering teams, business intelligence, and customer-facing applications. You will work across structured and semi-structured data, enabling modern analytics, AI workloads, and real-time event processing. You will collaborate closely with software engineers, platform engineers, product teams, and data consumers while leveraging AI-assisted development tools to accelerate delivery and improve engineering productivity.
Requirements
- Bachelor's degree in Computer Science, Software Engineering, Information Systems, or related field
- 5+ years of experience building production data platforms and pipelines
- Strong Python development experience
- Strong SQL development and query optimization skills
- Experience designing and maintaining ETL/ELT pipelines
- Experience working with relational databases (PostgreSQL, MySQL, Aurora)
- Experience working with NoSQL databases (DynamoDB, MongoDB, document databases)
- Experience building data lakes and cloud-native data architectures
- Experience with Snowflake, BigQuery, Redshift, or similar cloud data warehouses
- Experience with AWS cloud services including S3, Lambda, EventBridge, DynamoDB, ECS/EKS, and IAM
- Experience building event-driven and streaming data solutions
- Experience working with Kafka, SNS/SQS, Kinesis, or similar messaging technologies
- Experience with data orchestration tools such as Airflow, Prefect, or Dagster
- Experience with Git and modern CI/CD pipelines (Azure DevOps, GitHub Actions, or similar)
- Experience building highly available, scalable production systems
- Strong understanding of data modeling, partitioning, indexing, and performance optimization
- Experience monitoring and operating production data platforms
Nice To Haves
- Experience with Apache Spark or distributed processing frameworks
- Experience with Debezium, CDC, or change data capture architectures
- Experience with Iceberg, Delta Lake, or modern lakehouse technologies
- Experience with OpenSearch or Elasticsearch
- Experience supporting AI and machine learning data platforms
- Experience building feature stores or vector databases
- Experience with document processing (OCR, Textract, PDF extraction)
- Experience in media, payroll, financial systems, or enterprise SaaS platforms
- Experience using AI-assisted development tools such as Claude Code or GitHub Copilot
Responsibilities
- Design, build, and operate scalable data platforms
- Build reliable, cloud-native data pipelines that ingest, transform, secure, and serve operational data
- Work across structured and semi-structured data
- Enable modern analytics, AI workloads, and real-time event processing
- Collaborate closely with software engineers, platform engineers, product teams, and data consumers
- Leverage AI-assisted development tools to accelerate delivery and improve engineering productivity
- Build enterprise-scale data ingestion and transformation pipelines
- Build cloud-native data platforms supporting analytics and operational workloads
- Build real-time event-driven data processing solutions
- Build data models supporting reporting, AI, and customer-facing applications
- Build secure, scalable integrations between enterprise systems
- Build automated monitoring, testing, and deployment pipelines for data infrastructure
Benefits
- Medical
- Dental
- Vision
- PTO
- Health and wellness programs
- Employee discounts
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
