Data Lake Studio Manager
About The Position
eBay is seeking a proactive, technically skilled Data Lake Studio Manager to oversee and expand its Data Lake Studio workspace. The platform helps data engineers, analysts, and business teams transfer data across storage systems and coordinate complex data pipelines. The manager will be responsible for the Studio platform’s operational success, feature adoption, and reliability. This includes enabling users to configure connectors for cross-storage transfers; building and scheduling automated ETL/ELT workflows; monitoring job execution; and providing transparency into data movement across cloud, on-premises, and object storage systems. The role also owns customer enablement and platform operations, cost optimization, pipeline observability, and oversight of security and governance for cross-storage data movement.
Requirements
- Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Systems, or a related field, or equivalent experience.
- At least 5 years of hands-on experience in Data Engineering, Data Platform Management, or Cloud Data Operations.
- At least 2 years in a team lead, product owner, or manager capacity.
- Demonstrated proficiency with storage architectures such as S3, ADLS, GCS, HDFS, and SFTP, and tools or frameworks for high-throughput data transfer.
- Extensive hands-on experience with enterprise orchestration tools such as Apache Airflow, Prefect, Dagster, Temporal, or cloud-native workflow managers.
- Experience with distributed computing frameworks such as Apache Spark or Apache Flink.
- Knowledge of open table formats such as Apache Iceberg, Delta Lake, or Apache Hudi, and metadata catalog integration.
- Strong proficiency in SQL and Python.
- Strong track record of improving developer or user experience for self-service analytics platforms.
- Excellent communication skills for working with software engineers and business collaborators.
- Hands-on experience with SLAs/SLOs, monitoring tools such as Prometheus, Grafana, or Datadog, and DataOps methodologies.
Responsibilities
- Direct the Studio’s data movement suite to maintain efficiency, low latency, and reliability for batch and streaming transfers across storage platforms.
- Coordinate connector connectivity, throughput efficiency, error handling, and schema evolution for cross-storage migrations and replication.
- Drive self-service ingestion frameworks so customers can register data sources and targets without manual infrastructure setup.
- Own the operational health and user experience of the Studio pipeline orchestration engine, including scheduling, event-driven triggers, and dependency management.
- Define standards and guidelines for customer pipeline development, including fault tolerance, automatic retries, and backfilling.
- Maintain pipeline observability through monitoring dashboards, real-time logging, and configurable alerts for failures and bottlenecks.
- Improve Studio user onboarding, documentation, templates, and self-service workflows.
- Monitor compute and network costs for data transfers and processing workloads, and establish prioritization quotas and cost-optimization practices.
- Troubleshoot customer and partner concerns related to pipeline failures, lockouts, and network efficiency bottlenecks between storage tiers.
- Ensure cross-storage data movements comply with organizational security policies, encryption standards, and compliance frameworks.
- Coordinate identity management, credential mapping, and role-based access control across external storage endpoints and Studio pipeline tasks.
Benefits
- Target bonus and restricted stock units, as applicable.
- Medical, financial, and other benefits.
- 401(k) eligibility.
- Paid time off, including PTO and parental leave.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Manager
