Engineering Manager, Ray Data
About The Position
Anyscale is seeking an engineering leader to manage the open source Ray Data team. Ray Data is a Python-native data processing engine for AI workloads on Ray, handling multimodal and structured data and powering batch inference, data loading for distributed training, and large-scale preprocessing. The team owns the open source library end to end, including its streaming execution engine; batch inference integrations; production training data loading; data source and format connectors; and performance, stability, and fault tolerance at scale. This is a first-line engineering management role. The manager will lead a team of engineers, remain involved in technical design and code, and own the team’s roadmap within the open source project. The role also involves working closely with Ray Data’s large, active community and external contributors, and collaborating across the Ray ecosystem and Anyscale to support the project’s growth and success.
Requirements
- 7+ years of software engineering experience, including 2+ years directly managing engineers; experience at a high-growth startup is ideal.
- A record of building high-performing distributed teams and helping them scale quickly while maintaining a good culture.
- Experience building and managing teams across geographic locations.
- Ability to maintain a high hiring bar, motivate the team, coach and mentor engineers, and handle performance management issues.
- Deep technical knowledge of distributed systems and data-intensive systems, such as data processing engines, query engines, or database internals.
- Hands-on depth in Python and performance-sensitive systems code sufficient to review designs and work through performance and stability problems with the team.
- A strong track record of execution, urgency, and excellent prioritization skills.
- Effective written and verbal communication with engineers, customers, and an open source community.
Nice To Haves
- Experience leading or maintaining a widely used open source project, including working with external contributors and project governance.
- Hands-on experience with Ray, Spark, Flink, Beam, or similar engines.
- Experience with Apache Arrow, columnar formats, or table formats such as Iceberg and Delta Lake.
- Experience with GPU workloads, LLM or multimodal batch inference, or data loading for large-scale model training.
- Experience operating data or ML infrastructure in production.
- Experience managing teams across US and India locations.
Responsibilities
- Set the technical direction and roadmap for open source Ray Data, balancing community needs, customer needs, and Anyscale’s product goals.
- Drive execution of the team’s key projects, from design review through release.
- Hire, coach, and grow a team of distributed systems and data infrastructure engineers.
- Manage day-to-day team operations, including planning, prioritization, project tracking, on-call, and release quality.
- Stay technically hands-on through design and code review, performance investigations, and escalations on critical production issues.
- Represent Ray Data in the open source community by working with external contributors and maintainers, reviewing Ray Enhancement Proposals, maintaining GitHub issue and pull request triage, and growing the contributor base.
- Partner with the Ray Core, Ray Train, Ray Serve, and Anyscale platform teams on cross-library features and end-to-end performance.
- Work with product management, field engineering, and customers to understand workloads and turn them into roadmap priorities.
- Build and maintain a team culture of collaboration, ownership, and high engineering standards.
- Communicate the team’s work through talks, tutorials, blog posts, and at Ray Summit.
- Help build and shape Anyscale as a company.
Benefits
- Equity awards for eligible employees.
- Comprehensive benefits package supporting health, well-being, and financial future.
- Health, dental, and vision coverage, with many plans up to 99% employer-covered.
- Flexible time off.
- Paid parental leave.
- Mental health support.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Manager
Education Level
No Education Listed
