Principal Engineer, Big Data Platform

Pinterest • San Francisco, CA

3d • Remote

About The Position

Pinterest is looking for a principal software engineer to lead the next generation of data infrastructure at Pinterest, which powers mission-critical big data and AI applications. You will be working on exciting big data and AI open-source technologies (Flink, Spark, Kubernetes, etc.) at the scale of exabytes of data to help Pinners discover and do what they love. At Pinterest, AI is a powerful partner that augments creativity and amplifies impact. The role involves leading the strategy and technical direction of Pinterest’s data infrastructure for big data and AI applications, building and scaling data infra frameworks and infrastructure to process petabytes-scale datasets, including compute engines, job management, resource management, scheduling, and remote shuffling. You will also work with internal customers on critical business use cases that rely on big data and provide thought leadership to the entire company on how data should be processed and stored more reliably, quickly, and efficiently at scale. Contributing to the team’s technical vision and long-term roadmap is also a key aspect of this role.

Requirements

• 12+ years of industry experience with a proven track record of technical excellence
• 8+ years of experience of building and supporting large scalable Kubernetes or big data platform
• Deep knowledge of big data / ML technologies (e.g. Flink, Spark, Presto, Kubernetes, Ray, PyTorch/TensorFlow)
• Proficiency in one or more programming languages (Java, Go, Scala, Python)
• Experiences in Kubernetes and AWS technologies
• Exceptional collaboration skills with cross-functional partners, with the ability to navigate ambiguity, make tradeoffs, and keep stakeholders aligned on priorities and progress.
• Bachelor’s degree in Computer Science, a related technical field, or equivalent experience.

Responsibilities

• Lead the strategy and technical direction of Pinterest’s data infrastructure for big data and AI applications
• Build and scale data infra frameworks and infrastructure to process petabytes-scale datasets, including compute engines, job management, resource management, scheduling and remote shuffling
• Work with internal customers on critical business use cases that rely on big data
• Provide thought leadership to the entire company on how data should be processed and stored more reliably, quickly and efficiently at scale
• Contribute to the team’s technical vision and long-term roadmap

What This Job Offers

• Job Type: Full-time
• Career Level: Principal