Principal Engineer, Data Engineering
Ripple
·
Posted:
April 27, 2023
·
Remote
About the position
This senior role at Ripple involves architecting and implementing data infrastructure for analytics and data-centric product features. The successful candidate will be responsible for creating a complete data platform, including data ingestion, processing, and governance. They will also play a key role in educating the company on the technology being developed and shipped. The role requires strong architectural vision, coding skills, communication and leadership abilities, and expertise in data ingestion and pipelining. Additionally, the candidate will collaborate with other teams to ensure the scalability, performance, and security of Ripple's data platforms.
Responsibilities
- Architect and implement the data infrastructure for analytics and data-centric product features at Ripple
- Create a complete data platform for unified data ingestion, distributed systems for processing, self-serve data lakes, and batch/stream ETL data pipeline for golden datasets for analytics
- Demonstrate thoughtfulness and curiosity in data ingestion, generation, and pipelining, as well as governance and security of data at Ripple
- Have a clear architectural vision and the ability to rapidly code and ship products
- Possess excellent communication and leadership skills
- Educate the firm on the technology being developed and shipped
- Represent Ripples Data Platform and Engineering as an expert member, participating in conversations and architectural discussions
- Architect, design, implement, and manage data platforms for Ripple on various hyper-scale platforms (AWS, GCP)
- Translate data needs into critical information for implementing scalable data platforms and self-service tools
- Provide technical input to Data Governance policies, standards, and processes related to data classification, ownership, access, and security
- Collaborate with service teams and other engineering and business partners on Data Infrastructure and Engineering roadmap planning
- Focus on observability for all database monitoring and implement auto remediation techniques
- Partner with service and performance teams for continuous architecture improvements, resiliency, and performance
- Own the delivery, quality, and reliability of the Financial Data Hub
- Develop data migration architecture and strategy for data migration across clouds
- Design, develop, and manage enterprise-level database systems with a focus on high-availability, clustering, cloud migration, security, performance, and scalability
- Design and implement a scalable data lake, including data integration and curation
- Build modular set of data services using Python/Scala, BigQuery/Presto SQL, API Gateway, Kafka, Apache Spark, etc.
- Deep knowledge in Data Warehouse architecture and integration
- Research, design, and experiment to execute fast proof of concepts
- Participate in the strategic development of methods, techniques, and evaluation criteria for projects and programs
- Work autonomously and take ownership of projects
- Create data applications with the ability to do searches, real-time data alerts, and APIs for large volumes of data
- Design and implement innovative data services solutions using Microservices and other UI and API related technologies
- Implement processes and systems to manage data quality
- Write unit/integration tests, contribute to engineering wiki, and document work
- Work closely with a team of frontend and backend engineers, product managers, and analysts
- Coach other engineers on best practices for designing and operating reliable systems at scale
- Design data integrations and data quality framework
- Execute the migration of data and processes from legacy systems
Requirements
- At least 12+ years' experience in designing and developing enterprise data architecture and engineering solutions that have supported massive workloads and data scale/volume
- Experience working with private and public clouds (AWS, GCP) and capacity management principles
- Design and implement a scalable data lake, including data integration and curation
- Build modular set of data services using Python/Scala, BigQuery/Presto SQL, API Gateway, Kafka, Apache Spark on EMR/data proc among others
- Deep knowledge in Data Warehouse architecture and integration
- Research, design, and experiment to execute fast proof of concepts to evaluate similar products
- Experience working autonomously and taking ownership of projects
- Create data applications with ability to do searches, real-time data alerts, APIs to pull the data on a large volume of data
- Design and implement innovative data services solutions using Microservices and other UI and API related technologies
- Implement processes and systems to manage data quality, ensuring production data is always accurate and available for key partners and business processes that depend on it
- Writes unit/integration tests, contributes to engineering wiki, and documents work
- Work closely with a team of frontend and backend engineers, product managers, and analysts
- Coaching other engineers on best practices for designing and operating reliable systems at scale
- Design data integrations and data quality framework
- Execute the migration of data and processes from legacy systems
Benefits
- Competitive salary, bonuses, and equity
- Competitive benefits that cover physical and mental healthcare, retirement, family forming, and family support
- Employee giving match
- Mobile phone stipend
- Twice a quarter R&R days for rest and recharge
- Generous wellness reimbursement and weekly onsite & virtual programming
- Generous vacation policy
- Industry-leading parental leave policies and family planning benefits
- Catered lunches and fully-stocked kitchens with premium snacks/beverages
- Team offsites, team bonding activities, happy hours, and more