Sr. AI Product Data Engineer
About The Position
rPotential is building a platform that helps large companies understand where to apply AI, how work is changing, and where people can be redeployed to higher-value work. We work directly with Fortune 500 companies and leading AI companies. We're a small team, move quickly, and expect everyone to be hands-on. Our product depends on data. We combine proprietary, third-party, and AI-generated datasets in Databricks and Postgres to power customer-facing products. We have production pipelines today, but they were built quickly and vary in how they are structured, tested, monitored, and operated. We're looking for a senior data engineer to take ownership of this layer, improve what exists, and establish a consistent way to build and run data pipelines going forward. This role also requires understanding the business context behind the data. You'll need to understand what the data represents, where it came from, where it can be misleading, and how it is used in the product.
Requirements
- 5+ years of data engineering experience with production pipeline ownership.
- Strong DataBricks and Unity Catalog experience, including workspace administration.
- Strong Python and SQL.
- Experience improving an existing data environment while it remained live.
- Strong business judgment around data and an interest in understanding what the data actually means.
- Comfortable working through ambiguous or messy data with product and business teams rather than waiting for a finished spec.
- Comfortable with AI coding tools
- Bay Area based or commutable and comfortable working in person three days per week.
Nice To Haves
- Azure, terraform, Postgres
- Entity resolution or identity matching.
- Labor-market, company, people, or other large third-party datasets.
- Data systems supporting AI or LLM products.
Responsibilities
- Own and improve our production data pipelines.
- Standardize pipeline structure, scheduling, retries, testing, monitoring, and backfills.
- Build monitoring around freshness, coverage, failures, and data quality.
- Own Databricks across jobs, Unity Catalog, permissions, environments, and cost.
- Work in our product monorepo alongside the engineering team.
- Work with product and business teams to understand new datasets and resolve data issues.
- Make it easier to take new data sources from prototype to production.
Benefits
- High ownership, direct work with the founder and engineering team, and the opportunity to define how data engineering is done at rPotential as the team grows.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
Education Level
No Education Listed
