Principal Data Platform Developer
About The Position
Builds and runs CLEAResult’s Databricks data platform, covering source ingestion, governed silver and gold layers, analytics data products, and AI-assisted pipelines. The role includes maintaining platform infrastructure and workflows, enforcing data governance, developing document-processing and retrieval pipelines, supporting reporting needs, evaluating LLM serving endpoints, and troubleshooting and documenting pipeline issues. The position requires production Databricks data engineering experience and collaboration with technical and business stakeholders. A bachelor’s degree in computer science, engineering, or a related field is required, or equivalent experience. Applicants must be authorized to work in the country where the position is located without employer sponsorship or support, and successful hires must pass pre-employment checks. If applicable, the employee must meet DOT qualification requirements and comply with applicable transportation regulations.
Requirements
- At least 3 years of data engineering experience, including hands-on production experience with Databricks.
- Strong Python, SQL, and PySpark skills.
- Experience with Delta Lake and Delta Live Tables/Lakeflow.
- Experience with Unity Catalog governance, including grants, tagging, column masking, and row filters.
- Experience deploying data infrastructure as code, such as Databricks Asset Bundles or Terraform, with Git-based CI/CD.
- Familiarity with Salesforce data models and CRM ingestion patterns.
- Working knowledge of Azure, including identity, service principals, and storage.
- Bachelor’s degree in computer science, engineering, or a related field, or equivalent experience.
- Clear communication and documentation skills, and ability to work with technical and business stakeholders.
- If applicable, meet all DOT qualification requirements and comply with applicable federal, state, and local transportation regulations.
- Must be authorized to work in the country where the position is located without employer sponsorship or support.
- Must pass pre-employment checks.
Nice To Haves
- Experience building or integrating LLM workflows, including prompting, retrieval/RAG, and model evaluation.
Responsibilities
- Build and maintain Databricks Asset Bundles for ingestion, transformation, data products, and orchestration jobs, following team standards and CI/CD practices for development and production using GitHub Actions and service principals.
- Run Salesforce Lakeflow Connect ingestion across client organizations, including sandbox re-authentication and field-level-security column-loss issues, while protecting PII-tagged silver tables.
- Enforce data governance through Unity Catalog grants, PII tagging, ABAC masking, and table-level access controls for sensitive intermediate data.
- Build document-processing and retrieval pipelines using foundation-model endpoints for classification, detection, and inference.
- Build reporting marts and data products to support business KPIs and stakeholder reporting.
- Evaluate and select LLM serving endpoints, including model bake-offs and migrations from deprecated endpoints.
- Debug failing pipelines and jobs, and document known defects and team runbooks.
Benefits
- Medical, dental, and vision insurance.
- Company-paid health care concierge service.
- 401(k) with company match.
- Paid vacation, sick, personal, and parental leave.
- Paid volunteer time.
- Employee Recognition Program with points redeemable for gift cards.
- Employee Assistance Program.
- Access to on-demand training courses.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Principal
