Professional, Sr. Databricks Platform Engineer

MVP Health Care•Schenectady, NY

1d•$121,767 - $161,950•Hybrid

About The Position

Join Us in Shaping the Future of Health Care At MVP Health Care, we’re on a mission to create a healthier future for everyone. That means embracing innovation, championing equity, and continuously improving how we serve our communities. Our team is powered by people who are curious, humble, and committed to making a difference—every interaction, every day. We’ve been putting people first for over 40 years, offering high-quality health plans across New York and Vermont and partnering with forward-thinking organizations to deliver more personalized, equitable, and accessible care. As a not-for-profit, we invest in what matters most: our customers, our communities, and our team. What’s in it for you: Growth opportunities to uplevel your career A people-centric culture embracing and celebrating diverse perspectives, backgrounds, and experiences within our team Competitive compensation and comprehensive benefits focused on well-being An opportunity to shape the future of health care by joining a team recognized as a Best Place to Work For in the NY Capital District, one of the Best Companies to Work For in New York, and an Inclusive Workplace. You’ll contribute to our humble pursuit of excellence by bringing curiosity to spark innovation, humility to collaborate as a team, and a deep commitment to being the difference for our customers. Your role will reflect our shared goal of enhancing health care delivery and building healthier, more vibrant communities.

Requirements

Bachelor’s degree in Computer Science, Engineering, Information Systems, or equivalent experience.
The availability to work full-time, hybrid
5+ years in data engineering, platform engineering, or cloud data operations.
3+ years hands-on Azure Databricks experience in production.
3+ years cloud (Azure) data platform engineering experience.
Curiosity to foster innovation and pave the way for growth
Humility to play as a team
Commitment to being the difference for our customers in every interaction

Responsibilities

Workspace Administration (Dev/Test/Prod): Administer, standardize, and optimize Azure Databricks workspaces across Dev/Test/Prod, including clusters, jobs, libraries, pools, policies, and secrets.
Platform Standards & Root-Cause Analysis: Define, implement, and enforce platform standards and guardrails; lead advanced troubleshooting and root-cause analysis in partnership with data, cloud, and security teams.
Spark Performance & Cost Optimization: Tune Spark workloads and cluster configurations to maximize performance and minimize cost across batch and streaming pipelines; develop visibility into compute consumption and efficiency trends.
Governance (Unity Catalog): Operate and enhance Unity Catalog governance, including secure access, lineage, auditing, and compliance alignment with enterprise and regulatory requirements.
Security & Identity Integration: Partner with Security and Cloud teams to implement Azure networking patterns, Azure Active Directory (AAD) integration, and workspace isolation controls.
DevOps (CI/CD & Git): Design, build, and maintain CI/CD pipelines and Git-based workflows for Databricks notebooks, jobs, Delta Live Tables, and Unity Catalog artifacts.
Infrastructure as Code (Terraform): Provision and manage Databricks infrastructure using Terraform (IaC) to enable consistent SDLC promotion, repeatability, and scalable operations.
Lakehouse Enablement & Reusable Patterns: Develop and promote lakehouse platform patterns, reusable frameworks, and standardized orchestration for batch and streaming data pipelines.
Observability & Reliability Engineering: Implement observability and reliability practices, including monitoring, dashboards, alerting, SLIs/SLOs, and incident response to improve platform stability and recoverability.
Technical Leadership & Continuous Improvement: Mentor engineers and document standards while staying current with Databricks and Azure capabilities; drive continuous improvement through pragmatic enhancements and best practices.
Contribute to our humble pursuit of excellence by performing various responsibilities that may arise, reflecting our collective goal of enhancing healthcare delivery and being the difference for the customer.