Staff Engineer, Databricks Platform

ModMed • Boca Raton, FL

2d

About The Position

ModMed is seeking a hands-on Senior/Staff Data Platform Engineer to own and evolve its established Databricks platform on AWS. The platform supports mission-critical data, analytics, and AI workloads using regulated healthcare data across multiple teams and products. This is an individual-contributor technical leadership role reporting to the Director of Data Platform Engineering, with no direct reports initially. It focuses on engineering shared platform foundations, guardrails, automation, observability, and self-service capabilities for Data Engineering, Analytics, ML Engineering, AI Engineering, and MLOps—not on owning application data engineering, data science, or MLOps work.

The role owns Databricks account- and workspace-level capabilities across development, test, and production; infrastructure as code and CI/CD; Databricks-specific AWS integrations and approved security controls; platform reliability, observability, cost management, and recovery; and reusable enablement patterns for data and AI teams. It partners with Cloud Platform, Cybersecurity, Data Governance, data owners, and workload teams, while respecting their ownership of enterprise AWS foundations, policies, business access approvals, workload code, models, and releases.

Success includes reproducible, reviewed, version-controlled platform configuration; auditable least-privilege access and production changes; measurable improvement in reliability, configuration drift, manual effort, and unit cost; documented self-service workload deployment patterns; and visible, forecastable, optimized Databricks costs and commitment utilization.

Requirements

• Demonstrated ownership of Databricks on AWS in a multi-team production environment, including account- and workspace-level administration.
• Strong experience with Unity Catalog, identity federation, service principals, compute policies, secure data-access patterns, lineage, and auditability.
• Deep production experience with Terraform and IaC practices, including modular infrastructure built with the Databricks and AWS providers, state management, source control, automated testing, and CI/CD.
• Strong AWS knowledge across IAM, VPC networking, private connectivity, S3, KMS, security groups, logging, and cross-account access.
• Experience operating cloud data platforms containing regulated, confidential, or highly sensitive data.
• Experience with observability, incident response, change management, capacity planning, and cloud cost optimization.
• Experience leading a significant platform migration, workspace or metastore consolidation, runtime upgrade, or comparable modernization under production load.
• Working knowledge of Spark and distributed computing sufficient to diagnose platform and workload-performance issues.
• Proficiency in Python for automation, plus practical shell scripting and REST API integration.
• Strong communication and influence skills, with demonstrated success establishing standards and self-service guardrails across organizational boundaries.

Nice To Haves

• Experience operating healthcare data platforms and implementing HIPAA-aligned controls for PHI.
• Experience with MLflow, Models in Unity Catalog, model serving, GPU workloads, feature engineering platforms, or GenAI infrastructure.
• Experience with multi-workspace, multi-account, or multi-region architectures and disaster-recovery testing.
• Databricks and/or AWS professional certification; certification is valuable but is not a substitute for production platform ownership.

Responsibilities

• Own Databricks account- and workspace-level capabilities across development, test, and production environments.
• Automate workspace provisioning and configuration, including identity federation, SSO/SCIM integration, service principals, workspace assignments, entitlements, and administrative controls.
• Administer Unity Catalog, including metastores, workspace bindings, storage credentials, external locations, lineage, auditing, and implementation of the access model defined by Data Governance and data owners.
• Define approved patterns for serverless and classic compute, jobs, interactive compute, SQL warehouses, and specialized compute when required.
• Maintain compute policies, runtime and library standards, tagging, auto-termination, quotas, compatibility testing, and LTS/deprecation plans while balancing security, performance, reliability, and cost.
• Own the Databricks platform IaC strategy and implementation, including reusable Terraform modules using the Databricks and AWS providers.
• Manage supported account, workspace, Unity Catalog, compute, and AWS integration configurations through code, with environment isolation, automated plans and tests, approval controls, and configuration-drift detection and remediation.
• Build tested CI/CD workflows for account, workspace, Unity Catalog, and shared-platform changes, including peer review, approvals, controlled promotion, and rollback.
• Provide reusable Declarative Automation Bundle templates and secure deployment patterns; workload teams remain responsible for their code and releases.
• Partner with Cloud Platform to automate Databricks-specific networking, private connectivity, IAM, S3, KMS, DNS, secrets management, and connectivity with services such as Aurora and Kafka, while consuming enterprise AWS guardrails.
• Implement approved controls for PHI and other sensitive data, including encryption, network isolation and egress, privileged access, logging, approval workflows, access reviews, and break-glass procedures.
• Ensure privileged and account-level changes are reviewed and approved by designated Cloud Platform, Cybersecurity, or Data Governance partners according to the affected control.
• Support security reviews, audit evidence, risk remediation, and production change-management requirements.
• Establish platform service levels, health indicators, alerts, dashboards, escalation paths, and runbooks using Databricks system tables, audit logs, and CloudWatch.
• Participate in a shared on-call and escalation rotation; lead diagnosis, root-cause analysis, and preventive action for complex Databricks and AWS integration incidents.
• Implement cost attribution, budgets, tagging, showback reporting, and optimization recommendations across teams and workloads.
• Track Databricks consumption against commercial commitments, forecast capacity, and provide technical analysis for planning and renewal decisions.
• Own Databricks recovery patterns and testing with Cloud Platform and workload owners, aligned with agreed recovery objectives.
• Build paved-road templates, standards, documentation, and support mechanisms that increase self-service and reduce one-off administrative requests.
• Partner with Data Engineering, Analytics, ML Engineering, AI Engineering, and MLOps to translate workload requirements into supported platform capabilities.
• Configure shared MLflow and Models in Unity Catalog capabilities and enable approved training, inference, and model-serving patterns as adopted.
• Troubleshoot platform issues without assuming ownership of pipeline logic, model development, evaluations, AI application behavior, or workload releases.
• Establish measurable baselines within the first 90 days for platform reliability, configuration drift, manual administrative effort, and unit cost, and demonstrate sustained improvement.

Benefits

• Comprehensive medical, dental, and vision benefits, including a company Health Savings Account contribution.
• 401(k) with a matching contribution each payday of 50% of the employee’s contribution deferred on up to 6% of compensation; matching contributions vest after one year of employment.
• Generous paid time off and paid parental leave programs.
• Company-paid life and disability benefits.
• Flexible Spending Account and Employee Assistance Programs.
• Company-sponsored Business Resource & Special Interest Groups.
• Professional development opportunities, including tuition reimbursement programs and unlimited access to LinkedIn Learning.
• Global presence and in-person collaboration opportunities; dog-friendly U.S. headquarters; hybrid office-based roles and remote availability for some roles.
• Weekly catered breakfast and lunch, treadmill workstations, Zen, and wellness rooms within the BRIC headquarters.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed