Databricks Lead (Staff IT Engineer)
About The Position
The Databricks Lead (Staff IT Engineer) is a hands-on technical leader responsible for designing, building, and operating enterprise-scale Databricks solutions. This role drives platform adoption, AI-native development practices, machine learning enablement, AIOps automation, and cloud modernization initiatives. The individual will lead technical delivery, mentor engineers, establish engineering standards, and ensure highly scalable, secure, and intelligent data and AI platforms. This role requires full-time onsite work in San Diego, CA or Boulder, CO (5 days per week). This role is not eligible for Qualcomm immigration sponsorship.
Requirements
- 8–12 years of overall IT experience.
- 4–5 years of hands-on Databricks experience.
- Strong experience implementing enterprise cloud and AI platforms.
- Databricks Platform
- Delta Lake
- Unity Catalog
- MLflow
- Mosaic AI
- Python, SQL, Spark
- Databricks Workflows
- REST APIs
- GitHub
- CI/CD Automation
- AIOps and Observability Platforms
- Generative AI tools and frameworks
- Technical mentoring
- Problem solving and innovation
- Cross-functional collaboration
- Continuous improvement mindset
- Strong communication skills
- 4+ years of IT-related work experience with a Bachelor's degree.
- 7+ years of IT-related work experience without a Bachelor’s degree.
- Completed advanced degrees in a relevant field may be substituted for up to two years (Master’s = one year, Doctorate = two years) of work experience.
Responsibilities
- Design and implement scalable Lakehouse architectures using Databricks, Delta Lake, Unity Catalog, Workflows, and Serverless capabilities.
- Develop reusable platform frameworks, accelerators, and automation capabilities.
- Optimize platform performance, reliability, scalability, and cost efficiency.
- Drive platform modernization and cloud-native engineering best practices.
- Promote AI-assisted software engineering using Claude Code, GenAI, and automation frameworks.
- Develop AI-powered engineering workflows to improve developer productivity and quality.
- Implement intelligent code generation, testing automation, root-cause analysis, and deployment optimization.
- Establish best practices for AI-enabled SDLC and platform operations.
- Build and operationalize machine learning solutions using MLflow, Mosaic AI, Feature Engineering, and Model Serving.
- Support predictive analytics, anomaly detection, forecasting, and intelligent automation use cases.
- Implement model lifecycle management and MLOps practices.
- Collaborate with AI teams to enable enterprise AI workloads on Databricks.
- Implement observability, monitoring, and automated incident management solutions.
- Build AI-driven operational insights for proactive issue detection and remediation.
- Improve platform reliability through self-healing automation and intelligent alerting.
- Drive reduction in operational toil through automation.
- Lead technical design reviews and solution architecture discussions.
- Mentor engineers and establish coding, testing, and deployment standards.
- Drive engineering excellence through automation and reusable frameworks.
Benefits
- competitive annual discretionary bonus program
- opportunity for annual RSU grants
- highly competitive benefits package
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
