Sr Databricks Data Engineer
About The Position
Join Deloitte’s AI & Engineering practice as a Databricks Engineer in its AI & Data practice. Design, build, and optimize cloud-based data engineering solutions for large-scale transformation. Work with business and technology leaders to modernize data platforms, enable analytics and artificial intelligence use cases, and deliver measurable business outcomes. The role focuses on enterprise-scale Databricks solutions that improve performance, expand digital capabilities, and help clients scale data-driven decision-making. As a Data Engineer III on the AI & Data team, you will lead data engineering practices and initiatives, own data pipelines and architectures, advance data quality and operational efficiency, and guide teams and technology adoption. You will also advise on governance, security, and compliance for cloud data ecosystems, communicate with diverse stakeholders, and oversee CI/CD and automation practices. The Core AI & Data practice helps organizations modernize data platforms, strengthen enterprise data foundations, and scale analytics and AI capabilities. Teams architect, engineer, and deploy cloud-based data solutions, working across business and technology functions on data modernization, governance, platform engineering, and insight delivery. Recruiting for this role ends on 1/27/2027. The role may require travel averaging 50%, depending on the work, clients, and industries served.
Requirements
- Bachelor’s degree in Computer Science, Engineering, or a related field.
- At least 5 years of hands-on data engineering experience focused on Databricks on AWS, Microsoft Azure, or GCP.
- Experience with Lakehouse architecture, Apache Spark, Delta Lake, cloud-native databases, storage solutions, and distributed compute platforms.
- Experience with data warehousing, third normal form (3NF), dimensional modeling, enterprise data lakes, incremental data loads, and metadata-driven ingestion and data quality frameworks using PySpark.
- At least 1 year leading complex, cross-functional data projects and technical teams.
- Experience with Delta Live Tables, Autoloader, Structured Streaming, Databricks Workflows, Apache Airflow, Unity Catalog, automated CI/CD pipelines, and performance optimization of data engineering pipelines, code, and compute resources.
- Ability to travel an average of 50%, based on work, clients, and industries served.
- Ability to work independently and collaboratively.
- Effective written and verbal communication skills.
- Meticulous attention to detail and quality of work product.
- Ability to build and sustain professional relationships.
- Ability to lead projects or workstreams.
- Ability to manage and prioritize multiple tasks in a fast-paced, dynamic environment.
- Strong interpersonal skills and professional demeanor.
- Ability to meet deadlines.
Nice To Haves
- Master’s degree in Computer Science, Engineering, or a related field.
- Experience in one or more of the AWS, Azure, and GCP cloud ecosystems and associated big data services.
- Experience tuning and optimizing performance in Databricks and Apache Spark environments.
- Experience with Databricks Lakeflow.
- Experience with artificial intelligence and machine learning solutions.
Responsibilities
- Establish, document, and promote best practices for data architecture, integration, and modeling.
- Oversee the design, development, and maintenance of robust data pipelines and architectures for large-scale enterprise data needs.
- Initiate and manage efforts to improve data quality, operational efficiency, and process scalability.
- Evaluate, pilot, and integrate big data and analytics technologies.
- Lead, coach, and develop teams of data engineers and architects to support technical growth and effective project delivery.
- Consult on, design, and implement governance, security, and compliance strategies for modern cloud data ecosystems.
- Communicate technical concepts and business value to executives, business leads, and technology teams.
- Oversee CI/CD implementation using tools such as Azure DevOps, AWS Code Pipeline, Jenkins, TFS, or PowerShell.
- Lead complex cross-functional data projects and technical teams.
- Deliver and optimize data engineering pipelines, code, and compute resources.
Benefits
- Broad range of employee benefits.
- Potential eligibility to participate in a discretionary annual incentive program, subject to program rules and applicable factors.
- Professional development opportunities, including on-the-job learning, formal development programs, mentorship, and leadership opportunities.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
