Databricks Architect
About The Position
CTG is seeking to fill a Databricks Architect position for our client. This role involves designing, developing, and implementing scalable data and analytics platforms using Databricks and modern cloud technologies. The architect will provide technical leadership in architecting end-to-end data solutions supporting data engineering, analytics, AI/ML, and business intelligence use cases. Responsibilities include translating business and IT requirements into modern data architecture components, leading the design and optimization of distributed data processing solutions leveraging Apache Spark and Delta Lake, and collaborating with various teams to deliver secure, reliable, and high-performance solutions. The role also involves defining best practices for data integration, ETL/ELT pipelines, orchestration, and data modeling, ensuring robust implementation of data security, governance, and compliance. Performance tuning, cluster configuration optimization, cost management, and supporting DevOps/CI/CD for data platforms are also key aspects. Additionally, the architect will contribute to modernization initiatives, including legacy system analysis, code restructuring, refactoring, and enabling integration of legacy and modern systems using cloud-native services and APIs.
Requirements
- Strong experience designing and implementing data platforms using Databricks
- Familiarity with AI and Machine Learning
- Deep knowledge of Apache Spark, Delta Lake, and distributed data processing concepts
- Proficiency in Python, Scala, or SQL for data engineering and analytics
- Experience with cloud platforms such as Azure (ADF, ADLS), AWS (S3, Glue), or GCP (GCS, BigQuery)
- Hands-on experience with data integration, ETL/ELT frameworks, and orchestration tools
- Strong understanding of data security, governance, and access control in cloud environments
- Experience with performance tuning, cluster configuration, and cost optimization
- Familiarity with DevOps and CI/CD practices for data platforms
- Ability to communicate complex technical concepts to non-technical stakeholders
- Experience with legacy modernization, code refactoring, and integration of legacy systems
- 8+ years of experience in data engineering, data architecture, or analytics engineering roles
- 3+ years of hands-on experience with Databricks in production environments
- Proven experience architecting cloud-based data platforms at scale
- Demonstrated experience working on AI/ML-enabled data solutions is required
- Experience leading technical teams or acting as a lead architect in enterprise environments
- Bachelor’s degree in Computer Science, Information Systems, Engineering, or a related field, or equivalent practical experience
- Excellent verbal and written English communication skills and the ability to interact professionally with a diverse group are required.
Responsibilities
- Design, develop, and implement scalable data and analytics platforms using Databricks and modern cloud technologies
- Provide technical leadership in architecting end-to-end data solutions supporting data engineering, analytics, AI/ML, and business intelligence use cases
- Translate business and IT requirements into modern data architecture components using established modernization frameworks
- Lead the design and optimization of distributed data processing solutions leveraging Apache Spark and Delta Lake
- Collaborate with data engineers, data scientists, platform engineers, and business stakeholders to deliver secure, reliable, and high-performance solutions
- Define and enforce best practices for data integration, ETL/ELT pipelines, orchestration, and data modeling
- Ensure robust implementation of data security, governance, access control, and compliance within cloud environments
- Perform performance tuning, cluster configuration optimization, and cost management across Databricks environments
- Support DevOps and CI/CD implementation for data platforms and analytics workloads
- Contribute to modernization initiatives including legacy system analysis, code restructuring, refactoring, and business logic extraction into reusable components
- Enable integration of legacy and modern systems using cloud-native services and APIs for reuse by systems of engagement
- Reuse and enhance digital modernization assets, methods, and collateral
- Work with cloud-native services such as AWS ECS, AWS Lambda, ElasticCache, and S3 as applicable
Benefits
- competitive benefit package
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
Number of Employees
501-1,000 employees
