Databricks Architect

NTT DATA • Plano, TX

1d

About The Position

Solution Design & Implementation: Architect and implement scalable, secure, and high-performance data solutions on the Databricks platform. Design end-to-end data pipelines, ensuring seamless integration with Databricks and AWS services. Data Integration: Utilize Informatica for ETL processes, ensuring efficient data integration and transformation across various data sources. AWS Expertise: Leverage AWS services such as Glue for data cataloging and ETL, and Lambda for serverless computing to enhance data processing capabilities. Technical Leadership: Lead technical engagements, collaborating with internal teams and client stakeholders. Guide data engineers, analysts, and developers in implementing best practices on Databricks. Optimization & Performance: Optimize Databricks solutions for cost efficiency, performance, and reliability. Implement advanced performance tuning and debugging techniques. Governance & Compliance: Establish and enforce governance, security, and compliance standards within Databricks environments.

Requirements

• 8+ years of experience in data architecture, engineering, or analytics roles.
• 5+ years of experience with modern data architecture principles, including cloud platforms (AWS, Azure, GCP).
• 3+ years of recent hands-on experience with Databricks.
• Proficiency in SQL, Python, and ETL tools such as Informatica.
• Expert knowledge of Databricks architecture, including data sharing, performance optimization, and data security.
• Hands-on experience with data modeling, schema design, and ELT pipeline development.
• Understanding of cloud-native services for data ingestion, transformation, and orchestration.

Nice To Haves

• Certifications in Databricks such as Databricks Certified Professional Data Engineer.

Responsibilities

• Architect and implement scalable, secure, and high-performance data solutions on the Databricks platform.
• Design end-to-end data pipelines, ensuring seamless integration with Databricks and AWS services.
• Utilize Informatica for ETL processes, ensuring efficient data integration and transformation across various data sources.
• Leverage AWS services such as Glue for data cataloging and ETL, and Lambda for serverless computing to enhance data processing capabilities.
• Lead technical engagements, collaborating with internal teams and client stakeholders.
• Guide data engineers, analysts, and developers in implementing best practices on Databricks.
• Optimize Databricks solutions for cost efficiency, performance, and reliability.
• Implement advanced performance tuning and debugging techniques.
• Establish and enforce governance, security, and compliance standards within Databricks environments.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Education Level: No Education Listed