Associate Principal - Architecture

LTIMindtree • Dallas, TX

42d

About The Position

Experience 11 to 15 years Primary Skills Technical Expert Acts as the primary technical authority for Databricks platform-related projects Team Leader Leads a team of data engineers, data scientists and analysts working on Databricks based solutions Project Manager Coordinates project activities timelines and deliverables related to data engineering and analytics initiatives Stakeholder Liaison Communicates with business stakeholders to understand requirements and translate them into technical solutions Mentor Provides guidance and mentorship to team members on best practices coding standards and platform usage Responsibilities of a Databricks Lead Architecture Design scalable efficient and secure data pipelines and architectures on Databricks Development Implementation Develop and deploy data processing workflows ETLELT pipelines and machine learning models on Databricks Platform Management Manage and optimize Databricks environment including cluster configuration performance tuning and cost management Data Governance Security Ensure data security compliance and governance policies are implemented within the platform Collaboration Work closely with data scientists' analysts and business teams to deliver data driven solutions Code Review Quality Assurance Review code and workflows to ensure quality efficiency and adherence to standards Troubleshooting Support Resolve technical issues related to Databricks environment and data pipelines Documentation Reporting Maintain documentation of architecture processes and best practices generate reports on platform usage and performance Continuous Improvement Stay updated with the latest features and best practices in Databricks and big data technologies and incorporate them into projects Secondary Skills Experience in leading data engineering projects using Databricks including the types of data pipelines ETL processes and analytics solutions< Experience in optimizing Spark based workloads in Databricks for performance and cost efficiency< Experience in data quality security and governance within Databricks environments especially when working with largescale or sensitive data< Good communication with experience handling client stakeholders well

Requirements

• Technical Expert Acts as the primary technical authority for Databricks platform-related projects
• Team Leader Leads a team of data engineers, data scientists and analysts working on Databricks based solutions
• Project Manager Coordinates project activities timelines and deliverables related to data engineering and analytics initiatives
• Stakeholder Liaison Communicates with business stakeholders to understand requirements and translate them into technical solutions
• Mentor Provides guidance and mentorship to team members on best practices coding standards and platform usage
• Experience in leading data engineering projects using Databricks including the types of data pipelines ETL processes and analytics solutions
• Experience in optimizing Spark based workloads in Databricks for performance and cost efficiency
• Experience in data quality security and governance within Databricks environments especially when working with largescale or sensitive data
• Good communication with experience handling client stakeholders well
• GCP Data Architecture
• MDM Conceptual
• Data Architecture
• Data Lake house Architecture
• Dimensional Data Modeling
• Should have experience in data pipeline design using Databricks
• Should have experience in optimizing Spark jobs in Databricks for performance cost efficiency
• Experience in multitenant architecture design on Databricks

Responsibilities

• Design scalable efficient and secure data pipelines and architectures on Databricks
• Develop and deploy data processing workflows ETLELT pipelines and machine learning models on Databricks
• Manage and optimize Databricks environment including cluster configuration performance tuning and cost management
• Ensure data security compliance and governance policies are implemented within the platform
• Work closely with data scientists' analysts and business teams to deliver data driven solutions
• Review code and workflows to ensure quality efficiency and adherence to standards
• Resolve technical issues related to Databricks environment and data pipelines
• Maintain documentation of architecture processes and best practices generate reports on platform usage and performance
• Stay updated with the latest features and best practices in Databricks and big data technologies and incorporate them into projects

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Industry: Professional, Scientific, and Technical Services
• Education Level: No Education Listed