Sr. Azure Data Architect – Databricks

Exavalu Solutions India Pvt Ltd • Dallas, TX

4d

About The Position

The Sr. Data Architect will design and implement end-to-end data architecture and ecosystems using Azure-native services and Databricks. The role builds scalable, high-performance data platforms and robust data frameworks, and partners with stakeholders to translate business requirements into reliable data solutions. It applies governance principles using Unity Catalog and develops ingestion and integration patterns based on Medallion Architecture.

The architect develops PySpark notebooks and procedures for advanced data transformation, applies data engineering best practices, and tunes performance in Databricks and Azure SQL databases. The role architects solutions supporting yield improvement, process control, predictive maintenance, and operational efficiency while ensuring governance, security, scalability, and compliance.

The position requires extensive hands-on experience with Azure Databricks and enterprise data platforms, including Azure Data Lake Storage, Azure Data Factory, Azure Synapse Analytics, and Azure SQL. Databricks work includes Unity Catalog design and governance, Delta Live Tables pipelines, CDC, data quality, transformations, scaling, and performance tuning. The role also works with ingestion and ELT tools such as Fivetran, HVR or equivalent replication tools, and dbt, as well as batch, streaming, and sensor/OT/ET data flows.

The architect contributes to governance, security, and operational excellence, including access controls, encryption, audit logs, privacy requirements, monitoring, observability, reliability, metrics, backup, failover, and disaster recovery. The posting also calls for strong communication, analytical, stakeholder-management, and architecture-documentation skills. Exavalu describes a flexible, diverse, and inclusive culture, including flexibility in work arrangements and a welcome-back program for people returning after an extended absence for health or family reasons.

Requirements

• 10+ years of overall experience and 5+ years of architecture experience in data architecture or data engineering roles.
• Strong hands-on experience with major enterprise data platforms.
• Extensive experience with Azure-native services, including Azure Data Lake Storage (ADLS), Azure Data Factory (ADF), Azure Synapse Analytics, and Azure SQL.
• Strong hands-on experience with Azure Databricks and its modern capabilities and best practices.
• Experience designing Unity Catalog catalogs and schemas, permissions and security, managed and external tables, lineage, and governance.
• Experience developing Delta Live Tables pipelines, including CDC implementation, transformations, data quality controls, scaling, and performance tuning.
• Experience with data ingestion tools such as Fivetran and HVR or equivalent CDC/replication tools, and ELT tools such as dbt.
• Familiarity with real-time/streaming data, batch processing, and sensor/OT/ET data flows.
• Strong expertise in data ingestion, transformation, and orchestration frameworks.
• Hands-on experience building insurance-specific data models.
• Proficiency in Python/PySpark and Spark SQL.
• Strong understanding of streaming versus batch processing and experience tuning Spark job performance.
• Ability to optimize data storage formats, partitioning strategies, and schema design.
• Experience working with object stores, data lakes, cloud compute scaling, and Databricks cluster management.
• Deep understanding of data governance, security and compliance, RBAC and attribute-based access control, encryption and audit logs, and data privacy requirements.
• Experience with monitoring and observability, reliability and metrics, and backup, failover, and disaster recovery strategies.

Nice To Haves

• Prior experience working on Insurance P&C cloud-based data and analytics projects.
• Strong analytical and problem-solving skills.
• Solid understanding of distributed data processing concepts.
• Strong communication and stakeholder-management skills.
• Ability to document architecture decisions, trade-offs, and technical recommendations.

Responsibilities

• Design and implement end-to-end data architecture and ecosystems using Azure-native services and Databricks.
• Build scalable, high-performance data platforms and robust data frameworks.
• Partner with stakeholders to translate business requirements into reliable data solutions.
• Implement data governance principles using Unity Catalog.
• Create and implement data ingestion and integration patterns using Medallion Architecture.
• Apply data engineering best practices and develop PySpark notebooks.
• Create procedures for advanced data transformation.
• Tune performance in Databricks and Azure SQL databases.
• Architect data solutions supporting yield improvement, process control, predictive maintenance, and operational efficiency.
• Ensure governance, security, scalability, and compliance across the platform.
• Develop reliable Delta Live Tables pipelines, including CDC, transformations, data quality controls, scaling, and performance tuning.
• Apply operational excellence practices, including monitoring, observability, reliability, metrics, backup, failover, and disaster recovery.
• Define and enforce best practices for Databricks workloads and Spark-based pipelines.
• Document architecture decisions, trade-offs, and technical recommendations.

Benefits

• Flexible work arrangements, which may include part-time work, working outside normal 9-to-5 business hours, or working remotely.
• Welcome-back program to help individuals return to the mainstream after a prolonged absence due to health or family reasons.
• Commitment to a diverse and inclusive workforce and culture.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed