USE Credit Union-posted 3 months ago
$130,000 - $150,000/Yr
Full-time
San Diego, CA
251-500 employees
Credit Intermediation and Related Activities

An Azure Databricks Architect is a technical role focused on designing and implementing scalable data solutions using Azure Databricks, acting as a subject matter expert in the platform to create efficient data pipelines, manage data lakes, and ensure data quality while leveraging the full capabilities of the Azure cloud ecosystem; essentially, they are responsible for architecting robust data solutions on Azure Databricks, considering data ingestion, processing, storage, and analysis needs within the context of an organization's business requirements.

  • Develop and implement data architecture using Azure Databricks, including data ingestion pipelines, data transformation logic, and data warehousing strategies.
  • Build and optimize ETL pipelines, particularly in Databricks using PySpark, Scala, or Spark SQL.
  • Create and manage Databricks notebooks for data exploration, pipeline development, and data visualization.
  • Design and optimize ETL/ELT pipelines using Apache Spark, Python, Scala, or SQL.
  • Proficiently set up, configure, and manage Azure Databricks clusters, optimizing performance and costs.
  • Design and build data integration solutions to connect diverse data sources, including on-premises systems, cloud-based applications, and APIs.
  • Monitor and optimize data processing performance on Databricks clusters to ensure efficient data processing at scale.
  • Implement data security measures and access controls to protect sensitive data within the Azure Databricks environment.
  • Ensure compliance with security and governance best practices, including RBAC, encryption, and data masking.
  • Work closely with data engineers, data scientists, business intelligence analysts, and business stakeholders to understand requirements and translate them into technical solutions.
  • Adhere to company policy and procedure, complete annual compliance training, and comply with business continuity and vendor management policy.
  • Comply with all Anti Money Laundering laws such as the Bank Secrecy Act, OFAC and USA PATRIOT Act.
  • Bachelor's degree and 8+ years related experience required, or equivalent combination of education and experience.
  • 8+ years of experience in big data architecture and cloud solutions, preferably on Azure.
  • Experience using Databricks and expertise in Azure services such as Azure Data Factory, Azure Data Lake, Azure Synapse, Azure SQL, Azure Functions, and Apache Spark.
  • Proficient in Azure Data Factory (ADF), Azure Databricks, Azure Synapse Analytics, Azure SQL Database, Azure Data Lake Storage (ADLS), and Azure Blob Storage.
  • Strong skills in data modeling, including star/snowflake schemas, normalizations, denormalizations, and working with NoSQL and relational databases.
  • Proficiency in Python, SQL, Scala, or other relevant languages for data manipulation and analysis.
  • Credit union or financial institution experience highly preferred.
  • Experience using Correlation Keystone is preferred, but not necessary.
  • Dax Programming for Power BI, CSharp.net preferred.
  • Salary range from $130,000 to $150,000 per year.
  • Equal Opportunity Employer.
  • Reasonable accommodation for applicants with physical, mental disabilities and sincerely held religious beliefs.
© 2024 Teal Labs, Inc
Privacy PolicyTerms of Service