Sr Data Engineer (Databricks)

Qode•Washington, PA

1d•$130,000 - $150,000•Hybrid

About The Position

We are seeking a Sr Data Engineer with Databricks expertise to join our Wealth Management client. This role involves working with business and technology partners to analyze data warehouse, analytics, and application needs, ensuring they align with our goals. You will evaluate proposed requests for architectural fit and own the technical platform on Azure, from implementation to ongoing operations. The position requires designing, developing, and coordinating cloud data and application solutions across various teams, providing technical direction, and developing cloud security and access control policies. Responsibilities also include maintenance, troubleshooting, monitoring system logs, capacity planning, cloud cost analysis, and providing after-hours support. You will also develop work plans and prepare status reports for management.

Requirements

Minimum of 8 years of experience developing software and data solution for enterprise environment focusing on data and cloud applications with direct experience with application development, data warehouse and BI design, implementation, and operation
Experience with cloud data warehouse and analytics technologies, specifically Azure Databricks, Azure Synapse, Azure Data Factory, Azure Data Lake Storage, Power BI, and other Azure data solutions
Knowledge or experience with infrastructure (operating system and networking) and cloud administration and automation such as Terraform for Infrastructure as Code.
Knowledge of DevOps in application development such as GitHub Actions, or Powershell
Experience with application development using Microsoft or cloud technologies (such as SQL, Azure, C#, Python)
Experience with Machine Learning, MLOps
Experience with investment management systems and processes
Experience with Agile methodologies in a cloud, application, database, data warehousing and BI space
BS or equivalent in Computer Science or related field or a combination of technical skills, cloud, operating system, networking, security, data application, data modeling and BI background
USC or GC ONLY

Responsibilities

Work with business and technology partners to analyze how the data warehouse, analytics and application will meet our goals
Evaluate all proposed requests to determine fit with data warehouse, analytics and application solution architecture.
Own data warehouse and analytics technical platform on Azure, from implementation to enhancements and ongoing operations
Design, develop and coordinate cloud data and application solutions/projects across diverse groups and areas including enterprise architecture, application development, identity and access management, network and data management
Provide technical direction and guidance to database and application developers
Develop cloud security and access control solution/policy, data and BI standards, guidelines and best practice for business groups and technical teams
Perform maintenance and troubleshooting activities for data warehouse, BI platform and other data projects and resolve issues
Monitor system application logs and identify potential issues and improvements to ensure smooth operations
Perform capacity planning, cloud cost analysis and optimization and provide recommendations to management
Provide after-hour system and application support
Develop multiple work plans for projects and prepare appropriate status reports and submit them to management