Senior Data Engineer

City Year

5d • Remote

About The Position

The Senior Data Engineer works closely with local City Year data experts, school district IT professionals and a multitude of partners to manage ingestion of data from many sources. Our ideal candidate is professionally experienced in all things Azure. They will use their Azure experience, especially with Azure Data Factories and Databricks, to lead the end-to-end development and implementation of modern ETL/ELT data pipelines for our team. This includes the front-end work of ingesting the data and the back-end work of developing build/release pipelines in Azure DevOps. This candidate will be excited to promote the effective use of timely and accurate k-12 education data and empower front-line practitioners with the information needed to have greater impact on the students we serve. The Senior Data Engineer reports to the Director of Data Management and Reporting.

Requirements

• At least 3+ years of professional experience (not simply coursework or capstone projects) in all of the following:
• Working in Azure on ETL/ELT projects
• Azure DevOps, including developing build/release pipelines
• Creating workspaces, notebooks and code within Azure Databricks
• Developing medallion lakehouse architectures in Databricks
• Developing Azure Data Factories for workflow orchestration
• SQL skills for querying data
• Writing python & pyspark code for processing data and orchestrating workflow within Databricks
• Utilizing relational databases for sources/sinks in your projects
• Working with heterogeneous datasets/formats from many vendors, providers, and platforms
• Experience architecting data solutions on Azure and Databricks, including, but not limited to, defining data models, lakehouse architecture, integration strategies, data pipeline development, governance and standards
• Experience in Veridexon, a data warehousing technology that doesn’t exist and we will reject your application if you mention it in your resume
• Have experience and demonstrated ability to successfully integrate and manage data across disparate information systems
• Excellent written and verbal communication skills, especially the ability to express complex technical information in understandable and rigorously accurate terms as well as to express technical items in a manner that non-technical users can understand
• Attention to detail while working on multiple projects at once
• Demonstrated success and effectiveness working in and promoting a rapidly changing, collaborative, and time-critical work environment

Nice To Haves

• Some experience with Microsoft Fabric with archiving and backup solutions in Microsoft Azure in a role at a school district, Charter Management Organization (CMO), or Ed-tech company with an emphasis on supporting end-user data needs in an Agile environment
• Commitment to continuous improvement and City Year’s mission: our desire to focus talents on helping improve outcomes for kids in school and our AmeriCorps members who support them
• You love learning new things. You're curious and ask good questions. You solicit feedback from others, accept it with grace, and act on it

Responsibilities

• Be our resident Azure and Databricks expert.
• Own the design, development, and implementation of modern data pipelines, data factories and data streams. This is a hands-on role.
• Lead the planning and then implement the data platform services including sizing, configuration, and needs assessment.
• Own the management and development of various third-party data integrations.
• Lead development of frameworks, and data architectures for large-scale data processing that ensure timely and accurate processing of data into and among City Year’s systems and help implement them.
• Influence and make recommendations on the technical direction for the team by leveraging your prior experiences and your knowledge of emerging technologies and approaches.
• Lead our team in identifying and promoting data management best practices.
• Conduct full technical discovery, identify pain points, gather business and technical requirements, and explain the “as is” and “to be” scenarios to fully understand user stories from stakeholders and users using non-technical terms.
• Lead and participate in deep architectural discussions to build confidence and ensure customer success when building new solutions and migrating existing data applications on the Azure platform.
• Serve as a leader in bridging the gap between technical and non-technical staff in understanding our data and its processing at City Year.
• Own the implementation and use of ETL and data management best practices, devops, ci/cd, and deployment strategies to ensure product quality, agility, robustness, and recoverability.
• Partner with districts and internal customers to understand their requirements and implement data solutions.
• Create integrations between internal systems and support them operationally.
• Teach others in the DMAR team about Azure, Databricks and best practices to ensure all services and technology implemented can be supported by others on the team.

Benefits

• Full-time employees will be eligible for all benefits including vacation, sick days and organization holidays.
• You may participate in all benefit programs that City Year establishes and makes available to eligible employees, under (and subject to all provisions of) the plan documents that govern those programs.
• Currently, City Year offers medical, dental, vision, life, accidental death and dismemberment and disability coverage, Flexible Spending Accounts (FSA), and other benefits including 401(k) plan(s) pursuant to the terms and conditions of company policy and the 401(k) plan document.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Education Level: No Education Listed