Senior Data Engineer
About The Position
Energy Management’s Analytics & Data Science team works to accelerate the transition toward a carbon-free future through software and analytics. We are looking for a Senior Data Engineer with strong Microsoft Azure experience to help us accomplish this mission. Working with the Analytics & Data Science team in DNV – Energy Management’s Technology group is more than just a job; it’s an opportunity to be part of a collaborative community where you can learn, grow, and thrive. Join a dynamic and diverse technology team that values innovation, impact, and sustainability. Help us build scalable, cloud-native data solutions that support demand side management, demand flexibility, and transportation electrification programs. This role is based at our DNV office in North Andover, MA, Chalfont, PA, New York, NY, Rochester, NY, Troy, NY presenting a dynamic hybrid schedule where employees will typically spend three (3) days per week working from either a DNV office. Further details regarding role-specific requirements will be shared during the interview process. The position may also be considered at other DNV offices within the Eastern Time Zone.
Responsibilities
- Develop & Optimize Data Pipelines: Build ETL/ELT pipelines using Azure Databricks (Spark), PostgreSQL, and Python to support large-scale data processing and analytics.
- Architect & Automate Workflows: Implement orchestration and messaging using Azure Service Bus and Databricks Workflows for automated, scalable operations.
- Enable ML & Advanced Analytics: Partner with ML engineers and software engineers to deliver clean, well-modeled data for feature engineering and operational ML pipelines.
- Ensure Data Quality & Reliability: Apply validation frameworks, observability tools, and monitoring using Azure Monitor and related services.
- Optimize for Performance & Scalability: Tune Spark jobs, queries, and pipelines for distributed computing efficiency.
- Implement DevOps & CI/CD: Build and maintain Azure DevOps pipelines for testing, version control, and automated deployment of data solutions.
- Foster Collaboration & Mentorship: Work with software developers, analytics engineers, and ML engineers to align data efforts with business goals, while mentoring junior engineers.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
Education Level
No Education Listed
Number of Employees
5,001-10,000 employees
