Senior Data Engineer

Paul Hastings•Boston, MA

44d•$125,800 - $194,750•Hybrid

About The Position

We have an opening for a Senior Data Engineer. The Senior Data Engineer is responsible for designing, building and maintaining modern data pipelines and data integration solutions in the Azure Cloud. This role enables enterprise analytics by delivering clean, timely and well-structured data to platforms such as Power BI. The Senior Data Engineer collaborates closely with architects, analysts and business stakeholders to deliver scalable, secure and efficient data solutions using Azure Data Factory, Azure Data Lake Storage (ADLS), Databricks, and Azure DevOps.

Requirements

Advanced proficiency in Azure Data Factory, Azure Data Lake Storage (Gen2) and Azure Databricks
Experience building and managing CI/CD pipelines in Azure DevOps for data solutions
Strong SQL and ELT experience
Hands-on expertise with PySpark and Databricks notebooks, including Delta Lake workflows
Familiarity with Git, branching strategies and repository management within Azure Repos
Understanding of data security and privacy best practices in Azure (RBAC, managed identities, encryption)
Bachelor’s degree in Computer Science, Engineering or related field (required)
7+ years of data engineering or BI development experience
3+ years building data solutions in Microsoft Azure Cloud
2+ years using Azure DevOps for automated builds, releases and pipeline orchestration
Proven experience working with structured and semi-structured data (Parquet, JSON, Avro)

Nice To Haves

familiarity with schema design (e.g., star, snowflake, denormalized)
Experience with Power BI dataflows, datasets and refresh configuration (preferred)
Familiarity with data cataloging and governance tools (e.g., Microsoft Purview) a plus
Experience with Agile tools such as Azure Boards or JIRA
Experience in professional services, legal, or financial environments preferred

Responsibilities

Design, build and manage scalable data pipelines using Azure Data Factory (ADF), Azure Data Lake Storage (ADLS) and Azure Databricks
Develop ELT frameworks for ingesting and transforming structured and semi-structured data from on-prem and cloud sources
Implement robust DevOps pipelines for code versioning, testing, deployment and monitoring using Azure DevOps
Ensure pipeline performance, reliability and observability using tools like Azure Monitor and Log Analytics
Collaborate with BI teams to support Power BI datasets and ensure optimal data model performance
Automate environment setup, configuration and permissions using infrastructure-as-code where possible
Write modular, testable code in Python or PySpark and manage version control with Git
Support production operations, root cause analysis and performance tuning
Maintain technical documentation for pipelines, data mappings and deployment processes
Work as part of an Agile delivery team, participating in sprints, retrospectives and code review
Mentor junior data engineers and promote best practices in CI/CD, version control, and release management

Benefits

Medical
Dental
Vision
Life/AD&D
Long Term Care
Short- and Long-Term Disability
Flexible Spending Accounts and Health Savings Account
Healthcare Concierge and Advocacy
Lifestyle Spending Account
Voluntary 401(k) Plan and Profit Sharing
10 Paid Holidays per year
generous PTO Program
Paid Parental Leave
Fertility Benefits
Breast Milk Shipping
Back-up Child Care
Elder Care
Tutoring
Wellbeing programs (Employee Assistance Program, Relationship Support, Mental Health, Menopause and Midlife Health and Well-Being Events)
Retirement Plan Consulting
Anniversary Bonus Program
Professional Development Programs
Transportation and Commuter Benefits
International Travel Insurance
Critical Illness, Hospital Indemnity and Accident Insurance
Auto/Home/Pet Insurance
Prepaid Legal Insurance
Employee Discounts