Data Engineer (remote US)

HarperCollins Publishers • ,

1d • $120,000 - $130,000 • Remote

About The Position

HarperCollins is seeking a Data Engineer to join its Data Management & Engineering team. The role develops, maintains, and supports enterprise data integration processes and pipelines across on-premises SQL Server and Microsoft Azure environments. It supports modernization of legacy data processing solutions into cloud-based architecture using Microsoft Fabric, Azure Data Lake, and enterprise Data Products.

The Data Engineer works closely with Data Governance, business stakeholders, application teams, and other engineering resources to ensure data is processed accurately, efficiently, and consistently across enterprise platforms. The role translates business requirements into reliable, scalable, and well-documented technical solutions.

Requirements

• Bachelor's degree in Computer Science, Software Engineering, Information Technology, or a related technical discipline.
• 3–5+ years of hands-on experience in data engineering, database development, or enterprise data integration, including building pipelines, migrating data, and modernizing environments.
• Strong proficiency in SQL Server, T-SQL, stored procedures, query optimization, and ETL/ELT development.
• Hands-on experience with Azure Data Factory, Azure Data Lake Storage, and Microsoft Fabric, including Data Pipelines, Lakehouse, and Notebooks.
• Proficiency in Python and/or PySpark.
• Understanding of data warehousing, relational data modeling, and Medallion Architecture.
• Experience with file-based integrations, batch processing, queue-driven workflows, data validation, monitoring, troubleshooting, and production support.
• Strong communication and collaboration skills, including the ability to work with business stakeholders, Data Governance, and technical teams to translate requirements into effective solutions.
• Strong analytical and problem-solving abilities, with attention to data accuracy, quality, and operational reliability.

Nice To Haves

• Familiarity with SSIS or legacy ETL tools.
• Experience modernizing legacy data platforms.
• Relevant Microsoft certifications.

Responsibilities

• Design, develop, and maintain data pipelines using Microsoft Fabric, Azure Data Factory, and related Azure technologies.
• Develop Python, PySpark, and SQL notebooks for data ingestion, transformation, and processing across Bronze, Silver, and Gold layers.
• Integrate data from files, databases, APIs, and enterprise applications.
• Implement data transformations, enrichment, aggregations, and business rules based on approved specifications.
• Optimize pipelines and SQL queries for performance, reliability, and scalability.
• Develop and maintain enterprise Data Products in collaboration with Data Governance.
• Maintain on-premises SQL Server databases, stored procedures, and ETL processes while supporting their migration to Azure and Microsoft Fabric.
• Develop and maintain automated, rule-based processing workflows, including queue-driven processing and batch orchestration.
• Manage file-based ingestion, feed tracking, processing status, and downstream data distribution.
• Implement data validation, reconciliation, logging, monitoring, and exception handling.
• Monitor production pipelines, troubleshoot failures and data discrepancies, and implement corrective actions.
• Ensure continuity, reliability, and timely delivery of business-critical data processes.
• Collaborate with business stakeholders and Data Governance to understand requirements, data definitions, business rules, and approved Data Product specifications.
• Translate business requirements into technical designs, identifying source data gaps, dependencies, and process improvements.
• Maintain technical documentation, including data flows, source-to-target mappings, transformation logic, and operational procedures.
• Perform unit and integration testing, support QA and business sign-off, and adhere to established development, deployment, and Data Governance standards.

Benefits

• Comprehensive and highly competitive benefits package covering physical health, retirement and savings, caregiving, emotional wellbeing, transportation, and other benefits.
• Elective benefits employees may select to fit their needs and personal situations.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level