Sr. Data Integration Engineer

SPS Health LLC•Milwaukee, WI

1d•$120,000 - $145,000•Hybrid

About The Position

The Senior Data Integration Engineer is responsible for the design, development, and operation of the data integration and ingestion processes that deliver partner and internal data into our analytics environment. The role owns the flow of data from source acquisition through the curated data warehouse tables consumed by reporting platforms, operational systems, and business analysts. This is a senior, hands-on engineering position spanning the full integration lifecycle: acquiring data from a wide range of external partner and internal systems, validating and conditioning that data on arrival, transforming it into the structures that support analysis and operations, and operating those processes reliably in production. The role is concerned equally with building new integrations and with the continued performance, accuracy, and timeliness of those already in service. A central objective of the position is to advance reusable, well-instrumented integration patterns that shorten the time required to onboard new data sources and that improve the reliability and transparency of data delivery to the business. The Senior Data Integration Engineer will also contribute substantially to the planned modernization of our data platform, evaluating and recommending tooling, architecture, and migration approach for leadership consideration. The role sets technical direction and development standards for data integration work and collaborates closely with data architects, business analysts, stakeholders across the organization, and the technical contacts of our external data partners.

Requirements

Six (6) or more years of professional data engineering, data operations, data platform operations, or related experience.
Advanced T-SQL development skills, including: Set-based rewriting of row-by-row and cursor-based logic, MERGE, upsert, and slowly changing dimension load patterns, Window functions and complex analytic queries, Execution plan analysis, index strategy, statistics management, and resolution of performance issues such as parameter sniffing, Transaction management and structured error handling within stored procedures, including correct rollback behavior on partial failure.
Demonstrated proficiency in Python for data engineering applications, including API-based data acquisition, file parsing and format handling, data validation, and the development of packaged, scheduled jobs. Familiarity with common data libraries such as requests and pandas is expected.
Demonstrated experience acquiring and integrating data from heterogeneous sources, including delimited, fixed-width, JSON, and XML file formats; REST APIs requiring authentication, pagination, and rate-limit handling; and direct database connectivity.
Proficiency with Git and collaborative development workflows, including branching, pull requests, and code review.
A code-first development approach, with integration logic authored and maintained in T-SQL and Python under source control.
Ability to analyze pipeline and query performance and to improve the reliability, scalability, and cost efficiency of data workloads.
Strong written communication skills, with the ability to produce runbooks, technical documentation, and incident reports, and to convey the business impact of technical issues to non-technical stakeholders.
Strong problem-solving skills, attention to detail, and demonstrated ownership of production systems.
Experience with Microsoft Azure data services such as Azure Data Factory or Microsoft Fabric, or comparable cloud orchestration platforms.
Experience migrating on-premises SQL Server integration workloads to a cloud platform.
Experience with dimensional modeling and data warehouse design.
Experience designing and rationalizing SQL Server Agent job dependencies and scheduling.
Experience establishing version control, code review, and repeatable deployment practices for database code.
Experience implementing CI/CD pipelines for database projects.
Experience handling protected health information under HIPAA, or comparably regulated data under an equivalent framework preferred.
Experience with healthcare or pharmacy data, including claims, eligibility, prescription, or delivery data preferred.
Familiarity with data governance, metadata management, and data lineage practices.

Nice To Haves

Bachelor’s degree in Computer Science, Engineering, Information Systems, or related field, or equivalent professional experience preferred.
Experience leading teams and developing supervisory staff preferred.

Responsibilities

Design, develop, and maintain data integration processes that acquire data from partner and internal sources, including flat file transfers over SFTP, REST API endpoints, and direct database connections.
Develop reusable, configuration-driven ingestion patterns that reduce the effort and elapsed time required to onboard new partner data feeds.
Develop and maintain the T-SQL transformation logic that carries data from landing and staging layers through to the curated warehouse tables supporting reporting, operational systems, and analyst queries.
Design ingestion processes to be idempotent and safely re-runnable, incorporating automated retry and restart behavior for failed executions.
Implement automated validation and quarantine processes so that records failing business-defined quality rules are isolated, reported, and prevented from reaching downstream consumers.
Implement data quality rules defined by the business, including schema validation, reconciliation, row count and threshold checks, and anomaly detection.
Establish monitoring, logging, and alerting for pipeline execution state, data freshness, and load completion, and automate the communication of ingestion status to stakeholders.
Diagnose and resolve production data incidents, determine root cause, coordinate remediation, and document preventive measures through runbooks and post-incident review.
Contribute to dimensional data model design in collaboration with the data architect and senior team members.
Establish and maintain version control, code review, and repeatable deployment practices for database and pipeline code.
Define and uphold development standards for data integration work through code review and technical guidance.
Evaluate and recommend tooling, architecture, and sequencing for the platform modernization effort for leadership consideration.
Migrate established integration workflows to modernized patterns incrementally and without disruption to production operations.
Maintain documentation of data feeds, dependencies, lineage, ownership, and escalation paths.
Perform all work involving protected health information in accordance with HIPAA requirements, including least-privilege access, secure transmission and storage of partner data, and the exclusion of PHI from logs and non-production environments.
Coordinate with partner technical contacts, as needed, to resolve file format, schema, and connectivity questions.
Provide occasional off-hours support for critical data load failures or production support rotations as needed to support timely response to critical data issues.
Support AI and machine learning initiatives by maintaining reliable, secure, and well-governed data pipelines and datasets used for model development, testing, deployment, monitoring, and ongoing performance evaluation.
Maintain confidentiality of information processed & follow company policies and procedures.