Staff Data Engineer, Core Migrations

Machinify

2d•$230,000 - $260,000•Remote

About The Position

As a Staff Data Engineer on the Migrations team, you will be the end-to-end technical and operational owner of client migrations. You will lead the full lifecycle, from analyzing legacy systems through production deployment, coordinating across Operations, Client Success, Product, Data Science, Platform Engineering, subject-matter experts, and clients. The role combines architectural decision-making, hands-on engineering, and program coordination, with ownership from discovery through go-live and hypercare. You will reconstruct business logic embedded in poorly documented legacy systems and translate it into stable, production-grade migration pipelines on the Machinify platform. You will own decisions about pipeline design, partitioning, and validation; document the reasoning behind them; and build automated reconciliation frameworks to verify migrated data against source systems. You will also establish reusable migration practices, mentor engineers, and improve tooling for future migrations.

Requirements

At least 8 years of hands-on Data Engineer or Software Engineer experience.
Demonstrated ability to independently own complex, multi-stakeholder technical projects from start to finish.
Strong Python and SQL skills, including fluency with complex and unfamiliar legacy code.
Deep Apache Spark expertise, including distributed processing, performance tuning, partitioning, and debugging at scale.
Advanced Apache Airflow skills, including designing and authoring production DAG architectures from scratch.
Experience actively using AI coding agents such as Claude, Copilot, Cursor, or equivalent for code generation, debugging, and documentation, and critically evaluating their output.
Ability to reconstruct intent from legacy ETL code and translate it to a modern stack, such as SSIS, SQL Server stored procedures, or T-SQL.
Experience designing automated data validation and reconciliation frameworks to verify migrated data against source systems.
Proficiency with AWS S3 and understanding of cloud-native data patterns, including partitioned object storage, secure cross-account access, and cost/performance trade-offs.
Ability to coordinate across engineering, operations, and client-facing teams while maintaining progress.
Ability to write accurate, concise, actionable discovery specifications, architecture documents, and stakeholder updates for technical and non-technical audiences.
Rigorous data-first mindset and experience building reconciliation frameworks to ensure data fidelity between legacy and modern systems.

Responsibilities

Lead technical discovery and due diligence for legacy systems, reconstruct business logic, and create lineage maps, mapping specifications, and risk analyses.
Reverse-engineer complex, undocumented legacy systems, including SSIS packages and stored procedures, to identify their intent and business logic.
Identify and resolve unknowns by coordinating with clients, subject-matter experts, and Operations.
Architect and build production migration pipelines using Airflow DAGs and Spark jobs, handling ingestion, edge cases, payer-specific carve-outs, business-rule exceptions, reconciliation, and steady-state handoff.
Make and document architectural decisions, including pipeline design, partitioning strategy, and validation approach.
Build automated reconciliation frameworks to verify migrated output against source data at row-level scale.
Own migration programs from kickoff through go-live and hypercare, including scope, sequencing, progress tracking, risk management, stakeholder alignment, and UAT through sign-off.
Codify migration knowledge through runbooks and retrospectives, mentor L3/L4 engineers, and develop reusable tooling for recurring migration challenges.