Senior Data Engineer

Cynet Systems•San Jose, CA

About The Position

We are seeking a Senior Data Engineer to lead the end-to-end migration of high-volume, high-complexity data into a new PostgreSQL-based infrastructure. This role involves designing, building, and maintaining robust ETL/ELT pipelines, mapping and reconciling complex legacy data structures, and ensuring data integrity through validation frameworks. The ideal candidate will have deep expertise in Python and SQL databases, with a strong understanding of data modeling, query optimization, and CI/CD practices.

Requirements

5+ years of experience in data engineering, with demonstrated ownership of large-scale data migration projects.
Strong to expert-level proficiency in Python for building and automating data pipelines.
Deep hands-on experience with PostgreSQL/Oracle, including schema design, query optimization, and performance tuning.
Proven experience designing and managing ETL/ELT pipelines at scale (millions of records).
Experience mapping and transforming complex, legacy data structures across disparate systems or business entities.
Strong understanding of data validation, reconciliation, and quality assurance techniques.
Solid grasp of data modeling principles (normalization, indexing, partitioning).
Experience with version control (Git) and CI/CD practices for data pipelines.
Expertise in Python and SQL databases.

Nice To Haves

Experience with orchestration tools (e.g., Airflow, Dagster, Prefect).
Familiarity with cloud data platforms (AWS, GCP, or Azure).
Experience with other relational or NoSQL databases and cross-database migrations.
Background working in regulated or high-stakes data environments.
Experience with containerization (Docker) and infrastructure-as-code.
Exposure to data quality/testing frameworks (e.g., Great Expectations, dbt tests).

Responsibilities

Lead the end-to-end migration (transformation and load) of high-volume, high-complexity data into a new PostgreSQL-based infrastructure.
Design, build, and maintain robust ETL/ELT pipelines capable of processing millions of records reliably and efficiently.
Map and reconcile complex legacy data structures across multiple business entities into a unified target schema.
Define and implement data validation frameworks to ensure integrity, completeness, and accuracy throughout the migration.
Optimize pipeline performance, including query tuning, indexing strategy, and batch/incremental load design in PostgreSQL.
Identify, troubleshoot, and resolve data quality issues, schema mismatches, and pipeline failures.
Document data mappings, transformation logic, and migration runbooks for engineering and business stakeholders.
Partner with business and technical stakeholders to align migration scope, timelines, and data requirements.
Establish monitoring, logging, and alerting to track pipeline health and data quality post-migration.
Mentor junior data engineers and contribute to engineering best practices and standards.