Data Engineer

Eigen Development • Tampa, FL

21h

About The Position

Shift4 is seeking a Data Engineer to build and maintain SQL- and Python-based jobs that turn raw payments data in Snowflake into reconciled, reliable datasets and reports. The role extends and improves a library of containerized Python scripts that run SQL against Snowflake on a schedule, with a focus on reliability, scalability, and maintainability.

This is a hands-on build role involving writing queries, wrapping them in clean, testable Python, packaging them in containers, and ensuring their outputs are correct. The Data Platform team owns ingestion into Snowflake; this position owns processing and data products after the data has arrived. The engineer works with finance, operations, and product stakeholders and applies appropriate security controls for a payments environment.

Requirements

• 1–3 years of experience in data engineering, analytics engineering, or backend roles building SQL-heavy data workloads in production.
• Strong SQL skills, including window functions, CTEs, aggregation and joins over large tables, query tuning, and reading execution plans.
• Hands-on Snowflake experience, including warehouses and credit/cost awareness, stages and COPY INTO, streams and tasks, time travel, VARIANT/semi-structured data, and Snowpark or the Snowflake Python connector.
• Solid Python skills for data work, including pandas or Polars, parameterized SQL, configuration and secrets management, structured logging, error handling, and unit tests.
• Docker fundamentals, including building images, running scheduled batch jobs in containers, and debugging container failures.
• Git and basic CI/CD habits; comfort with code review and writing testable, rerunnable jobs.
• Attention to numerical correctness and the ability to investigate totals that do not reconcile.

Nice To Haves

• Payments domain knowledge, including authorization, capture, settlement, refunds, chargebacks, interchange and scheme fees, merchant and acquirer data models, and reconciliation.
• Experience with orchestration and transformation tools such as Airflow, Prefect, Dagster, or dbt.
• AWS experience from the developer side, including S3, IAM, Secrets Manager, and ECS/Fargate scheduled tasks.
• Experience with data-quality frameworks such as Great Expectations, dbt tests, or Soda, and pipeline monitoring or alerting.
• Experience in a PCI or otherwise regulated environment.
• Exposure to BI tools that consume outputs, such as Sigma, Tableau, Power BI, or Looker.

Responsibilities

• Design, write, and maintain Python jobs that execute SQL over Snowflake to produce transaction, settlement, fee, and reconciliation datasets for internal and merchant-facing consumers.
• Write performant, readable SQL using window functions, CTEs, incremental logic, and semi-structured JSON/VARIANT handling on large transaction tables.
• Package jobs as Docker containers and run them on the existing scheduler; ensure each job is idempotent, rerunnable, parameterized, and observable through logging and alerts for failures or bad output.
• Implement data-quality checks and reconciliation controls to ensure accuracy and consistent totals across sources and reporting periods.
• Monitor and manage Snowflake costs through warehouse sizing, clustering, query profiling, and avoiding wasteful scans.
• Work with finance, operations, and product stakeholders to translate payments questions into correct, maintainable queries.
• Apply payments-appropriate security controls, including secrets handling, least-privilege access, PII masking, and audit trails.
• Document the job library and contribute to code review, testing, and CI practices.
• Proactively explore and apply AI tools to improve efficiency across query authoring, testing, and documentation.

What This Job Offers

• Job Type: Full-time
• Career Level: Entry Level
• Education Level: No Education Listed