Sr Data Engineer

Omm IT Solutions

32d•Onsite

About The Position

We are seeking a hands-on, delivery-focused Senior Data Engineer to help build and scale our cloud data platform. In this role, you will design and develop modern data pipelines using PySpark, Snowflake, and AWS to optimize cloud data workloads. The ideal candidate combines strong engineering fundamentals with cloud-native data expertise and is capable of translating complex business needs into robust, performant, and well-documented data solutions. Experience within Fannie Mae, Freddie Mac, or equivalent GSE/mortgage enterprise environments is highly valued.

Requirements

6+ years of hands-on data engineering experience in large-scale enterprise environments.
Deep proficiency in building distributed data processing pipelines, handling S3 Parquet/Delta files, and implementing complex transformations and deduplication logic.
Strong hands-on experience with SnowSQL, Snowpipe, Streams, Tasks, and Role-Based Access Control (RBAC).
Proven track record establishing Iceberg tables, Dynamic Tables, and Materialized Views.
Robust working knowledge of AWS services, including S3, Glue, EMR, Lambda, IAM, Step Functions, CloudWatch, and Redshift.
Mastery of advanced SQL techniques (window functions, CTEs, complex joins) alongside strong Python programming skills for automation, scripting, and orchestration utilities.
Solid understanding of data warehousing, ELT/ETL patterns, data lakes, and lakehouse architectures using tools like Airflow or AWS Step Functions.
Strong verbal and written communication skills with the ability to articulate technical decisions clearly to both technical peers and business leaders.

Nice To Haves

Prior experience working within heavily regulated environments such as financial services, mortgage banking, or GSE programs (Fannie Mae / Freddie Mac).
Hands-on experience with legacy ETL frameworks (e.g., IBM DataStage) to support modernization initiatives.
Familiarity with continuous integration and continuous deployment pipelines for data infrastructure (Git, Jenkins, GitHub Actions, Terraform).
Exposure to automated data quality and validation frameworks (e.g., Great Expectations, dbt testing suites).
Knowledge of real-time streaming platforms like Apache Kafka or AWS Kinesis.
AWS Certified Data Analytics, AWS Certified Solutions Architect, or SnowPro Core/Advanced certifications.

Responsibilities

Design and build scalable batch and streaming data pipelines using PySpark for large-scale data processing.
Migrate legacy, on-premises ETL workloads (e.g., IBM DataStage, Informatica) to high-performing PySpark and Snowflake cloud pipelines.
Write production-grade PySpark code to read from Amazon S3 (Parquet/Delta files), execute complex transformations, and process massive datasets efficiently.
Design and implement robust deduplication strategies for high-volume datasets using PySpark.
Build and manage Snowflake warehouses, schemas, and data models optimized for enterprise analytics and business intelligence reporting.
Design and implement Apache Iceberg tables in Snowflake to support open lakehouse architectures and data interoperability.
Build and maintain Snowflake Dynamic Tables and Materialized Views to enable near real-time analytics and query acceleration.
Optimize distributed Spark jobs by leveraging partitioning, caching, broadcast joins, and shuffle optimization.
Tune Snowflake workloads using clustering keys, micro-partition pruning, query profiling, precise warehouse sizing, and strategic result caching.
Continuously monitor and optimize Spark jobs, Snowflake queries, and AWS infrastructure to balance speed and cloud expenditure.
Implement data validation, lineage tracking, and monitoring solutions across all pipeline stages to ensure high data integrity.
Partner closely with data architects, business analysts, Technical Program Managers (TPMs), and corporate stakeholders to deliver dependable data products.
Author comprehensive technical designs, data schemas, and operational runbooks to ensure every pipeline is maintainable and audit-ready.