Senior Data Engineer – Credit Risk (Hybrid) - New York, NY

NavitasPartners•New York, NY

1d•Hybrid

About The Position

"Navitas Partners, LLC" is seeking a highly experienced Senior Data Engineer to join a Credit Risk technology team supporting enterprise-scale risk data platforms. The ideal candidate will lead architecture discussions, design scalable data pipelines, and ensure reliable, compliant, and high-quality data processing across modern cloud-based lakehouse environments. This role requires deep expertise in financial data engineering, credit risk domains, and large-scale distributed data processing using PySpark and Databricks.

Requirements

12+ years of experience in data engineering or data development, preferably in financial services or banking
Strong domain expertise in Credit Risk and Counterparty Risk
Familiarity with regulatory frameworks such as Basel III/IV, IFRS 9, CECL, FRTB
Expert-level proficiency in Python and PySpark/Apache Spark
Hands-on experience with Azure Databricks , Delta Lake , and Medallion Architecture
Strong SQL skills including joins, window functions, and performance optimization on large datasets
Experience building ingestion pipelines from core banking, trading, and market data systems
Knowledge of workflow orchestration tools such as Airflow or Databricks Workflows
Experience with CI/CD tools including Git, Jenkins, and Azure DevOps in regulated environments
Understanding of cloud platforms (AWS certification or equivalent preferred)
Experience producing architecture diagrams, data flow documentation, and data dictionaries
Agile delivery experience using tools such as JIRA, Confluence, and Zephyr
Strong communication skills with ability to bridge technical and risk/business stakeholders

Nice To Haves

Strong focus on data governance, data quality, and regulatory compliance
Experience working with quant teams and risk modeling systems
Ability to quickly adapt to evolving financial technologies and regulatory requirements
Proactive, collaborative, and detail-oriented mindset in high-stakes environments

Responsibilities

Lead architecture and technical design discussions for Credit Risk data platforms using modern data engineering frameworks and cloud-native technologies
Design and implement scalable batch and streaming data pipelines using PySpark within a Medallion Lakehouse architecture on Databricks
Build and maintain data ingestion pipelines from upstream systems (loan origination, trading systems, market data feeds) into cloud storage (S3/ADLS) using Parquet and Delta Lake formats
Implement partitioning strategies, Z-order optimization, and schema evolution for high-performance data processing
Develop and optimize large-scale PySpark transformations for credit and counterparty risk datasets ensuring accuracy, auditability, and regulatory compliance across Bronze, Silver, and Gold layers
Support modeling and optimization of risk metrics including PD, LGD, EAD, EPE, PFE, CVA for downstream analytics and reporting
Integrate with external risk/XVA engines and manage orchestration of long-running batch computations
Ensure platform reliability, observability, lineage tracking, security, and regulatory compliance (Basel III/IV, FRTB, CECL)
Design and maintain APIs, data contracts, and technical documentation aligned with audit and compliance standards
Collaborate closely with risk, quant, compliance, and engineering teams to deliver scalable data solutions