Senior Data Engineer

BDIPlus • New York, NY

1d

About The Position

BDIPlus is seeking a Senior Data Engineer to support a Fortune 100 financial services client’s real-time intelligence initiatives. In this role, you will work in a virtualization-first architecture using Denodo, building both virtual and physical data products that are governed, high-quality, and AI-ready. You will play a critical role in establishing trusted, reusable data assets—each backed by formal data contracts, quality controls, and full lineage.

Requirements

• 5+ years of data engineering experience in enterprise-scale environments
• Strong expertise with Denodo (VQL development, query optimization, caching, governance)
• Experience with Apache Iceberg on AWS (S3, Glue, Athena), including schema evolution and partitioning
• Hands-on experience with Informatica
• Experience implementing data quality frameworks
• Strong SQL and programming skills (Python, Spark; dbt preferred)
• Experience with CDC technologies (e.g., Debezium, Informatica CDC, or equivalent)
• Familiarity with API development (FastAPI, Node.js) and OpenAPI standards
• Experience working with AI-assisted development tools (Claude Code preferred; training available)

Nice To Haves

• Experience with mainframe data environments (COBOL copybooks, VSAM, AIX/AS400 extraction)
• Financial services or insurance domain experience
• Familiarity with multi-tier data certification frameworks (e.g., Bronze/Silver/Gold or Foundation/Production/Enterprise)
• Experience with API gateways and AI integration patterns (e.g., Kong Gateway, APIGEE, MCP)
• Experience designing and enforcing formal data contracts and dependency management
• Consulting or client-facing experience with structured knowledge transfer
• AWS certifications (Solutions Architect, Data Analytics) are a plus

Responsibilities

• Build governed virtual data models in Denodo, including cross-system joins, canonical schemas, ABAC policies, and column-level masking
• Configure and optimize semantic layer connectivity (JDBC/ODBC, connection pooling, failover)
• Implement data quality rules (completeness, validity, uniqueness, anomaly detection, scoring)
• Configure data catalog, lineage, and marketplace publishing
• Build physical data products: Source extraction (mainframe, CDC, COBOL copybooks), ETL/ELT pipelines using Informatica IDMC or dbt, Data Lakehouse storage using Apache Iceberg on AWS S3
• Define and implement machine-readable data contracts (YAML/JSON), including schema guarantees, SLAs, and dependency tracking
• Develop APIs, SLA monitoring dashboards, reconciliation processes, and production-grade delivery patterns
• Partner with client teams through demos, rotations, and structured knowledge transfer
• Leverage AI-assisted development (Claude Code) to accelerate delivery and improve productivity

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed
• Number of Employees: 11-50 employees