Senior Data Engineer

Trepp•New York, NY

12d•$185,000•Hybrid

About The Position

The Data Platform Team powers all products at Trepp by using distributed data technologies built on our data lake. The Pipeline Team plays a crucial role in designing and maintaining scalable, performant data pipelines that support all product lines at Trepp. As a member of the Data Platform Pipeline Team, you’ll be using open-source and cloud technologies to build, optimize, and maintain robust data pipelines. You’ll collaborate with multiple product teams and engineering teams to solve complex problems in our financial and analytical products for CRE and Banking.

Requirements

Bachelor’s Degree
At least 5 years of experience in application development
At least 3 years of experience in big data technologies
Demonstrated experience in dimensional data modeling and relational schema design for enterprise data warehouses and data lakes
At least 3 years of experience with cloud computing (AWS, Microsoft Azure, Google Cloud)
Experience with Agile and SDLC, Git workflows, and CI/CD
Hands-on experience building and deploying data pipelines on AWS core services (S3, EMR, Glue, Lambda, Step Functions)
Applicants for employment in the US must have work authorization that does not now or in the future require sponsorship of a visa for employment authorization in the United States and with Trepp (i.e., H1-B visa, F-1 visa (OPT), TN visa or any other non-immigrant status).

Nice To Haves

5 + years of experience in application development including Python, SQL, Scala, or Java
5+ years of experience with Distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark)
2+ years of experience working on real-time data and streaming applications
2+ years of experience working on ML ops or related
3+ years of data warehousing experience (Redshift or Snowflake)
2+ years of experience with UNIX/Linux including basic commands and shell scripting
Experience integrating generative AI tooling, LLM APIs (e.g., Amazon Bedrock, SageMaker), or vector databases into production pipelines
Experience with Infrastructure as Code (IaC) tools such as Terraform or AWS CloudFormation/CDK

Responsibilities

Design, build, and maintain scalable batch and near real-time data pipelines that power Trepp’s data-driven products.
Lead complex data engineering projects from planning through delivery, while managing stakeholder expectations and timelines.
Translate high-level business and product requirements into data models, technical designs, and production-ready implementations.
Embed data quality, validation, monitoring, and reliability checks into pipelines to ensure trusted production data.
Develop deep subject matter expertise in internal frameworks and use that expertise to shape roadmaps and prioritize initiatives.
Break down ambiguous ideas into actionable tasks, project plans, and clear execution milestones.
Evaluate technical options through research, clearly articulate tradeoffs, and recommend solutions grounded in business and engineering impact.
Champion data engineering best practices that improve scalability, maintainability, performance, and platform innovation.
Design and integrate AI-driven workflows, including LLM orchestration, embedding generation, and vector search pipelines, into core data processing systems.
Architect and operate reliable data ingestion and transformation pipelines using native AWS cloud infrastructure.

Benefits

Base + target bonus compensation structure
Medical, Dental, Vision insurance
401K (with employer match)
Life insurance, long term disability, short term disability all covered by the company
Flexible paid time off (PTO)
Sixteen (16) weeks paid primary caregiver leave (Biological, adoptive, and foster parents are all eligible)
Four (4) weeks paid parental leave
Wellness subsidy
Pet insurance
Laptop + WFH equipment
Career progression plan
Pre-tax commuter benefit with company subsidy (For NYC-office based employees only)
Fun company events and volunteering opportunities