Lead Data Engineer

Harbor Capital Advisors, Inc.Chicago, IL

About The Position

We are seeking a Lead Data Engineer to drive the design, architecture, and delivery of scalable data platforms, AI-powered data products, and enterprise-grade data services. This role includes technical leadership, architectural decision-making, and cross-functional influence across the organization. You will lead the development of end-to-end data ecosystems—including data pipelines, storage, APIs, and AI-enabled services—leveraging modern cloud infrastructure and emerging AI/LLM capabilities. You will play a critical role in shaping how data is transformed into actionable insights and production-grade tools that directly support investment decisions and client outcomes. In addition to building, you will mentor engineers, set technical standards, and guide the evolution of our data platform, ensuring scalability, reliability, and alignment with long-term business strategy. You will partner closely with teams across Multi-Asset Solutions, Investment Research, Investment Products, Accounting, Marketing, and Distribution to translate complex business needs into robust, high-impact data solutions. This role offers a high degree of ownership and visibility, operating at the intersection of data engineering, AI innovation, and investment decision-making. It is ideal for engineers who think beyond implementation, design systems, mentor others, balance coding with strategic thinking, navigate ambiguity, and take ownership of systems and outcomes.

Requirements

  • Deep expertise in data engineering fundamentals (Python, SQL) with production experience
  • Proven experience designing scalable, reliable data architectures and pipelines
  • Strong experience with AWS cloud ecosystem (infrastructure, CI/CD, observability)
  • Hands-on experience with AI-enabled applications and data pipelines
  • Strong experience with Snowflake (including Cortex AI) or similar platforms
  • Expertise in DevOps, containerization (Docker), and CI/CD pipelines
  • Strong knowledge of database systems (Postgres, Aurora, or similar)
  • Experience with Infrastructure as Code (CloudFormation, Pulumi; Terraform a plus)
  • Experience integrating and leveraging LLM developer tools (Claude Code, Copilot, etc.)
  • Bachelor’s or Master’s degree in Computer Science, Engineering, Information Systems, or related field (quantitative disciplines also welcome)
  • 10+ years of experience in Data Engineering, Software Engineering, or related fields
  • 2–4+ years of experience leading projects or teams (formal or informal leadership)
  • Experience delivering production-grade data platforms or AI-enabled systems

Nice To Haves

  • Experience leading or mentoring engineering teams
  • Experience in financial services or regulated environments
  • Knowledge of data governance, lineage, and security frameworks
  • Experience designing data products or internal platforms
  • Familiarity with front-end or full-stack development for end-to-end ownership

Responsibilities

  • Lead the architecture, design, and implementation of scalable data platforms and AI-enabled data products
  • Define and enforce data engineering standards, best practices, and design patterns
  • Own the end-to-end lifecycle of data pipelines, data services, and APIs from concept through production
  • Drive adoption of AI/LLM capabilities within data workflows and products
  • Partner with business stakeholders to translate complex requirements into technical solutions
  • Mentor and guide engineers, providing technical leadership and code reviews
  • Improve system reliability, observability, and performance across the data stack
  • Lead technical decision-making on tools, frameworks, and architecture
  • Ensure data governance, security, and compliance standards are embedded into systems
  • Collaborate across teams and contribute across the stack when needed
© 2024 Teal Labs, Inc
Privacy PolicyTerms of Service