Team Lead, Software Engineering - Data Transformation

CoStar Group•Richmond, VA

1d•Hybrid

About The Position

CoStar Group is seeking a Team Lead, Data Transformation to lead a new team building next-generation data transformation pipelines, services, and integrations within the finance tech organization. The initial focus will be on building data migration and transformation processes for external systems to be used in the enterprise contracting system. This involves receiving data from multiple sources of varying format, frequency, and type, and processing it in a repeatable, configurable, and recurring state to simplify financial data conversion across systems and acquisitions. This is a hands-on leadership role where you will manage and grow the team, own its deliverables, coordinate with partner teams, and remain directly involved in design and implementation. You will set the standard for engineering practices through your own work, reviews, and mentorship, and will be accountable for the reliability of the team's production systems. The team will also standardize the data environment across billing systems to improve visibility into patterns, errors, and optimizations for business operations, which handle billions of dollars in payments annually.

Requirements

Bachelor's degree required from an accredited, not-for-profit, in-person college/university.
Track record of commitment to prior employers.
6+ years of software or data engineering experience, including 2+ years leading a team as a manager, team lead, or senior/lead individual contributor setting technical direction across engineers.
Demonstrated experience owning delivery for a team, including planning, prioritization, and coordination with product management and business stakeholders, and excellent written and verbal communication with technical and non-technical audiences.
Advanced SQL and strong data modeling skills (dimensional/star schema, normalized, or Data Vault).
Hands-on proficiency in Python, Scala, Java, or C#/.NET.
Production experience with a distributed processing engine, a cloud data warehouse or lakehouse platform (e.g., Databricks/Spark, Snowflake, BigQuery, Redshift, Synapse/Fabric) and orchestration tooling (e.g., Airflow, Dagster, Databricks Workflows, Azure Data Factory), including batch and incremental ingestion from heterogeneous external sources (APIs, flat-file/SFTP feeds, third-party vendors, database replication/CDC).
Experience designing complete applications rather than pipelines alone (back-end services, APIs and integrations, with clear separation of concerns and consistent use of established design patterns), including configurable, reusable frameworks that support multiple integrations or business processes rather than hard-coded, single-purpose implementations.
Track record of accountability for production systems, including on-call ownership, incident response and driving down operational burden, supported by data quality, validation and reconciliation controls for externally sourced data and by version control and CI/CD in a major cloud environment (AWS, Azure, or GCP).
Experience with agentic engineering and AI-assisted development tools (e.g., Claude Code, GitHub Copilot) to accelerate software delivery.

Nice To Haves

Production-scale expertise with Databricks or an equivalent lakehouse platform (Delta Lake, Unity Catalog, declarative pipelines, Databricks SQL, Workflows), open table formats (Delta Lake, Iceberg, Hudi), and a modular transformation framework with tested, version-controlled models (dbt, SQLMesh, or equivalent).
Experience with managed ingestion and replication tooling (Fivetran, Airbyte, Qlik Replicate, AWS DMS, Debezium, Kafka Connect) and streaming or near-real-time ingestion (Kafka, Kinesis, Spark Structured Streaming, Flink).
Data quality and observability frameworks (Great Expectations, Soda, dbt tests, Monte Carlo), cataloging, lineage and governance tooling (Unity Catalog, AWS Glue Data Catalog, Collibra, Alation), and monitoring tools such as Kibana or Datadog.
Familiarity with microservices architecture, containerization (Docker, Kubernetes), and event-driven or serverless architecture (RabbitMQ, SQS, Kafka, Azure Service Bus).
AWS data services (S3, Glue, EMR, Lambda, Step Functions, Redshift), Terraform, and backend data services in C# / .NET.
Payments or financial systems experience a plus.
Demonstrated use of AI-assisted development practices (Claude Code or similar) to accelerate delivery, code generation and review along with root-cause analysis.

Responsibilities

Lead and grow a team of data and software engineers, including hiring, coaching, performance management, and career development.
Own the team's deliverables end-to-end, from scope and technical direction through quality and on-time delivery.
Plan and coordinate with other engineering teams, product management, and business stakeholders, setting realistic commitments and communicating progress, risks, and tradeoffs.
Own the architecture of the team's systems, balancing data platform design (ingestion, transformation, storage) with full application design (services, APIs, integrations, and configurability).
Stay hands-on, contributing directly to the design and implementation of pipelines, services, and APIs, and participating in design and code reviews.
Be accountable for production support of the team's systems, including monitoring, alerting, incident response, and reliability.
Hold a high bar for engineering practices and standards across testing, code quality, CI/CD, documentation, observability, and security.

Benefits

Comprehensive healthcare coverage: Medical / Vision / Dental / Prescription Drug
Life, legal, and supplementary insurance
Virtual and in-person mental health counseling services for individuals and family
Commuter and parking benefits
401(k) retirement plan with matching contributions
Employee stock purchase plan
Paid time off
Tuition reimbursement
On-site fitness center and/or reimbursed fitness center membership costs (location dependent), with yoga studio, Pelotons, personal training, group exercise classes
Access to CoStar Group’s Employee Resource Groups
Complimentary gourmet coffee, tea, hot chocolate, fresh fruit, and other healthy snacks