Senior Data Engineer - Debt Solutions - San Diego, CA

CoStar Group • San Diego, CA

1d • $146,000 - $190,000 • Hybrid

About The Position

CoStar Group is seeking a Senior Data Engineer to join its rapidly growing Debt Solutions Team. This role will be instrumental in building out sitewide tracking architecture and feeding data into KPI dashboards to monitor site performance, product usage, and consumer behavior. The engineer will contribute to the data engineering architecture for Debt Solutions, collaborate across departments for analysis and insights, and be involved in various aspects of data management, including designing and maintaining automated data infrastructure, delivering data engineering and analytics solutions, and facilitating data access and analysis. The San Diego team is a full-stack product development organization utilizing modern tools and technology. This position is located in San Diego, CA, with a hybrid work schedule: Monday to Thursday in the office and work from home on Fridays.

Requirements

• Bachelor's degree required from an accredited, not-for-profit, in-person college/university
• A track record of commitment to prior employers
• 6+ years of hands-on experience in software development, delivering high-quality solutions
• Strength in Data Engineering, with strong proficiency in Python, PySpark, and SQL
• Solid foundation in Object-Oriented Programming principles and best practices
• Demonstrated success in building and launching data-driven products operating at terabyte scale
• Proven record of designing and implementing Enterprise Level secure, and accurate Data platform
• Ability to translate technical requirements into robust architecture, data models, and ETL strategies
• Practical experience with cloud-based databases, including both relational and non-relational systems
• Knowledge of business intelligence software (i.e. Power BI or similar tools)
• Understanding of AI technologies and hands-on experience leveraging AI tooling in development (Claude Code, GitHub Copilot, Cursor, Gemini, Grok)
• Ability to retrieve, synthesize, and present critical data in structures that is immediately useful to answering specific ad-hoc questions
• Hands-on data experience on optimization, data quality checks, data accuracy and query performance improvement

Nice To Haves

• Hands-on experience or familiarity with big data platforms such as Databricks or Snowflake
• Proficiency with cloud platforms, including AWS or Azure
• Experience working with NoSQL databases (e.g., DynamoDB) and object storage systems (e.g., Amazon S3, GCS, etc.)
• Practical knowledge of event streaming technologies, especially Kafka
• Experience leveraging AWS compute and container services for scalable solutions
• Familiarity with large language models (LLMs), prompt engineering, or agent-based architectures
• Great analytical and problem-solving skills, with the ability to tackle complex technical challenges
• Strong Ability to understand technical challenges and complex designs, and communicate into simple business ideas, or solutions with senior management/stakeholders

Responsibilities

• Design and Oversee implementation of dimensional modeling, database design, and cloud data platform structures (e.g., Databricks, Snowflake, BigQuery)
• Develop data platform framework to build secure data warehouses and data lakehouses applying principles of Medallion Architecture
• Design, develop, and maintain scalable data pipelines and ETL processes using Databricks, Snowflake, and other AWS services
• Implement and optimize Spark jobs, data transformations, and data processing workflows in Databricks
• Building and deploying AI/ML models: Integrate Machine Learning into data pipelines, leverage Databricks ML and AWS ML to develop predictive models and drive business insights
• Define and Apply AI-assisted software development practices to improve engineering productivity and code quality
• Understand various data domains and be able to translate them through data transformations to suit various business needs
• Oversee automated data infrastructure: Collaborate with software engineering and product teams to streamline data tracking in our products
• Ability to integrate data from disparate sources via cross domain data stitch into existing data model
• Implement data governance, lineage tracking, testing, and monitoring frameworks to guarantee data integrity
• Hands-on data optimization and query performance
• Liaise with key company teams (DBA, DevOps, SecOps…) for data accessibility and analysis
• Mentor and manage engineers and analysts in the normal execution of their responsibilities.
• Successfully implement software development projects to specification and within scheduled timelines and budget parameters
• Leverage AWS DevOps and CI/CD best practices to automate the deployment, through terraform, and management of data pipelines and infrastructure.

Benefits

• Generous compensation and performance-based incentives
• Internal training
• Tuition reimbursement
• Comprehensive healthcare coverage: Medical / Vision / Dental / Prescription Drug
• Life, legal, and supplementary insurance
• Virtual and in person mental health counseling services for individuals and family
• Commuter and parking benefits
• 401(K) retirement plan with matching contributions
• Employee stock purchase plan
• Paid time off
• On-site fitness center and/or reimbursed fitness center membership costs (location dependent), with yoga studio, Pelotons, personal training, group exercise classes
• Access to CoStar Group’s Diversity, Equity, & Inclusion Employee Resource Groups
• Complimentary gourmet coffee, tea, hot chocolate, fresh fruit, and other healthy snacks

What This Job Offers

• Job Type: Full-time
• Career Level: Senior