Lead Data Engineer

Inizio Partners Corp • Los Angeles, CA

6d • Onsite

About The Position

We are seeking a Lead Data Engineer with a strong data engineering background, specifically requiring Databricks expertise, along with experience in data modeling, building pipelines, orchestration, and supporting reporting teams. This role involves collaborating with client stakeholders to gather requirements, structure solutions, and ensure high-quality, timely delivery. The position requires designing and optimizing data models, building and monitoring scalable data pipelines on Databricks, and managing end-to-end pipeline operations. The Lead Data Engineer will also lead a project team, provide technical leadership, and prepare project documentation. The expected work split is 50% technical (data modeling, coding, orchestration, monitoring) and 50% management (client collaboration, requirements gathering, solution design, presentations, team management, mentoring).

Requirements

• 8-12 years' experience in data engineering and analytics roles
• Bachelor's or Master's degree in analytics, computer science/engineering, economics, mathematics, or related areas.
• Experience building and maintaining ETL/ELT pipelines
• Solid understanding of data warehousing concepts and dimensional data modeling
• Familiarity with workflow orchestration tools such as Airflow or similar
• Experience working with cloud data platforms or modern data infrastructure
• Entrepreneurial hands-on approach to work.
• Demonstrated leadership ability and willingness to take initiative
• Superior analytical and problem solving skills
• Outstanding written and verbal communication skills
• Effective time management and attention to detail
• Hands on experience in using SQL, Python and Workflow Schedulers (Apache Airflow, Cron)
• Experience in leading team and coordinating with internal / external stakeholders
• Databricks is a must
• Strong proficiency in SQL, Python, and PySpark

Nice To Haves

• Experience in using Cloud Platforms (AWS / GCP / Azure)
• Experience in using Visualization tools (Tableau / Power BI)
• Experience with Big Data Technologies (Hadoop, Hive, Hbase, Pig, Spark, etc.)

Responsibilities

• Collaborate with client stakeholders to gather requirements, structure solutions, and ensure high-quality, timely delivery.
• Design and optimize data models, data marts, and Lakehouse/warehouse layers with strong focus on medallion architecture, query optimization, and performance engineering.
• Build, orchestrate, and monitor scalable data pipelines on Databricks, ensuring reliable ingestion, transformation, CDC handling, and incremental load strategies.
• Manage end-to-end pipeline operations including performance tuning, data quality monitoring, alerting, and issue resolution across production workloads.
• Lead a project team of data engineers supporting multiple workstreams and provide technical leadership through code reviews, best-practice guidance, reusable pattern creation, and mentorship to engineering team members.
• Prepare and maintain project documentation to support project execution and delivery.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior