Director of Data Engineering

Insight2Profit•Chicago, IL

About The Position

As the Director of Data Engineering, you will lead execution in a fast-paced client service environment while helping shape the strategic direction of this critical capability area. You will lead a growing team of data professionals responsible for delivering high-quality solutions across the client engagement life cycle. Bringing a strong hands-on technical background, you will define and enforce the engineering standards, processes, and patterns that the team operates by, with particular depth in Databricks/Spark and PostgreSQL, and champion the use of AI and AI-assisted tooling to accelerate data engineering work. You will provide technical leadership and coaching, drive alignment across Delivery teams and clients, and ensure quality, speed, and consistency in execution in a heavily client-focused, process-oriented environment. You will also partner closely with Delivery leadership, Software Development, and Product Management to advance the Data Engineering roadmap, standardize deliverables, and support evolving data and ETL technology solutions.

Requirements

Bachelor’s degree in Computer Science, Data Engineering, Information Systems, or a related field
At least eight (8) years of progressive experience in Data Engineering or related technical disciplines, including hands-on engineering experience and experience leading technical teams, large-scale initiatives, or client-facing delivery functions.
Deep technical knowledge and experience with traditional Data Engineering techniques and technologies, data transformation patterns, and data governance and security concepts.
Strong knowledge and understanding of the modern and emerging data engineering platforms, techniques, and methodologies.
Experience with and understanding of Data Lakehouse Architecture, big data processing, and related technologies.
Deep, hands-on experience with Databricks/Spark, including practical fluency across the platform’s core capabilities for data engineering workloads.
Strong working knowledge of PostgreSQL, including schema design, query optimization, performance tuning, and operational best practices.
Demonstrated, current use of AI and AI-assisted tooling in day-to-day data engineering workflows, including code generation, pipeline scaffolding, SQL authoring, code review, documentation, and troubleshooting.
Strong process orientation with a proven track record of defining, documenting, and enforcing engineering standards, delivery processes, and quality practices across a technical team.
Heavy client-service orientation with demonstrated success operating in a client-facing technical leadership capacity.
Experience with and understanding of modern Business Intelligence / Analytics platforms like Power BI (preferred), Qlik, Tableau, etc.
Strong business acumen and aptitude.
Understanding of standard software development methodologies, tools, and best practices.
Ability to work in a fast-paced environment and effectively deal with competing priorities.
Excellent problem-solving, organizational, analytical, time management, verbal, and written communication skills.
Excellent communication and interpersonal skills, with demonstrated ability to present information in a group or individual setting.

Nice To Haves

Advanced degree and/or relevant certifications in Data Engineering, Cloud Technologies or related disciplines preferred.
Experience in pricing, profitability, or commercial strategy domains.
Experience in professional services or consulting strongly preferred
Understanding, exposure, and experience with agile methodologies like Scrum is a plus but not required.
Experience with near- or off-shore resourcing models is preferred.

Responsibilities

Align and collaborate with Delivery leaders to ensure appropriate priorities and urgency are maintained for all active engagements.
Set appropriate expectations and proactively manage them.
Manage team priorities and alignment with clients and engagements.
Ensure all requests are addressed in a timely manner.
Scale the team as needed.
Responsible for hiring, onboarding, and managing performance of a large and growing team.
Establish, enforce, and maintain Data Engineering standards, best practices, processes, and implementation patterns across our Databricks/Spark and PostgreSQL platforms, leading the team with strong hands-on technical credibility.
Ensure appropriateness and soundness of all solutions.
Drive the adoption of AI and AI-assisted tooling across the data engineering team (including code generation, pipeline development, documentation, and troubleshooting) to materially improve productivity, code quality, and delivery velocity.
Operate with a heavy client focus, maintaining direct visibility into client engagements and ensuring the team consistently delivers against client expectations on quality, timing, and outcomes.
Collaborate with Enterprise, Data, and other architects to extend and expand data platform and its capabilities.
Oversee data governance and quality, ensuring data accuracy, consistency, and compliance with industry standards.
Establish and enforce data security and privacy protocols to safeguard sensitive information.
Stay current with emerging trends and technologies in the data engineering field, incorporating them into the organization's data strategy.
Mentor and guide a team of data engineers, fostering a culture of excellence, continuous learning and professional growth.
All team members are expected to support the organization’s commitment to information security, confidentiality, and system reliability. This includes following established policies and procedures, safeguarding sensitive and client information, reporting potential security concerns, and supporting incident response or business continuity efforts as needed.
Additional responsibilities as needed to support team and business priorities.