Lead Data Engineer (T2)
About The Position
Creative Artists Agency (CAA) is seeking a Lead Data Engineer to lead data engineering execution in support of complex application development initiatives. This role ensures data integrations, pipelines, and models align with established architectural standards. The successful candidate will collaborate with teammates and other engineers to design, build, and execute data migrations between systems, including data mapping, transformation, validation, and reconciliation to ensure data accuracy and completeness. Utilizing software development skills in an agile environment, the Lead Data Engineer will develop new data pipelines, APIs, and data management tools. Experience with continuous delivery will support the end-to-end lifecycle, including design, deployment, testing, operations, monitoring, and support.
Requirements
- Proficiency in Python with a history of delivering production pipelines.
- Experience building production scripts or services used for data movement and validation.
- Real SQL and NoSQL production experience.
- Expertise with SQL Server and MongoDB.
- Comfortable modeling data, analyzing source and target schemas, writing complex queries, and resolving data quality issues during migration.
- Experience maintaining projects using a distributed version control system such as Git and GitHub.
- Experience in automated pipeline deployment and schemas into production environments.
- Experienced with agile methods and the concepts behind continuous delivery.
- Experience with security, testing, performance, JSON, and REST.
- Strong verbal and written skills.
- Passion for and experience shipping and maintaining production software products and data migrations.
- Attention to detail, problem-solving, simplicity, quality, and moving quickly are highly prized.
Nice To Haves
- Experience with the Azure data stack (Azure Data Factory, Azure Data Lake, Tabular Model / DAX, Microsoft Fabric) is preferred, though strong experience with comparable technologies is also valued.
- Experience in automated pipeline deployment and schemas into production environments would be great.
- Experience and opinions around data quality, test strategy, and test automation.
Responsibilities
- Lead data engineering execution in support of complex application development initiatives.
- Ensure data integrations, pipelines, and models align with established architectural standards.
- Design, build, and execute data migrations between systems, including data mapping, transformation, validation, and reconciliation.
- Develop new data pipelines, APIs, and data management tools.
- Support the end-to-end lifecycle: design, deployment, test, operations, monitoring and support.
Benefits
- Benefits
- Discretionary bonus
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
Education Level
No Education Listed
