Sr. Lead Data Engineer
About The Position
Cox Automotive Corporate Services, LLC is seeking a Sr. Lead Data Engineer in Atlanta, Georgia. This role involves overseeing the design, deployment, and operation of technology architecture, solutions, and software for capturing, managing, storing, and utilizing structured and unstructured data. The engineer will establish structures to channel data from multiple inputs and store it using distributed (cloud) structures, local databases, and other forms. Responsibilities include supervising the development of tools leveraging AI, machine learning, and big data techniques for data cleansing and transformation, leading the creation and governance of design standards, and providing senior guidance to engineering teams on data acquisition, aggregation, and warehousing infrastructure. The role requires collaboration with senior management and Product Management on data product design and development, managing the design of complex automated pipelines for ETL processes, and leading the development of architectural standards, tools, and methodologies for technical consistency and quality. The engineer will ensure the development of reliable batch data pipelines, monitor database and infrastructure metrics for performance issues, provide senior oversight of scalable data schemas and models, and research and recommend database and data pipeline solutions. The position also involves demonstrating ownership of initiatives and driving them to completion, as well as leading and developing team capabilities through coaching and mentorship. The position reports to the Atlanta, GA office, with telecommuting permitted in accordance with company policy, provided the employee lives within commuting distance of the office.
Requirements
- Bachelor's or foreign equivalent degree in Computer Science, Information Technology, Engineering, Management Information Systems or a related field and 8 years of experience designing, developing, or implementing highly scalable, high volume software systems and components.
- 8 years of experience in defining data architecture and infrastructure roadmaps to deliver scalable, secure, and reliable data platforms supporting analytics, AI or Business Intelligence.
- 8 years of experience working with relational and NoSQL databases, including Amazon RDS SQL Server, Azure SQL Instances, or Amazon DynamoDB.
- 8 years of experience using application lifecycle methodologies.
- 8 years of experience working with business intelligence tools and platforms.
- 8 years of experience implementing and maintaining data cataloging platforms including AWS Glue or Collibra to support data discoverability, lineage and compliance initiatives.
- 3 years of experience using database systems, ETL processes, or cloud platforms, including AWS Database Migration Service, Amazon RDS SQL Server, AWS Data Pipeline, or AWS Transfer for SFTP or Simple Storage Service.
- 3 years of experience building and maintaining software running in a CI/CD pipeline, including GitLab CI/CD or AWS CodePipeline.
- Must live within commuting distance of the Atlanta, GA office.
Nice To Haves
- Master's or foreign equivalent degree in Computer Science, Information Technology, Engineering, Management Information Systems or a related field and 6 years of experience.
- Ph.D. or foreign equivalent degree in Computer Science, Information Technology, Engineering, Management Information Systems or a related field and 3 years of experience.
Responsibilities
- Oversee design, deployment, and operation of technology architecture, solutions, and software to capture, manage, store, and utilize structured and unstructured data from internal and external sources.
- Establish structures based on business and technical requirements to channel data from multiple inputs and store using distributed (cloud) structures, local databases, and other storage forms.
- Supervise development of tools and programming leveraging artificial intelligence, machine learning, and big data techniques to cleanse, organize, and transform data.
- Lead creation and governance of design standards and assurance processes for software, systems, and applications.
- Provide senior guidance and leadership to engineering teams through design, development, and execution of data acquisition, aggregation, and warehousing infrastructure.
- Collaborate with senior management, Product Management, and other engineers in design and development of data products.
- Manage design of complex automated pipelines for optimal extraction, transformation, and loading of data from diverse data sources.
- Lead development of architectural standards, tools, and methodologies to ensure technical consistency and quality across the company's data management applications.
- Ensure development of reliable, accurate, and up-to-date batch data pipelines that make data easily consumable by end-users and other systems.
- Monitor database and infrastructure development metrics to identify performance issues and develop resolution strategies.
- Provide senior oversight of design and implementation of scalable data schemas and models.
- Research, analyze, and recommend database and data pipeline solutions after analyzing costs, benefits, and end-use.
- Demonstrate ownership of initiatives and drive them to completion.
- Lead and develop team capability through ongoing coaching and mentorship.
Benefits
- Health care insurance (medical, dental, vision)
- Retirement planning (401(k))
- Paid days off (sick leave, parental leave, flexible vacation/wellness days, and/or PTO)
- Flexible vacation with pay
- Seven paid holidays throughout the calendar year
- Up to 160 hours of paid wellness annually
- Bereavement leave
- Time off to vote
- Jury duty leave
- Volunteer time off
- Military leave
- Parental leave
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
