Lead Data Engineer
About The Position
Design, develop, and implement machine learning pipelines (batch and real-time) to support forecasting, marketing, revenue management, and customer behavior analytics. Develop innovative data solutions that enable revenue optimization, personalization, and data-driven decision-making across business functions with minimal oversight. Apply industry-standard software engineering principles throughout the lifecycle, including requirements gathering, design, development, testing, deployment, and monitoring. Build and optimize scalable data ingestion, transformation, and processing workflows using distributed computing frameworks. Leverage cloud-based services, including AWS Data Pipeline, AWS Glue, AWS EMR to design reliable, high-performance data infrastructure. Implement robust data modeling, feature engineering, and data quality validation to support production-grade machine learning and AI solutions. Collaborate with data scientists, analysts, and business stakeholders to translate analytical needs into deployable systems. Troubleshoot and resolve production defects while implementing long-term fixes. Lead and mentor junior engineers in best practices for big data engineering, AI-driven applications, version control, CI/CD, and DevOps automation. Ensure compliance with enterprise data governance, security, and privacy standards, while conducting performance tuning of large-scale pipelines. Research and recommend emerging technologies in AI, cloud, and data engineering to enhance enterprise capabilities.
Requirements
- Master's degree or foreign equivalent in Business Analytics, Computer Science and Engineering, or related field and three years of experience in Data Engineering, Analytics, or related occupation OR Bachelor's degree or foreign equivalent in Business Analytics, Computer Science and Engineering, or related field and five years of experience in Data Engineering, Analytics, or related occupation
- Progressive experience in application development and object-oriented programming using Scala and Python
- Working with large data sets, data lakes, and cloud technologies
- Utilizing SQL and NoSQL databases such as Hive, AWS Redshift, Microsoft SQL Server, MySQL, and REST API integration
- Leveraging cloud-based services, such as AWS Glue, EMR, or Athena for scalable data engineering
- Implementing distributed computing frameworks, such as Apache Spark, Hadoop MapReduce, and Apache
- Applying DevOps tools and CI/CD practices
Responsibilities
- Design, develop, and implement machine learning pipelines (batch and real-time)
- Develop innovative data solutions that enable revenue optimization, personalization, and data-driven decision-making
- Apply industry-standard software engineering principles throughout the lifecycle
- Build and optimize scalable data ingestion, transformation, and processing workflows using distributed computing frameworks
- Leverage cloud-based services, including AWS Data Pipeline, AWS Glue, AWS EMR to design reliable, high-performance data infrastructure
- Implement robust data modeling, feature engineering, and data quality validation
- Collaborate with data scientists, analysts, and business stakeholders
- Troubleshoot and resolve production defects while implementing long-term fixes
- Lead and mentor junior engineers in best practices for big data engineering, AI-driven applications, version control, CI/CD, and DevOps automation
- Ensure compliance with enterprise data governance, security, and privacy standards
- Conduct performance tuning of large-scale pipelines
- Research and recommend emerging technologies in AI, cloud, and data engineering
Benefits
- Health/Dental/Vistion ins.
- 401(K)
- PTO
- Life/Disability Ins. & more
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
