Data Engineer (Gen AI)
About The Position
The Enterprise Architecture organization is looking for a Data Engineer to join the team and help build the next generation of data infrastructure and AI-enabled workflows. In this role, you will be responsible for designing, building, and maintaining scalable data pipelines, data lake platforms, and analytics solutions that support enterprise-wide AI initiatives and advanced analytics capabilities. You will partner closely with internal development teams and IT leadership to architect data solutions that meet diverse use-cases across the organization. This role offers the opportunity to work with cutting-edge data technologies, AI knowledge bases, and cloud-native data platforms as we scale our data operations. Your work will focus on delivering robust, well-documented data solutions and establishing best practices for data engineering across the enterprise.
Requirements
- 4+ years of hands-on data engineering experience with modern data stack technologies
- Strong experience with AWS cloud services, particularly: S3, AWS Glue, Athena, EMR, Lambda
- Proficiency in Python for data processing, ETL, and application development
- Experience with PySpark on EMR for large-scale data processing
- Strong SQL skills for data analysis and transformation
- Experience building and maintaining ETL/ELT pipelines at scale
- Experience with Kafka for streaming data pipelines and real-time data processing
- Knowledge of data lake architectures and modern table formats (e.g., Iceberg)
- Experience with CI/CD practices using Git, version control systems, and containerization (Docker)
- Understanding of data modeling, data warehousing concepts, and analytics best practices
- Exceptional problem-solving and analytical skills
- Excellent collaboration and communication (verbal and written) skills
- Self-motivated with ability to work independently and manage multiple priorities
- Willingness and enthusiasm to learn AI/ML technologies, stay current with emerging data engineering trends
Responsibilities
- Design, build, and maintain scalable data crawlers and ETL/ELT pipelines to ingest data from various sources including enterprise collaboration tools, web applications, and internal databases
- Develop and manage data lake platform infrastructure including S3-based storage, Iceberg tables, AWS Glue data catalog, and cloud data warehouse solutions for analytics workloads
- Build and optimize real-time streaming data pipelines using Kafka for event-driven analytics and data processing
- Create and maintain data transformation pipelines to clean, curate, and prepare data for analytics and AI/ML applications, supporting multiple data formats including structured, semi-structured, and unstructured data (text, images, audio, video)
- Develop Python-based applications for data ingestion, processing, and integration supporting Gen AI RAG workflows and knowledge base systems
- Collaborate with internal development teams, data scientists, and stakeholders to understand requirements, architect appropriate data solutions, and create comprehensive technical documentation
- Monitor, troubleshoot, and optimize data pipelines for performance, reliability, and data quality
- Write clean, maintainable, well-tested code following software engineering and data engineering best practices
- Create comprehensive technical documentation for data pipelines, architectures, and platform capabilities
Benefits
- Competitive salary, annual performance-based bonus and stock grant
- Retirement plan 401(k) with competitive company match
- Excellent health and wellness benefits, including medical, dental, and vision benefits. Company paid medical healthcare premium.
- Wellness screenings and assessments, health coaches and counseling services through an Employee Assistance Program (EAP)
- Paid time off and a generous parental leave policy
- Daily company lunch allowance provided and a fully stocked kitchen with healthy options for breakfast and snack
- Corporate events including team outings, dinners, volunteer activities and company sports teams
- Education reimbursement and learning opportunities
- Modern offices with multi-monitor setups
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Mid Level
Education Level
No Education Listed
