Senior AWS Data Engineer

Inizio Partners Corp•Newark, NJ

About The Position

Lead the design, development, and optimization of large-scale, reliable, and secure data pipelines and data lake architecture on AWS. Architect and implement end-to-end data solutions, including data ingestion, storage, transformation, and analytics using AWS services (Glue, Redshift, S3, Lambda, EMR, Kinesis, Athena, RDS, etc.). Mentor and guide a team of data engineers, conducting code reviews and fostering best practices in data engineering and cloud architecture. Collaborate with data scientists, analysts, and business stakeholders to translate requirements into scalable and maintainable solutions. Oversee migration of data from legacy systems to AWS-based data lakes and data warehouses. Develop and enforce standards for data quality, security, and governance. Drive the adoption of DevOps, CI/CD, and infrastructure-as-code practices within the data engineering team. Ensure solutions are cost-effective, performant, and aligned with enterprise data strategy. Stay current with advancements in AWS technologies and data engineering trends and evaluate new tools and frameworks for potential adoption. Troubleshoot complex data issues and provide technical leadership in problem resolution.

Requirements

Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
10+ years of experience in data engineering.
At least 3 years in a technical leadership or lead engineer role.
Extensive hands-on experience with AWS data services (Glue, Redshift, S3, Lambda, EMR/Spark, Kinesis, Athena, RDS, API Gateway, etc.).
Proficient in programming languages such as Python and SQL.
Strong experience designing, implementing, and managing data lakes, data warehouses, and data ingestion pipelines on AWS.
Proven experience with ETL/ELT processes, data modeling, and big data frameworks.
Experience with DevOps practices, CI/CD pipelines, and infrastructure-as-code tools (e.g., CloudFormation, Terraform).
Superior analytical and problem-solving skills.
Ability to work on a problem independently and prepare client-ready deliverables with minimal or no supervision.
Good communication skills for client interaction.
Systematic, analytical problem-solving approach with strong ownership over data quality, performance, and delivery.
Ability to quickly evaluate new AWS data and analytics services and determine fit for data pipelines or application architecture.
Hands-on experience developing data workflows and infrastructure using IaC frameworks such as CloudFormation or Terraform (as needed).
Working knowledge of CI/CD pipelines primarily to support data application deployments (Jenkins, CodePipeline, etc.).
Proficient with Git for versioning data processing code, libraries, and application components.
Skilled in writing production-grade code in Python, Bash, PowerShell, or similar languages, focusing on data processing and backend development.
Experience using Docker for packaging applications and data-processing workloads, with exposure to containerized data services (ECS, EKS, etc.).
Comfortable developing and troubleshooting in Linux environments.
Solid understanding of key AWS data services and application primitives such as S3, EC2, Glue, EMR, Lambda, RDS, DynamoDB, CloudWatch, and VPC networking concepts.
Strong knowledge of AWS security and IAM as it relates to data pipelines, encryption (KMS), secure data access (IAM roles/policies), and audit controls.
Hands-on experience with ETL, distributed compute, and big data frameworks such as Spark, Glue, Hadoop/EMR, Impala, or similar tooling.
Deep understanding of relational databases, SQL optimization, and application-to-database interaction patterns.
Familiarity with log analytics and observability platforms (Splunk, ELK, Prometheus, Grafana) as they relate to monitoring data pipelines and applications.
Excellent problem-solving, communication, and organizational skills.

Nice To Haves

Experience with Shell scripting.
Experience with Scala.

Responsibilities

Design, develop, and optimize large-scale, reliable, and secure data pipelines and data lake architecture on AWS.
Architect and implement end-to-end data solutions, including data ingestion, storage, transformation, and analytics using AWS services.
Mentor and guide a team of data engineers, conducting code reviews and fostering best practices in data engineering and cloud architecture.
Collaborate with data scientists, analysts, and business stakeholders to translate requirements into scalable and maintainable solutions.
Oversee migration of data from legacy systems to AWS-based data lakes and data warehouses.
Develop and enforce standards for data quality, security, and governance.
Drive the adoption of DevOps, CI/CD, and infrastructure-as-code practices within the data engineering team.
Ensure solutions are cost-effective, performant, and aligned with enterprise data strategy.
Stay current with advancements in AWS technologies and data engineering trends and evaluate new tools and frameworks for potential adoption.
Troubleshoot complex data issues and provide technical leadership in problem resolution.
Debug complex data workflows, optimize application and ETL code, and automate data transformation processes.
Quickly evaluate new AWS data and analytics services and determine fit for data pipelines or application architecture.
Develop data workflows and infrastructure using IaC frameworks such as CloudFormation or Terraform.
Support data application deployments using CI/CD pipelines (Jenkins, CodePipeline, etc.).
Write production-grade code in Python, Bash, PowerShell, or similar languages, focusing on data processing and backend development.
Develop and troubleshoot in Linux environments.
Implement and manage data lakes, data warehouses, and data ingestion pipelines on AWS.
Apply ETL/ELT processes, data modeling, and big data frameworks.
Lead, mentor, and coach engineers in a collaborative team environment.