Data Engineer (AWS, Spark)

OpenDataJobs • Washington, DC

4d • Hybrid

About The Position

Peregrine Advisors is seeking a motivated Data Engineer to join their team. This role involves building data pipelines on Amazon Web Services (AWS) for a federal agency, focusing on ingesting, processing, storing, and ensuring the quality of data at scale. The position is an initial assignment with opportunities to grow within the firm, contributing to new capabilities and lines of business. Peregrine Advisors is a data and technology innovation hub and a Benefit Corporation committed to delivering value to clients and the public.

Requirements

• Sole United States citizenship and the ability to obtain a Public Trust determination.
• Hybrid role based in the Washington, DC metropolitan area, requiring regular commuting into DC.
• 4+ years of data engineering experience.
• Bachelor's degree.
• Spark ETL on AWS (Glue, Amazon EMR) in Python and PySpark.
• S3 data-lake design (Parquet, partitioning, lifecycle) feeding Apache Iceberg tables, Amazon Aurora PostgreSQL, and DynamoDB.
• Experience with event orchestration (Lambda, Step Functions, SQS/SNS) with secrets management and monitoring.
• Experience with data quality, validation, and lineage.
• Experience with infrastructure-as-code (CloudFormation or Terraform).
• Basic proficiency in writing, PowerPoint, and Excel.

Nice To Haves

• Master's degree in a relevant field.
• Trino or comparable federated SQL across the lake and relational stores.
• Apache Ranger-governed access.
• Legacy ETL migration (for example DataStage).
• Federal information technology or high-volume data experience.
• Familiarity with AI-assisted developer tooling.

Responsibilities

• Build Spark-based extract, transform, and load (ETL) pipelines on AWS using Glue, Amazon EMR, Lambda, and Step Functions, in Python and PySpark.
• Design and implement a data lake on S3 (Parquet, partitioning, lifecycle) feeding Apache Iceberg tables, PostgreSQL on Amazon Aurora, and DynamoDB.
• Implement federated SQL access using Trino.
• Incorporate event orchestration, secrets management, monitoring, and data quality, validation, and lineage into the data lake.
• Develop and maintain infrastructure-as-code using CloudFormation or Terraform.
• Contribute to new capabilities, tools, and lines of business within the firm.

Benefits

• Medical, dental, and vision with the employee premium fully paid and half of dependent premiums.
• Employer-paid life, accidental death, and short-term and long-term disability insurance.
• A 401(k) matched 100% up to 4% of salary, vesting immediately.
• Unlimited paid time off.
• Sponsored professional certifications and continuing education.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level