Associate Data Engineer

Candid

23h•Remote

About The Position

Candid is a nonprofit that provides the most comprehensive data and insights about the social sector. We get you the information you need to do good. Candid currently has an opportunity for an Associate Data Engineer. The Associate Data Engineer supports the day-to-day operations of Candid’s cloud data platform. This role is responsible for maintaining ingestion and transformation pipelines built on Apache Iceberg, validating data outputs through schema and structural changes, and assisting with storage management, platform observability, and metadata operations. The Associate Data Engineer develops foundational skills across the modern data lakehouse stack while taking direct ownership of pipeline maintenance, documentation, and validation activities.

Requirements

1– 3 years of experience in data engineering, analytics engineering, or a closely related technical role; internships and relevant academic project work considered.
Solid SQL skills, including writing, reading, and debugging queries against relational or columnar data stores.
Familiarity with cloud data concepts: object storage (Amazon S3), columnar file formats (i.e. Parquet), data-interchange formats (JSON, XML), or open table formats (i.e. Apache Iceberg).
Experience with or exposure to distributed SQL query engines such as Trino or Starburst
Familiarity with AWS services such as S3 and Glue.
Exposure to Apache Airflow, SSIS or another workflow orchestration platform
Experience writing or maintaining data pipelines in Python.
Familiarity with on-prem relational data systems (i.e. Microsoft SQL Server).
Strong attention to detail, especially around data validation and output accuracy.
Strong analytical and problem-solving skills.
Excellent written and verbal communication skills; ability to document findings clearly for both technical and non-technical audiences.
Ability to work independently and collaboratively as part of a distributed team.
Willingness to perform other duties and special projects as needed/requested.
Sensitivity and respect for racial, gender, sexual orientation, and cultural differences.
Champions and represents Candid’s core values: We’re driven, direct, accessible, curious, and inclusive.

Responsibilities

Serve as the primary owner of ingestion pipelines and transformation table adjustments. Ensure continued, reliable data delivery and apply routine changes as business and schema needs evolve. Validate transformation outputs against expected results after schema or structural changes, documenting findings and escalating anomalies to the appropriate teams.
Assist with scheduling compaction and cleanup jobs to maintain Iceberg table health and query performance. Support partition evolution and snapshot retention management to control storage growth.
Assist in implementing and maintaining CloudWatch metrics, alarms, and dashboards to ensure pipeline visibility. Contribute to tracking and reporting on platform performance metrics. Help maintain AWS Glue metadata refresh and statistics jobs that support query planning and optimization within the data platform.
Assist with coordinating schema changes across ingestion and transformation layers to maintain consistency end to end. Collaborate with the Data Operations Engineer to communicate impacts and sequence changes safely.
Support and maintain RBAC and ABAC (least privilege, standardized roles, and consistent tagging). Participate in access reviews and audits, documenting changes and escalating risks as needed.