Sr. Data Engineer (Hybrid)

American Medical Association•Chicago, IL

2d•$115,523 - $150,972•Hybrid

About The Position

The American Medical Association (AMA) is seeking a Sr. Data Engineer for their Information Technology team. This is a hybrid position requiring 3 days a week in the Chicago office. The role is crucial in implementing and maintaining the AMA’s enterprise data platform to support analytics, interoperability, and responsible AI adoption. The Sr. Data Engineer will partner with various teams including platform engineering, data governance, data science, IT security, and business stakeholders to deliver high-quality, reliable, and secure data products. This position contributes to the AMA’s modern lakehouse architecture, optimizing data operations, and embedding governance and quality standards into engineering workflows. As a senior technical contributor, this role will mentor junior engineers and implement engineering best practices within the data platform function, aligning with leadership's architectural direction.

Requirements

Bachelor's degree in Computer Science, Engineering, Information Systems, or related field preferred or equivalent work experience and HS diploma/equivalent education required.
5+ years of experience in data engineering within cloud environments
Demonstrated hands-on experience with modern data platforms (Databricks preferred).
Proficiency in Python, SQL, and data transformation frameworks.
Experience designing and operationalizing ETL/ELT pipelines, orchestration workflows (Airflow, Databricks Workflows), and CI/CD processes.
Solid understanding of data modeling, structured/unstructured data patterns, and schema design.
Experience implementing governance and quality controls: metadata, lineage, validation, stewardship workflows.
Working knowledge of cloud architecture, IAM, networking, and security best practices.
Demonstrated ability to collaborate across technical and business teams.
Exposure to AI/ML engineering concepts, feature stores, model monitoring, or MLOps patterns.
Experience with infrastructure-as-code (Terraform, CloudFormation) or DevOps tooling.

Nice To Haves

Experience in people management preferred.

Responsibilities

Build and maintain scalable data pipelines and ETL/ELT workflows supporting analytics, operational reporting, and AI/ML use cases.
Implement best practice patterns for ingestion, transformation, modeling, and orchestration within a modern lakehouse environment (e.g., Databricks, Delta Lake, Azure Data Lake).
Develop high-performance data models and curated datasets with strong attention to quality, usability, and interoperability; create reusable engineering components and automation.
Collaborate with the Architecture Team, the Data Platform Lead, and federated IT teams to optimize storage, compute, and architectural patterns for performance and cost-efficiency.
Build model-ready data sets and feature pipelines to support AI/ ML use cases; serve as a technical coordination point supporting business units’ AI-related infrastructure needs.
Collaborate with data scientists and AI Working Group to operationalize models responsibly and maintain ongoing monitoring signals.
Embed data governance, metadata standards, lineage tracking, and quality controls directly into engineering workflows; ensure technical implementation and alignment within engineering workflows.
Work with the Data Governance Lead and business stakeholders to operationalize stewardship, classification, validation, retention, and access standards.
Implement privacy-by-design and security-by-design principles, ensuring compliance with internal policies and regulatory obligations.
Maintain documentation for pipelines, datasets, and transformations to support transparency and audit requirements.
Monitor and troubleshoot pipeline failures, performance bottlenecks, data anomalies, and platform-level issues.
Implement observability tooling, alerts, logging, and dashboards to ensure end-to-end reliability.
Support cost governance by optimizing compute resources, refining job schedules, and advising on efficient architecture.
Collaborate with the Data Platform Lead on scaling, configuration management, CI/CD pipelines, and environment management.
Collaborate with business units to understand data needs, translate them into engineering requirements, and deliver fit-for-purpose data solutions; share and apply best practices and emerging technologies within assigned initiatives.
Work with IT Security and Legal/ Compliance to ensure platform and datasets meet risk and regulatory standards.
Lead, mentor, and provide management oversight for staff.
Responsible for setting objectives, evaluating employee performance, and fostering a collaborative team environment.
Responsible for developing staff knowledge and skills to support career development.
May include other responsibilities as assigned.