Sr. Data Engineer

Cargill•Atlanta, GA

About The Position

The Senior Data Engineer will be a core contributor to the architecture, delivery, and operations of Cargill’s internal business intelligence and data platform. The platform delivers automated ETL, ad-hoc and automated analytics, and embedded dashboards to users across the enterprise. This role owns day-to-day data engineering architecture, accelerates feature delivery and performance tuning, and mentors data professionals so the platform can scale sustainably with enterprise demand. The position spans Azure and on-premises data systems, including Azure Data Factory, Azure Blob Storage, Azure SQL Server, and on-premises ETL and data-acquisition components. The role also supports embedded Apache Superset dashboards, platform monitoring, automation and self-healing service scaling, and the integration and evaluation of AI capabilities and technologies.

Requirements

5–6 years of experience designing, building, and supporting enterprise data platforms.
Bachelor’s degree in Computer Science, Data Engineering, Information Technology, or a related field, or equivalent practical experience.
Strong hands-on experience with Azure Data Factory, Azure Blob Storage, and Azure SQL Server.
Advanced proficiency in T-SQL, including query optimization, execution-plan analysis, indexing strategies, stored procedures, tables, and views.
Demonstrated experience building reliable ETL/ELT pipelines integrating on-premises systems with Azure cloud environments.
Experience managing database schema changes, migrations, maintenance, and production performance issues.
Experience implementing CI/CD practices for database and data-pipeline deployments, including version control, peer review, deployment, and rollback processes.
Strong understanding of data modeling, data quality, platform monitoring, access controls, audit logging, and secure deployment practices.
Ability to independently make technical and architectural decisions in a complex enterprise environment.
Strong communication and stakeholder-management skills, including the ability to collaborate across global teams and mentor other data professionals.

Responsibilities

Own and evolve the data platform architecture, maintaining architecture standards, decision records, and consistency of data feeds, models, and pipeline design.
Analyze and optimize query performance through indexing strategies, execution-plan analysis, and tuning slow workloads; lead break-fix efforts for performance degradation.
Design, build, and maintain reliable, idempotent ETL/ELT pipelines that integrate on-premises source systems with Azure.
Manage database lifecycle operations, including schema changes, migrations, and maintenance, with a zero-downtime discipline.
Deliver feature work to clear and prevent delivery backlogs.
Own CI/CD for database and pipeline deployments, including version control, peer review, and audited rollouts and rollbacks.
Monitor platform health, pipeline success rates, and resource and cost consumption, and drive capacity and cost optimization.
Design and build Apache Superset dashboards, balancing dashboard authoring with architectural priorities; support automatic mapping and customer embedding through the platform’s tagging system.
Monitor the platform on behalf of customers, including automated and AI-driven features, to ensure capabilities operate reliably.
Contribute to automation and self-healing service scaling to minimize manual intervention, including quality, governance, and monitoring of AI-generated outputs as generative-AI dashboard creation matures.
Contribute to integrating AI capabilities into the platform, including Model Context Protocol (MCP), Superset MCP, and Azure AI Foundry readiness.
Evaluate emerging data and AI technologies for adoption.
Mentor other data professionals and collaborate across global teams.