Sr. Solution Engineer

Associated Wholesale Grocers•Kansas City, KS

3h•Remote

About The Position

Associated Wholesale Grocers (AWG) is transforming our business intelligence, analytics, and data automation capabilities. The goal is to level up data warehousing and data management capabilities, including curated and cleansed data sets, certified data models and sources. We are adapting to cloud-based data automation for data cleansing, extraction, loading, transformations and visualizations. Another goal is to expand storage and compute capacity for ingestion of structured and unstructured data. In addition, we are hardening security constraints, in the areas of role-based access, anonymized data sets and intrusion prevention constraints. As a Data Engineer on the Data Analytics Team, you will be integral to this journey. Your primary area of responsibility will be building a strong data ecosystem which supports a new company-wide business intelligence and analytics environment. You will assist in the design and development of standardized datasets derived from multiple systems of record, using ETL/ELT and developing scalable solutions. A key area of success for this role is developing or applying existing development and scripting skills in process automation to orchestrate data pipelines from business systems, data lakes and third-party data repositories. You will be creating and loading dimensional models that will be used for analysis and reporting. As you work on projects and enhancements you will assist in refining data governance within business processes and the technical tool chain. A key to success in this role is to quickly learn the meaning of business data and understand the business domain. This role includes hands-on development experience and participation in regular on-call duties. This role works independently to deliver assignments on time and budget. You will receive assignments from team leadership or more senior members of the team, follow defined project approaches, and help ensure delivery. You will participate in research to find solutions and use data to complete metrics and corrective actions.

Requirements

Must be fluent in SQL/SQL packages/Views/Triggers in the following environments (preferably SQL Server, Oracle, DB2). Must have completed at least one full cycle of building a data lake and/or a data warehouse, and orchestration of data movement from source to target (with proper mapping, formatting and transformation) through data pipelines such as Azure Data Factory.
5+ years of experience working with Python, PySpark and Jupyter/Databricks notebooks.
2+ years of experience in cloud technologies - Microsoft Azure would score higher, but AWS and GCP will be considered.
Ability to learn scheduling tools such as Cawa and Cleo Harmony.
1+ years of experience in CI/CD process such as Jenkins, GitHub or Azure DevOps
Experience working with one or more analytical systems (Databricks, Snowflake, Redshift, Big query, Azure Synapse, etc.) and ETL/ELT processes.
Knowledge and skills with Power BI will be important, not critical.
Problem solving skills to address complex data engineering challenges.
Ability to multi-task and change course with priorities.
Strong business communications skills for requirements gathering and development.
Must have a foundational knowledge of data science concepts, including causal inference and predictive modeling.
Should meet at least 75% of the skills listed in the sections of Position Responsibilities listed above.
Worked with or built Data Lakes, Data Warehouse in a hybrid environment (cloud / on premises).
Good understanding of data modelling , mapping and a through data quality /governance practices
5+ years overall data warehouse and/or data lake methodologies, and data solution deliveries.
3+ years of experience as a data engineer and/or software engineer.
3+ years of experience building ETL/ELT processes in Azure.
2+ years of experience working with Azure Data Engineering workflow orchestration tools such as Azure Data Factory, Azure Databricks Pipelines
Bachelor's Degree or equivalent professional experience

Nice To Haves

Experience working with supply chain, warehouse management, and inventory management.
Experience with end-to-end supply chain or merchandising.
Experience working in an IBM DB2, SQL Server or Oracle environment.
Experience working with multiple analytical Tools/Languages.
Basic knowledge and understanding of statistics.
Exposure to data streaming tools and technologies like Kafka, TensorFlow or similar.
Experience with MLOps principles is highly desirable. This includes knowledge of how to productionize and deploy machine learning models, monitor their performance, and manage their lifecycle.
Exposure to Generative AI concepts and applications. Understanding the basics of large language models (LLMs), Agents and Generative AI concepts and their applications/use cases, even at a high level, is a plus

Responsibilities

Troubleshoot and develop solutions to problems and provide permanent fixes (being proactive).
Collaborate with other developers and support engineers to resolve issues.
Participate in an on-call rotation for integration and transformation pipelines, legacy systems, and modern data platforms that may involve occasional off-hours and weekend support.
Take ownership of following on-call procedures and demonstrate ownership.
Maintain documentation for legacy systems, pipelines, and modernization implementation.
Provide training as needed.
Build real-time and batch data integrations from disparate source systems into the data lake/data warehouse/delta lake.
Build and maintain scalable ETL processes using workflow automation tools such as Azure Data Factory, Azure Databricks, Cawa and Cleo Harmony
Provide documentation (including Technical Design Documents) for installations and deliverables.
Design dimensional models and be able to cleanse and transform raw data into structured formats.
Administer and maintain data pipelines used for loading data into the data warehouse or Lakehouse.
Scale and tune data pipelines, data sets, and SQL queries.
Responsible for Unit testing, support of Integration testing, and User Acceptance Testing.
Implement proper security mechanisms for data solutions using Azure Key Vault, Entra IDs etc.
Execute Azure DevOps CI/CD processes, DABs, create RFCs and present in CAB.
Collaborate with business partners to understand their business problems or needs and desired outcomes.
In collaboration with other members of the team and user community, configure cloud-based capabilities that enable the AWG community.
Communicate priorities to the technology organization to drive effective business solutions.
Maintain verbal and written issue identification and status updates to cross-functional teams and leadership.
Responsible for understanding and adoption of IT Architecture practice.
Continuous learning of existing and future data technologies and tools.
Apply standard methodologies to analyze, design, develop, test, and deploy data management solutions.
Participate in agile story authoring, sizing, and demo sessions for developed features.
Participate in code reviews and receive feedback from the team.
Propose process improvements for continuous department maturity.