Manager, Data Engineering

7-Eleven•Irving, TX

1d•Hybrid

About The Position

Confer with business analysts and architects to gather application requirements. Meet regularly with business users, analysts, and architects to understand functional needs, user experience expectations, and system specifications. Use tools like MS Office, Confluence, Lucidchart, and Jira to document and clarify requirements ensuring the development team has clear guidance. Consult with architects and engineers on software system design and maintenance. Collaborate with technical architects and engineers to discuss system design, data flow, user interfaces, backend architecture, and long-term maintenance strategies. Evaluate cost constraints and select appropriate tools and frameworks for implementation. Design, develop, and modify software systems. Architect data flows, APIs, and database schemas. Develop software using Python, PySpark, SQL, Shell scripts, and frameworks like MapReduce, NumPy, and Pandas. Perform unit testing, code reviews, and push code to repositories for collaborative development. Utilize cloud platforms such as Microsoft Azure and Databricks for deployment and data management. Develop and direct software system testing and validation. Execute comprehensive unit, integration, and system testing to identify and resolve defects. Write API documentation and prepare test procedures for QA teams to automate testing processes and ensure software quality. Coordinate with DevOps team on continuous integration and deployment. Work closely with DevOps to configure build pipelines, trigger automated builds on code merges, and deploy applications through testing, staging, and production environments. Use tools like Kubernetes, Docker, and shell scripting for smooth CI/CD operations. Modify existing software to correct errors, enhance capabilities, and improve performance. Investigate and prioritize bugs reported by users. Analyze code and logs to reproduce and fix issues. Periodically optimize code for better performance and implement requested feature enhancements to meet evolving business needs. Assess technical and security risks, suggest and implement solutions. Review code to identify security vulnerabilities and technical risks such as single points of failure or capacity issues. Recommend and apply preventive measures to safeguard system integrity and reliability. Participate in Scrum meetings, daily standups, and weekly status meetings. Engage in Agile ceremonies to report progress, discuss challenges, and align with the team. Present project updates and milestones to customers and management to maintain transparency and collaboration. Coordinate with offshore team, conduct knowledge sharing sessions, and assist with application incidents. Lead knowledge transfer sessions to keep offshore teams updated on application changes. Track their deliverables, resolve blockers, and provide hands-on support during production incidents to ensure timely resolution and minimize downtime. Manage and mentor a team of two to five data engineers, providing guidance on technical best practices, career development, and initiative execution. Responsible for participation in hiring and performance reviews.

Requirements

Bachelor’s or foreign equivalent degree in Computer Science, Computer Engineering, Software Engineering, Management Information Systems, Mathematics, Biotechnology, or a related field, and 6 years of progressive, post-baccalaureate experience in the job offered or as a Staff Software Engineer, Staff Data Engineer, Data Engineering Manager, or in a related/similar position.
Experience therein to include 6 years of experience in the following: data engineering, including database architecture and design, Extract, Transform and Load (ETL) processes, data lakes, data warehousing, and batch and streaming technologies including Striim and Confluent Kafka; using Python, SQL, and cloud platforms including AWS and Azure.
Experience therein to include 2 years of experience leading and mentoring a team of engineers.
Experience therein to include 1 year of experience with data orchestration tools, including Airflow, data processing frameworks including Spark and Databricks, and data visualization tools including Tableau and Power BI.
Hybrid role, ability to work from home.

Responsibilities

Confer with business analysts and architects to gather application requirements.
Meet regularly with business users, analysts, and architects to understand functional needs, user experience expectations, and system specifications.
Use tools like MS Office, Confluence, Lucidchart, and Jira to document and clarify requirements ensuring the development team has clear guidance.
Consult with architects and engineers on software system design and maintenance.
Collaborate with technical architects and engineers to discuss system design, data flow, user interfaces, backend architecture, and long-term maintenance strategies.
Evaluate cost constraints and select appropriate tools and frameworks for implementation.
Design, develop, and modify software systems.
Architect data flows, APIs, and database schemas.
Develop software using Python, PySpark, SQL, Shell scripts, and frameworks like MapReduce, NumPy, and Pandas.
Perform unit testing, code reviews, and push code to repositories for collaborative development.
Utilize cloud platforms such as Microsoft Azure and Databricks for deployment and data management.
Develop and direct software system testing and validation.
Execute comprehensive unit, integration, and system testing to identify and resolve defects.
Write API documentation and prepare test procedures for QA teams to automate testing processes and ensure software quality.
Coordinate with DevOps team on continuous integration and deployment.
Work closely with DevOps to configure build pipelines, trigger automated builds on code merges, and deploy applications through testing, staging, and production environments.
Use tools like Kubernetes, Docker, and shell scripting for smooth CI/CD operations.
Modify existing software to correct errors, enhance capabilities, and improve performance.
Investigate and prioritize bugs reported by users.
Analyze code and logs to reproduce and fix issues.
Periodically optimize code for better performance and implement requested feature enhancements to meet evolving business needs.
Assess technical and security risks, suggest and implement solutions.
Review code to identify security vulnerabilities and technical risks such as single points of failure or capacity issues.
Recommend and apply preventive measures to safeguard system integrity and reliability.
Participate in Scrum meetings, daily standups, and weekly status meetings.
Engage in Agile ceremonies to report progress, discuss challenges, and align with the team.
Present project updates and milestones to customers and management to maintain transparency and collaboration.
Coordinate with offshore team, conduct knowledge sharing sessions, and assist with application incidents.
Lead knowledge transfer sessions to keep offshore teams updated on application changes.
Track their deliverables, resolve blockers, and provide hands-on support during production incidents to ensure timely resolution and minimize downtime.
Manage and mentor a team of two to five data engineers, providing guidance on technical best practices, career development, and initiative execution.
Responsible for participation in hiring and performance reviews.