Data Engineer
About The Position
The Electric Operations Technology Solutions (EOTS) Data Engineer builds, operationalizes, and maintains data pipelines supporting data, analytics, and AI use cases across hybrid and multi-cloud environments. The role translates data into consumable formats, supports integration styles such as virtualization, replication, and streaming, and ensures data reliability and scalability. The Data Engineer leads standardization of data curation, data taxonomy, and solution delivery across the EOTS organization for Databricks and Azure environments. As a member of the Electric Operations Technology Solutions Leadership Team, the position engages in strategic delivery of business solutions supporting multiple business areas within Southern Company. The role is onsite four days per week and remote one day per week, based in Atlanta, Georgia, or Birmingham, Alabama.
Requirements
- Bachelor’s degree in Computer Science, Engineering, Machine Learning, or a related field.
- Nine or more years of experience in data engineering or related roles.
- Experience with batch and real-time data processing frameworks.
- Experience with big data technologies such as Hadoop, HIVE, and Spark.
- Experience with AI/ML solutions in Python.
- Four or more years of experience with on-premises tools such as MSBI, Informatica, Oracle Golden Gate, and SQL Server.
- Two or more years of experience with Microsoft Azure tools such as Data Lake, Data Factory, Databricks, Synapse, and Power BI.
- Experience with containerization technologies such as Docker or OpenShift.
- Proficiency in data modeling, access, storage, and quality tools.
- Familiarity with Agile, DevOps, and CI/CD methodologies.
- Strong communication and collaboration skills.
- Ability to lead and mentor others.
- Familiarity with performance optimization and data governance.
- Hands-on expertise with Databricks, SQL, Python, and Apache Spark.
- Proficiency in data modeling for analytics and machine learning.
Responsibilities
- Design and develop methods to process structured, semi-structured, and unstructured data using batch and real-time techniques.
- Develop release pipelines to automate build, version control, and deployment tasks.
- Build and maintain reusable templates, code libraries, and semantic models.
- Establish gatekeeping and guardrail processes for production data promotion.
- Collaborate with cross-functional data, business, and technical stakeholders.
- Promote data reusability and modular data product delivery.
- Define and enforce data quality and integrity controls.
- Provide technical guidance to less experienced data engineers, business users, and fellow data engineers.
- Optimize performance using parallel processing and minimal data movement.
- Stay current with data technologies and share insights, knowledge, and best practices across the organization.
- Standardize, design, and develop Databricks applications, data integrations, and automated business processes.
- Integrate data from multiple systems, databases, APIs, and business applications.
Benefits
- Competitive base salary.
- Annual incentive awards for eligible employees.
- Health, welfare, and retirement benefits intended to support physical, financial, and emotional/social well-being.
- May be eligible for additional compensation, such as an incentive program.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
