IT Data Engineer IV

South State Bank•Columbia, SC

3d•$87,282 - $139,425•Hybrid

About The Position

The IT Data Engineer IV serves as a senior technical lead for enterprise data engineering, responsible for designing, building, and improving scalable data platforms, Snowflake data solutions, and dbt-based transformation frameworks that support analytics, reporting, and operational decision-making across South State Bank. This role leads complex data engineering efforts, establishes standards for reliable and well-documented pipelines, and partners closely with business departments, data owners, architects, compliance, and technology teams to translate business needs into durable data solutions. The Data Engineer IV also mentors other engineers, drives continuous improvement in data quality and observability, and ensures data solutions align with enterprise architecture, governance, and regulatory expectations.

Requirements

Bachelor’s degree in Computer Science, Data Engineering, Information Systems, or a related technical field, or equivalent combination of education and relevant professional experience.
8-10+ years of progressive data engineering experience, with at least 3 years in a senior or lead technical capacity.
5+ years of hands-on experience designing, developing, and optimizing solutions in Snowflake, including SQL development, data modeling, performance tuning, security/access patterns, and production support.
5+ years serving in a senior engineer, technical lead, solution lead, or architecture-influencing capacity.
5+ years of hands-on experience with dbt, including model development, testing, documentation, macros, environment promotion, and CI/CD integration.
5+ years of experience with SQL and Python for data engineering, automation, transformation, and data quality validation.
3+ years of experience with pipeline orchestration tools.
Experience partnering directly with business departments, data owners, reporting teams, architects, compliance, and other technology groups to gather requirements, communicate design decisions, and deliver production data solutions.
Experience supporting production data pipelines, troubleshooting data quality issues, conducting root cause analysis, and implementing monitoring or preventive controls.
Expert-level ability to design maintainable data pipelines using SQL, Python, Snowflake, dbt, and enterprise orchestration patterns.
Strong understanding of production-grade engineering practices, including code reviews, version control, CI/CD, environment promotion, automated testing, documentation, monitoring, and incident response.
Ability to troubleshoot complex pipeline, transformation, performance, and data quality issues across multiple systems.
Advanced command of dbt for analytics engineering: incremental models, snapshots, seeds, macros, tests, and multi-environment deployment strategies.
Advanced hands-on experience with Snowflake, including schema design, role-based access patterns, warehouse sizing, query optimization, data sharing concepts, cost/performance tradeoffs, and integration with cloud storage and transformation tools.
Strong proficiency with cloud-native data warehouses and lakehouses: Snowflake, AWS Redshift, Azure Synapse Analytics, or GCP BigQuery.
Working knowledge of modern cloud data architecture patterns across Azure, AWS, or GCP, with preference for experience integrating Snowflake into enterprise data ecosystems.
Proficiency with containerization and orchestration: Docker and Kubernetes for deploying data workloads.
Demonstrated understanding of data engineering patterns that support AI/ML and advanced analytics use cases, including semantic model development, reusable feature data, model-ready datasets, feature store design and management, and scalable analytical data products.
Experience integrating AI-enabled workflows into data pipelines and analytics applications, including unstructured data processing, semantic search patterns, embedding-based approaches, and workflow orchestration.
Knowledge of MLOps and production analytics practices, including model lifecycle support, workflow automation, environment promotion, monitoring frameworks, governance considerations, performance monitoring, drift awareness, and continuous improvement of data inputs and outputs.
Familiarity with orchestrating AL agent workflows and tool-calling patterns using frameworks such as MCP or similar technologies.
Proficiency with real-time and event-driven data platform concepts, including message streaming, event ingestion, pub/sub patterns, and integration with enterprise data platforms.
Experience designing or supporting stream processing patterns for near-real-time transformation, enrichment, aggregation, and delivery of data to downstream consumers.
Knowledge of change data capture and incremental data movement patterns used to keep analytical, operational, and reporting data assets current and reliable.
Strong understanding of data quality frameworks, automated data observability and lineage tracking (Atlan, Monte Carlo).
Experience with metadata management, data cataloging (Collibra, Alation, or Apache Atlas), and data classification practices in regulated environments.
Familiarity with data mesh and data product ownership principles.
Ability to design scalable, cost-optimized data architectures applying medallion (Bronze/Silver/Gold), data vault, or dimensional modeling patterns.
Strong technical communication skills: able to produce architecture decision records (ADRs), design documents, and present complex technical concepts to both technical peers and non-technical business stakeholders.
Ability to facilitate requirements gathering and design sessions with business stakeholders and technical teams.
Proven ability to influence cross-functional teams and drive consensus on enterprise data initiatives.
Proven ability to mentor engineers, lead code and design reviews, and drive adoption of engineering best practices across a team.
Detail-oriented with strong analytical skills; able to organize complex data landscapes and identify actionable insights and improvement opportunities.
Exceptional written and verbal communication skills with the ability to facilitate requirements workshops, present architectural recommendations, document design decisions, communicate project status, and influence stakeholders across technical and non-technical audiences.
Demonstrated initiative and ownership in ambiguous, fast-paced environments.
Ability to sit for extended periods and work extensively on a computer for sustained periods.

Nice To Haves

Master’s degree in Computer Science, Data Engineering, Information Systems, Data Science, or related discipline.
Experience with streaming, lakehouse architecture, metadata management, data observability, or AI/ML data preparation is preferred.
Demonstrated experience architecting ML/AI data infrastructure including feature stores, MLOps pipelines, and LLM-based data processing workflows.
Prior experience in financial services, banking, or another regulated industry is strongly preferred.
Cloud data platform certification such as AWS Certified Data Analytics – Specialty, Microsoft Certified: Azure Data Engineer Associate, GCP Professional Data Engineer, or Snowflake SnowPro Certifications (Core, Associate, Advanced).
dbt Certification or equivalent analytics engineering credential.
Experience with lakehouse storage formats: Delta Lake, Apache Iceberg, or Apache Hudi.
Exposure to vector databases or retrieval-augmented generation (RAG) pipelines is preferred.

Responsibilities

Architect, design, and deliver enterprise data platform solutions leveraging Snowflake, dbt, SQL, Python, and modern data integration technologies, ensuring scalable, secure, reliable, and maintainable data pipelines and analytical data products.
Establish and enforce data engineering standards, best practices, and governance frameworks for code quality, CI/CD, data testing, observability, documentation, metadata management, security, and operational excellence.
Partner with business stakeholders, data owners, architects, compliance teams, and technology partners to translate business requirements into data solutions, data models, integration patterns, and implementation roadmaps that deliver measurable business value.
Lead the support, monitoring, and continuous improvement of enterprise data platforms by resolving complex production issues, improving performance, enhancing data quality, and ensuring operational reliability.
Provide technical leadership, mentorship, and strategic direction for the data engineering practice, including architectural reviews, coaching engineers, evaluating emerging technologies, and supporting advanced analytics, AI/ML, and enterprise data initiatives.