Senior Data Engineer

Vericast•San Antonio, TX

3d•$130,000 - $150,000

About The Position

Vericast is seeking a Senior Data Engineer to build big data pipelines, automate marketing campaigns, and support data requirements for analytics, reporting, and AI applications. The role expands data infrastructure and pipeline architecture and optimizes data flow and collection into the Data Lake environment. The Senior Data Engineer works closely with analytics, reporting, data science, AI, and business teams to provide consistent, optimal data delivery architecture across projects. The position supports the data needs of multiple teams, systems, and products and contributes to high-quality, data-driven marketing products in an agile environment, collaborating with developers and business partners.

Requirements

Bachelor's degree in Computer Science, Information Technology, or a related field, with 5+ years of relevant experience; alternatively, a Master's degree in one of these fields.
5+ years of experience in Data Engineering, ETL development, or Data Platform Engineering.
Experience in a Financial Institution, Banking, FinTech, MarTech, AdTech, Marketing Services, or Marketing Agency environment.
Experience supporting customer, campaign, audience, marketing performance, attribution, advertising, or transactional data at scale.
Strong experience with Python and PySpark for building production-grade data pipelines.
Experience designing, building, and supporting Data Lakehouse environments.
Experience with Apache Airflow, Iceberg, Hive, S3, and Trino.
Experience with cloud platforms such as AWS, Azure, or GCP.
Experience with Agile software development methodologies.
Experience with GitLab and CI/CD processes.
Experience supporting AI and machine learning applications.
Understanding of machine learning models and the data requirements needed to support Data Science teams.
Programming skills in object-oriented or functional scripting languages such as Python and PySpark, including testing, logging, and data observability.
Experience with distributed systems and parallel data processing using tools such as Spark, PySpark, Hadoop, Kafka, and Hive.
Proficiency with relational databases.
Strong knowledge of Linux/Unix-based systems.
Hands-on experience with Apache Ranger and Rancher/Kubernetes.
Strong analytical, conceptual, and problem-solving skills.
Strong communication skills that promote cross-team collaboration.

Nice To Haves

Master's degree in Computer Science, Information Technology, or a related field.
Experience supporting data requirements for AI applications.
Understanding of machine learning models and algorithms, with the ability to interface with the Data Science team.
Experience building REST APIs.

Responsibilities

Develop scalable data pipelines and build integrations to support growth in data volume and complexity.
Build a robust Data Lakehouse for Vericast's marketing solutions business and financial institution clients, supporting pre-sales, campaign execution, campaign performance analysis, and benchmarking.
Collaborate with AI, Analytics, and business teams to improve data models that feed AI and analytics tools, increase data accessibility, and foster data-driven decision-making.
Implement processes and systems to monitor data quality and ensure production data availability and accuracy for stakeholders and dependent business processes.
Analyze data to troubleshoot data-related issues and support resolution.
Provide post-deployment support and respond quickly to unexpected production issues.
Partner with business units and engineering teams to shape long-term data platform architecture strategy.