Senior Data Engineer
About The Position
Vericast is seeking a Senior Data Engineer to build big data pipelines, automate marketing campaigns, and support data requirements for analytics, reporting, and AI applications. The role expands data infrastructure and pipeline architecture while optimizing data flow and collection into the Data Lake environment. The engineer works with analytics, reporting, and data science teams to establish consistent, effective data delivery architecture across projects, supporting the needs of multiple teams, systems, and products in an agile environment. The work contributes to data-driven marketing products in collaboration with developers and business partners.
Requirements
- Bachelor’s degree in Computer Science, Information Technology, or a related field and 5+ years of relevant experience, or a master’s degree in one of these fields.
- At least 5 years of experience in data engineering, ETL development, or data platform engineering.
- Experience in a financial institution, banking, FinTech, MarTech, AdTech, marketing services, or marketing agency environment.
- Experience supporting customer, campaign, audience, marketing performance, attribution, advertising, or transactional data at scale.
- Strong experience with Python and PySpark for production-grade data pipelines.
- Experience designing, building, and supporting Data Lakehouse environments.
- Experience with Apache Airflow, Iceberg, Hive, S3, and Trino.
- Experience with AWS, Azure, or GCP.
- Experience with Agile software development methodologies.
- Experience with GitLab and CI/CD processes.
- Experience supporting AI and machine learning applications.
- Understanding of machine learning models and the data requirements of data science teams.
- Strong programming skills in object-oriented or functional scripting languages such as Python and PySpark, including testing, logging, and data observability.
- Experience with distributed systems and parallel data processing tools such as Spark, PySpark, Hadoop, Kafka, and Hive.
- Proficiency with relational databases.
- Strong knowledge of Linux/Unix-based systems.
- Hands-on experience with Apache Ranger and Rancher/Kubernetes.
- Excellent analytical, conceptual, and problem-solving skills.
- Strong communication skills that promote cross-team collaboration.
Nice To Haves
- Master’s degree in Computer Science, Information Technology, or a related field.
- Experience building REST APIs.
- Experience supporting data requirements for AI applications.
- Understanding of machine learning models and algorithms and ability to interface with the Data Science team.
Responsibilities
- Develop scalable data pipelines and build integrations to support increasing data volume and complexity.
- Build a robust Data Lakehouse supporting Vericast’s marketing solutions and financial institution clients, including presales, campaign execution, performance analysis, and benchmarking.
- Collaborate with AI, analytics, and business teams to improve data models that feed AI and analytics tools and increase data accessibility.
- Implement processes and systems to monitor data quality and ensure production data availability and accuracy.
- Analyze data to troubleshoot data-related issues and support their resolution.
- Provide post-deployment support and respond to unexpected production issues.
- Partner with business units and engineering teams to shape long-term data platform architecture strategy.
Benefits
- Medical coverage
- Dental coverage
- Vision coverage
- 401(k)
- Generous paid time off allowance
- Life insurance
- Employee assistance
- Pet insurance
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
