About The Position

The Senior Data Engineer works with minimal instruction to design, implement, and optimize complex data solutions. This role is responsible for non-routine data assignments, involving the development and automation of data systems, writing advanced SQL queries, and resolving moderately complex data issues. The engineer plays a key role in cross-functional collaboration and data-driven decision-making, ensuring the timely availability of accurate data for business needs. Note: This role may come into contact with confidential or sensitive customer information requiring special treatment in accordance with Red Hat policies and applicable privacy laws.

Requirements

  • Bachelors in Computer Science, Engineering, Information Systems or related field OR Bachelor's degree with demonstrable coding experience
  • 3+ years of experience in pipeline architecture, production deployment
  • 4+ years of Python programming experience
  • 3+ years of demonstrable experience supporting business intelligence/analytics stakeholders
  • Demonstrable experience handling data engineering tool migrations

Nice To Haves

  • Tools: Snowflake, Databricks, Git, S3, Airflow, OpenShift (Kubernetes), Tableau, REST APIs, DBT
  • Data Modeling: previously built analytics apps with tools such as Tableau, PowerBI, or Qlik Sense

Responsibilities

  • Pipeline Architecture: Architect and implement complex, high-volume data pipelines between Snowflake and Databricks utilizing PySpark, dbt, and Astronomer/Airflow to solve marketing attribution and targeting challenges. Contribute to and maintain various DBT projects in the marketing domain.
  • Reporting and Data Science Enablement: Support mature reporting suite, including keeping the lights on for existing BI reporting, standing up new data products for consumers, and providing key inputs for data science modeling. Support efforts for AI ready data and emerging AI functionalities.
  • Best Practices & CI/CD: Champion and implement engineering best practices, including managing CI/CD pipelines with Git and Tekton to ensure reliable, repeatable code delivery in line with company objectives
  • Production Deployment: Lead the deployment and maintenance of containerized data science models and API Ingestion Pipelines on Red Hat OpenShift (Kubernetes), managing secrets, network routes and container orchestration.
  • SQL Expert: Use SQL to perform UAT for data migrations, investigate data anomalies, and fix pipeline bugs. Write and update advanced SQL using window functions, common table expressions, and transformations (e.g. Flatten).
  • Security, Compliance and Vulnerability Management: Lead application security initiatives by integrating automated code quality and vulnerability assessments (e.g., SonarQube) into pipelines. Collaborate directly with Information Security (InfoSec) teams to navigate compliance audits, remediate security findings, and adhere to strict enterprise security guidelines.

Benefits

  • Comprehensive medical, dental, and vision coverage
  • Flexible Spending Account - healthcare and dependent care
  • Health Savings Account - high deductible medical plan
  • Retirement 401(k) with employer match
  • Paid time off and holidays
  • Paid parental leave plans for all new parents
  • Leave benefits including disability, paid family medical leave, and paid military leave
  • Additional benefits including employee stock purchase plan, family planning reimbursement, tuition reimbursement, transportation expense account, employee assistance program, and more!
© 2024 Teal Labs, Inc
Privacy PolicyTerms of Service