Equifax-posted 9 days ago
Full-time • Mid Level
Alpharetta, GA
5,001-10,000 employees

Equifax is where you can power your possible. If you want to achieve your true potential, chart new paths, develop new skills, collaborate with bright minds, and make a meaningful impact, we want to hear from you. We're looking for a highly skilled and experienced Senior Data Engineer to join our team. You'll be instrumental in designing, building, and maintaining our next-generation Data Ingestion platform and its infrastructure. This role requires a deep understanding of distributed systems and hands-on expertise with cutting-edge big data technologies in a cloud-native environment. You will be responsible for creating robust, scalable, and fault-tolerant data pipelines that process massive datasets for both real-time and batch-based analytics. Your work will directly enable our data scientists, analysts, and business teams to derive critical insights and make data-driven decisions.

  • Apply the knowledge of data characteristics and data supply pattern, develop rules and tracking process to support data quality model.
  • Prepare data for analytical use by building data pipelines to gather data from multiple sources and systems.
  • Integrate, consolidate, cleanse and structure data for use by our clients in our solutions.
  • Perform design, creation, and interpretation of large and highly complex datasets.
  • Stay up-to-date with the latest trends and advancements in GCP and related technologies, actively proposing and evaluating new solutions.
  • Understand best practices for data management, maintenance, reporting and security and use that knowledge to implement improvements in our solutions.
  • Implement security best practices in pipelines and infrastructure.
  • Develop and implement data quality checks and troubleshoot data anomalies.
  • Provide guidance and mentorship to junior data engineers.
  • Review dataset implementations performed by junior data engineers.
  • BS degree in a STEM major or equivalent discipline; Master’s Degree strongly preferred
  • 5+ years of experience as a data engineer or related role
  • 5+ years of software engineering experience
  • Advanced skills using programming languages such as Python or SQL and intermediate level experience with scripting languages
  • 5+ years experience with Cloud technology: GCP, AWS, or Azure
  • 5+ years experience designing and developing cloud-native solutions
  • Experience building and maintaining moderately-complex data pipelines, troubleshooting issues, transforming and entering data into a data pipeline in order for the content to be digested and usable for future projects
  • Demonstrates advanced Git usage and CI/CD integration skills
  • 5+ years of hands-on experience in data engineering or a related field.
  • Expert-level proficiency in SQL, Python or relevant programming languages
  • Demonstrated experience designing and implementing distributed data processing solutions at scale.
  • In-depth, hands-on experience with BigData tools such as Apache Spark or Apache Beam.
  • Proven experience with Google Cloud Platform (GCP) services for data engineering, including Dataflow, Dataproc, BigQuery, and Pub/Sub.
  • Experience with workflow orchestration tools like Apache Airflow (or GCP's Cloud Composer).
  • Solid understanding of data modeling, data warehousing, and ETL/ELT best practices.
  • Background of Large Data Analysis and building AI ML Models would be a huge plus.
  • Excellent problem-solving skills and the ability to troubleshoot complex distributed systems.
  • Effective communication and collaboration skills, with the ability to articulate technical concepts to both technical and non-technical stakeholders
  • comprehensive compensation and healthcare packages
  • 401k matching
  • paid time off
  • organizational growth potential through our online learning platform with guided career tracks
© 2024 Teal Labs, Inc
Privacy PolicyTerms of Service