Data Engineer

MFour Mobile Research•Kansas City, KS

6d•Onsite

About The Position

The Data Engineer will design, implement, and manage scalable data pipelines, databases, and processing systems to support engineering, product, and analytics teams. This includes developing and administering platforms like Databricks and Snowflake, optimizing data architecture, and ensuring performance, scalability, and reliability.

Requirements

Bachelor’s degree in Computer Science, Engineering, or a related field (or equivalent experience).
3+ years of professional experience in data engineering or related fields.
Proficiency in Databricks development and cluster administration.
Hands-on experience with Snowflake development and administration, including performance tuning and security configuration.
Strong understanding of relational databases, particularly PostgreSQL and MySQL.
Expertise in programming languages like Python, Scala, or SQL for data processing and automation.
Familiarity with ETL/ELT tools and frameworks such as Apache Airflow or dbt.
Experience with cloud platforms (AWS, Azure, or GCP).

Nice To Haves

Knowledge of distributed systems and big data frameworks like Spark.
Experience with CI/CD pipelines for deploying data workflows.
Familiarity with real-time streaming technologies (Kafka, Kinesis, etc.).

Responsibilities

Design, build, and maintain scalable data pipelines to process large volumes of structured and unstructured data.
Optimize data architecture for both real-time and batch processing workflows.
Integrate and maintain PostgreSQL, MySQL, Databricks, and Snowflake within our data ecosystem.
Develop and manage workflows on Databricks to support ETL/ELT processes.
Optimize Databricks performance for distributed computing and data processing.
Monitor, troubleshoot, and ensure the reliability of Databricks clusters and jobs.
Design and optimize Snowflake schemas and data warehouses for efficient querying.
Develop and manage data ingestion pipelines for Snowflake using modern tools and frameworks.
Administer Snowflake environments, including account management, performance tuning, and resource optimization.
Administer PostgreSQL and MySQL databases, ensuring high availability and performance.
Implement and enforce data governance practices to ensure data security and compliance.
Partner with analytics and engineering teams to define data needs and deliver solutions for business insights.
Work with product teams to align data solutions with product goals and user requirements.
Set up monitoring and alerting systems for data platforms and pipelines.
Proactively debug and resolve data-related issues to ensure system reliability.
Stay current with emerging trends and advocate for tools and practices that enhance data engineering efficiency.
Document workflows, pipelines, and platform configurations to foster team collaboration and knowledge sharing.