Data Platform Software Engineer
About The Position
We are seeking a skilled Data Platform Software Engineer to join our Data Platform team in Kansas City, MO. In this role, you will design, code, implement, support, and optimize a cloud-native data stack that leverages best-in-class open-source tools. The ideal candidate will design, build, support, and maintain an opinionated, resilient, and scalable data platform in a private cloud environment—enabling data-driven decision-making, analytics, and machine learning, while providing out-of-the-box deep insights. This role blends data engineering, software development, and infrastructure management, leveraging languages and tools such as Java, Python, Apache Iceberg, Flink, Spark, Kafka, and Trino.
Requirements
- Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related field.
- 3+ years of experience in software and data engineering, with a strong focus on cloud-based data platforms.
- Strong programming skills in Java or Python.
- Knowledge of Apache Iceberg, Flink, Spark, Kafka, and Trino.
- Familiar with metadata management platforms like DataHub or Apache Atlas.
- Familiar with cloud-native infrastructure tools such as Kubernetes, Ansible, Salt, etc.
- Familiar with CI/CD and DevOps operations.
- Strong analytical and problem-solving skills.
- Effective communication and collaboration with cross-functional teams.
Responsibilities
- Develop and maintain robust, fault-tolerant data ingestion and transformation pipelines using Java, Python, Apache Flink, Spark, and Trino.
- Define flexible and scalable data schemas using Apache Iceberg.
- Support both batch and real-time data processing, including integration with Apache Kafka.
- Ensure reliability, observability, and integrity of data pipelines.
- Support metadata management platforms such as DataHub for data cataloging, lineage, and governance use cases.
- Collaborate with data stakeholders to align metadata solutions with organizational needs.
- Enforce governance policies related to data quality, privacy, and compliance (e.g., GDPR, CCPA).
- Implement fine-grained access controls, encryption, and auditing with a focus on regulatory compliance and data traceability.
- Automate data pipelines, workflows, deployment, and infrastructure provisioning and configuration using tools like ArgoCD, Apache Airflow, Ansible, Salt, and Kubernetes.
- Implement CI/CD pipelines, actions, and workflows on GitHub to deploy updates and enhancements.
- Optimize data storage and queries using Apache Iceberg, Flink, and Spark to ensure high performance and low-latency access.
- Identify and address performance bottlenecks; implement partitioning, caching, and indexing strategies.
- Monitor data platform health using tools such as Prometheus and Grafana dashboards.
- Configure real-time alerts to proactively detect and resolve pipeline failures or data issues.
- Troubleshoot and resolve platform outages and data incidents promptly.
- Work with other development teams and engineers to understand data needs and deliver performant, scalable solutions.
- Collaborate with cross-functional teams (Cloud Engineering, Network, and DevOps/Solutions Engineering) to troubleshoot and resolve infrastructure issues.
Benefits
- medical, dental, and vision coverage
- a 401(k) plan with company match
- paid time off, holidays, and parental leave
- professional development reimbursement opportunity
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Mid Level
