Kafka Platform Engineer
About The Position
The Kafka Platform Engineer is responsible for engineering and operating the enterprise Kafka platform, the company's strategic data movement backbone. The role focuses on platform stability, scalability, data safety, and enabling producer and consumer teams. It supports real-time data streaming between on-premises systems and cloud and SaaS platforms, and owns integration deliverables from design through production deployment. The engineer deploys and manages connectors in Confluent Kafka environments, containerizes and deploys microservices and connectors on OpenShift, and implements automated CI/CD. The position involves Kafka administration, building custom connectors, monitoring platform and connector health, production support, troubleshooting, and automation. It also requires close collaboration with business teams, architects, product owners, platform teams, and co-sourced/offshore teams. The engineer serves as a subject matter expert for Kafka-based integrations and cluster administration. A bachelor's degree in computer science, engineering, or a related technical field and seven years of relevant work experience are required. An equivalent combination of education and related experience may be considered. The posting lists Confluent Certified Administrator for Apache Kafka (CCAAK), Confluent Certified Developer for Apache Kafka (CCDAK), and Confluent Certified Cloud Operator (CCAC) certifications.
Requirements
- Strong hands-on Kafka experience in production environments.
- Understanding of distributed systems, messaging semantics, and fault tolerance.
- Experience with Kafka deployments on Kubernetes platforms, including OpenShift and Docker.
- Experience with multi-region high-availability on-premises deployments.
- Experience building producer and consumer applications using Kafka connectors.
- Experience building custom microservices in Python or Java.
- Experience with Kafka security, including TLS, ACLs, and RBAC.
- Ability to troubleshoot complex, cross-team data-flow issues.
- Bachelor's degree in computer science, engineering, or a related technical field; an equivalent combination of education and related experience may be considered.
- Seven years of relevant work experience; an equivalent combination of education and related experience may be considered.
- The posting lists Confluent Certified Administrator for Apache Kafka (CCAAK), Confluent Certified Developer for Apache Kafka (CCDAK), and Confluent Certified Cloud Operator (CCAC) certifications.
Nice To Haves
- Experience with Schema Registry, Kafka Connect, and MirrorMaker.
- Experience with Oracle Golden Gate or a similar CDC tool for data movement.
- Familiarity with enterprise integration patterns and platforms such as APIM, Camunda, webMethods, or Oracle SOA.
Responsibilities
- Deploy and manage Kafka connectors, including CDC, JDBC, IBM MQ, and Redshift connectors, in Confluent Kafka environments.
- Containerize and deploy microservices and connectors on OpenShift using Docker, YAML, and Kustomize.
- Implement CI/CD pipelines using GitHub Actions and Argo CD for automated deployments.
- Perform Kafka administration tasks, including topic creation, partitioning, replication, and monitoring.
- Collaborate with architects and business teams to understand integration requirements and deliver scalable solutions.
- Monitor Kafka clusters and connector health using Confluent Control Center, Prometheus, Grafana, Dynatrace, Splunk, and OpenShift-native monitoring.
- Build and maintain custom connectors for specialized data sources and sinks.
- Support production deployments and troubleshoot issues across Kafka pipelines and OpenShift environments.
- Deploy Confluent connectors for data integrations using cloud managed services, including AWS and Azure.
- Support real-time data streaming between on-premises systems and cloud or SaaS platforms.
- Coordinate with co-sourced and offshore teams on development and deployment tasks.
- Conduct code reviews and quality checks, and ensure adherence to standards.
- Apply 12-Factor principles and microservices architecture to ensure Kafka pipeline reliability and scalability.
- Align Kafka solutions with enterprise integration patterns and cloud strategies in collaboration with architects and platform teams.
- Automate deployment, monitoring, and operational tasks wherever possible.
- Own integration deliverables from design through production deployment.
- Act as a subject matter expert for Kafka-based integrations and cluster administration.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
