Kafka Platform Engineer
About The Position
The Kafka Platform Engineer is responsible for engineering and operating the enterprise Kafka platform, the company’s strategic data movement backbone. The role focuses on platform stability, scalability, data safety, and enabling producer and consumer teams. The engineer deploys and manages Kafka connectors in Confluent environments, containerizes and deploys microservices and connectors on OpenShift, implements CI/CD pipelines, performs Kafka administration, and supports production deployments and troubleshooting. The position also supports real-time streaming between on-premises systems and cloud or SaaS platforms, builds custom connectors, automates operational tasks, and owns integration deliverables from design through production deployment. The engineer serves as a subject matter expert for Kafka-based integrations and cluster administration, coordinating with business teams, architects, product owners, platform teams, and co-sourced/offshore teams.
Requirements
- Strong hands-on Kafka experience in production environments.
- Understanding of distributed systems, messaging semantics, and fault tolerance.
- Experience with Kafka deployments on Kubernetes platforms such as OpenShift and Docker.
- Experience with multi-region high-availability on-premises deployments.
- Experience building producer/consumer applications using Kafka connectors.
- Experience building custom microservices in Python or Java.
- Experience with Kafka security, including TLS, ACLs, and RBAC.
- Ability to troubleshoot complex, cross-team data-flow issues.
- Bachelor's degree in computer science, engineering, or a related technical field.
- Seven years of relevant work experience; an equivalent combination of education and related experience may be considered.
Nice To Haves
- Experience with Schema Registry, Kafka Connect, and MirrorMaker.
- Experience with Oracle Golden Gate or a similar CDC tool for data movement.
- Familiarity with enterprise integration patterns and platforms such as APIM, Camunda, webMethods, or Oracle SOA.
- Confluent Certified Administrator for Apache Kafka® (CCAAK).
- Confluent Certified Developer for Apache Kafka® (CCDAK).
- Confluent Certified Cloud Operator (CCAC).
Responsibilities
- Deploy and manage Kafka connectors, including CDC, JDBC, IBM MQ, and Redshift connectors, in Confluent Kafka environments.
- Containerize and deploy microservices and connectors on OpenShift using Docker, YAML, and Kustomize.
- Implement CI/CD pipelines using GitHub Actions and Argo CD to automate deployments.
- Perform Kafka administration tasks, including topic creation, partitioning, replication, and monitoring.
- Collaborate with architects and business teams to understand integration requirements and deliver scalable solutions.
- Monitor Kafka clusters and connector health using Confluent Control Center, Prometheus, Grafana, Dynatrace, Splunk, and OpenShift-native monitoring.
- Build and maintain custom connectors for specialized data sources and sinks.
- Support production deployments and troubleshoot issues across Kafka pipelines and OpenShift environments.
- Support real-time data streaming between on-premises systems and cloud or SaaS platforms.
- Coordinate with co-sourced/offshore teams on development and deployment tasks.
- Conduct code reviews and quality checks and ensure adherence to standards.
- Apply 12-Factor principles and microservices architecture to support the reliability and scalability of Kafka pipelines.
- Align Kafka solutions with enterprise integration patterns and cloud strategies in collaboration with architects and platform teams.
- Automate deployment, monitoring, and operational tasks wherever possible.
- Own integration deliverables from design through production deployment.
- Act as the subject matter expert for Kafka-based integrations and cluster administration.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
