Senior Software Engineer - Streaming (Kafka)

Innovaccer Analytics

3h

About The Position

As a Senior Software Engineer on the Streaming team, you will own the messaging backbone of Innovaccer's on-premise platform: Kafka and Schema Registry running on OpenShift via Strimzi, plus the self-hosted queueing layer that replaces cloud-managed queue services. A large population of producer and consumer touchpoints across the platform moves from cloud queue services onto this backbone, and you will define the patterns, libraries, and infrastructure that make that shift reliable at scale.

Requirements

• B.E., B.Tech., M.Sc. degree in Computer Science or a related technical field.
• 5+ years of backend or data infrastructure experience, with significant production Kafka ownership covering both operations and client-side development.
• Experience running Kafka on Kubernetes (Strimzi preferred) and managing schema evolution with Schema Registry.
• Hands-on experience with at least one additional broker technology: RabbitMQ, ActiveMQ/Artemis, or NATS/JetStream.
• Strong grasp of messaging semantics: delivery guarantees, idempotency, ordering, and exactly-once processing patterns.
• Professional development experience with Java, Python, or Go.

Nice To Haves

• Experience with KEDA or event-driven autoscaling is a plus
• Regulated-environment experience is a plus

Responsibilities

• Deploy and operate Kafka and Schema Registry on OpenShift via Strimzi: topic and partition design, retention policies, security, and quota management.
• Redesign integrations built on cloud queue services (SQS/Service Bus patterns) onto self-hosted brokers, defining repeatable patterns for hundreds of producer and consumer touchpoints.
• Build shared client patterns and libraries: retries, dead-letter queues, idempotent consumers, ordering guarantees, and backpressure handling.
• Integrate messaging with the platform's function runtime through KEDA-driven scaling of queue consumers.
• Own capacity planning for the messaging tier: broker disk and IO sizing, throughput modeling, and growth headroom.
• Monitor consumer lag and broker health, run failure-mode testing, and harden the tier against node and site failures.

Benefits

• Generous Paid Time Off: 20 days of fixed time off per year, in addition to company holidays
• Best-in-Class Parental Leave
• Recognition & Rewards: monetary incentives and company-wide recognition
• Comprehensive Insurance Coverage: medical, dental, and vision insurance
• 100% company-paid short- and long-term disability and basic life insurance
• Optional perks: discounted legal aid and pet insurance

What This Job Offers

• Job Type: Full-time
• Career Level: Senior