Senior Kafka Platform Engineer

Charles Schwab Inc.•Austin, TX

7d•$164,000 - $180,000•Onsite

About The Position

Event Streaming team in Enterprise Middleware organization is seeking a Senior Kafka Platform Engineer with experience to lead the evolution of its enterprise event-streaming platform and drive enterprise-scale event streaming, Confluent Cloud adoption, platform efficiency and AI-enabled SDLC transformation. This role will provide technical leadership across Confluent Platform and Confluent Cloud, drive cloud modernization and platform resiliency, and build automation to enhance scalability, security, reliability, performance, stability, and developer productivity. The ideal candidate combines deep Kafka administration expertise with a strong Java development background, hands-on experience in cloud and Kubernetes environments, and the ability to apply AI-assisted engineering to accelerate delivery, improve operational efficiency, and support secure, business-critical solutions at scale.

Requirements

Bachelor’s degree in computer science, Engineering, or related field
10+ years of hands-on software engineering experience with proven technical leadership
5+ years of expertise in large-scale distributed systems and mission-critical infrastructure
Strong hands-on experience with GitHub Copilot and other AI-assisted development tools, including advanced prompt engineering and workflow integration to accelerate engineering
Proven ability to leverage AI-assisted development (e.g., Copilot, code generation) and build AI-driven agents/workflows to improve team productivity and engineering efficiency
Deep expertise in Confluent Kafka (capacity planning, administration, internals, performance tuning)
Strong experience with Kafka clusters, replication, disaster recovery, and data pipelines
Hands-on experience with Kubernetes, Confluent Operator, and cloud-native modernization (GCP)
Strong development background on Linux/Unix platforms
Experience with data modeling and streaming technologies (Avro, Protobuf, JSON Schema)
Strong automation mindset with hands-on expertise in Python, Ansible, and CI/CD pipelines (Jenkins, DevOps tooling) to drive operational efficiency and reduce manual effort
Solid foundation in security engineering (encryption, TLS, certificate and key management)
Strong focus on observability, reliability, automation, and production readiness with proactive risk mitigation
Experience leading architecture and execution across cross-functional, large-scale programs
Strong understanding of networking, load balancing, OS fundamentals, and performance optimization
Experience in regulated environments (financial services preferred)
Strong collaboration skills across engineering, architecture, infrastructure, and business teams
Experience working with globally distributed/offshore teams
Strong Agile delivery experience (Scrum/Kanban/Scaled Agile)
Strong leadership and soft skills including strategic thinking, decision-making, mentoring/coaching, and driving engineering excellence
Excellent communication, influencing, and stakeholder management skills, with ability to align diverse teams toward common goals
High ownership mindset with ability to prioritize, multitask, and lead through high-pressure production situations
Strong problem-solving skills with a proactive, innovation-driven approach
Self-driven, adaptable, and customer-focused with strong relationship management and team-building capabilities

Nice To Haves

Experience applying AI-Driven Development Lifecycle (AI-DLC) practices to modernize SDLC pipelines through intelligent automation, improving development, testing, and release efficiency
Demonstrated contributions to architecture standards, reusable frameworks, or open-source initiatives in distributed systems or event-driven platforms
Deep expertise in Kafka platform administration and development, including large-scale event streaming systems and platform modernization in enterprise environments
Certifications in Confluent Kafka, Confluent Cloud, and major cloud platforms (GCP/AWS/Azure) are a strong plus.

Responsibilities

Design and build AI agents, custom skills, or equivalent frameworks to automate engineering and operational workflows at scale
Leverage AI-powered tools (e.g., Copilot, AI agents) to accelerate development, automate testing, improve code quality, and enable intelligent SDLC workflows aligned to AI-DLC vision.
Design agentic workflows and multi-agent orchestration patterns to solve complex distributed systems, integration, and operational challenges
Act as the senior technical authority for Confluent Platform and Confluent Cloud, providing architecture, design, and implementation guidance to solve complex enterprise integration challenges at scale.
Lead solution architecture across cross-functional initiatives, ensuring alignment with cloud modernization, resiliency, security, and AI-driven engineering priorities.
Perform deep-dive analysis of complex production issues, drive root cause identification, and implement sustainable remediation strategies with full documentation.
Design and build automation frameworks and self-service tools to streamline Kafka operations, reduce manual effort, and improve developer productivity and platform scalability.
Lead seamless migration of on-prem Kafka workloads to Confluent Cloud using cluster linking, schema portability, and hybrid streaming patterns—ensuring zero business disruption.
Drive execution across Kafka, infrastructure, security, and application teams to remove dependencies, mitigate risks, and ensure on-time delivery of committed milestones.
Contribute to enterprise architecture forums, defining and enforcing best practices for event streaming, integration patterns, and platform standardization.
Establish best practices for testing, reliability, observability, and operational readiness, ensuring high availability and scalable platform performance.
Identify systemic and architectural risks early, leading mitigation efforts to prevent client impact and ensure platform stability.
Implement enterprise-grade security controls (mTLS, RBAC, encryption) and drive continuous remediation of vulnerabilities, ensuring compliance with regulatory and audit requirements.
Interpret technical requirements to build and enhance Kafka configurations, message flows, and integration solutions supporting business-critical applications.
Partner with distributed teams (including offshore engineering) to ensure consistent standards, efficient delivery, and 24x7 operational support across time zones.