Principal Software Engineer, AI Platform

Redpanda Data•San Francisco, CA

4h•Hybrid

About The Position

Redpanda is the first runtime and control plane for agent-data interaction — a unified platform that combines streaming, SQL analytics, and intelligent connectivity with the governance layer enterprise AI agents need in production. Built on infrastructure already trusted by Fortune 500 companies and fast-growing startups, Redpanda enforces governance entirely outside the agent's data path, controlling what agents see, limiting what they do, and capturing a tamper-proof record of everything they touch, so organizations never have to choose between innovation and control. About the Role: We're looking for a Principal Software Engineer to help architect and build Redpanda's Agentic Data Plane (ADP), the control and execution layer that powers how AI agents safely interact with real-world data across the technological sprawl of the modern enterprise. Redpanda’s Agentic Data Plane enables intelligent agents to autonomously access, reason over, and act on enterprise data in real time, while providing their human counterparts with the visibility and control to ensure and validate that they do so securely. It combines Redpanda’s industry-leading low-latency streaming layer, a distributed query engine for real-time context, 300+ connectors for seamless data integration, and a global policy and observability framework that enforces access, records intent, and enables replayable audits. You'll work on identifying and integrating the next generation of ADP capabilities as the agentic AI landscape evolves, translating emerging research and industry shifts into concrete engineering proposals and product bets. You'll partner closely across Engineering, with our Product and GTM teams, as well as directly with strategic customers to shape the product’s architecture and direction. This is a hybrid role based in our San Francisco office.

Requirements

10+ years of software engineering experience, with a demonstrated track record of leading the design and delivery of complex, large-scale distributed systems
Experience building or contributing to emerging areas of agentic AI infrastructure, e.g. eval frameworks, orchestration layers, context management, AI gateways, or similar
Fluency with AI-assisted development at the level of agentic orchestration, i.e. composing multi-agent coding workflows with subagents to dramatically accelerate research and prototyping
Deep expertise in at least one systems language (Go preferred) and comfort working across the stack
Understanding of streaming data systems, messaging infrastructure, or data plane/control plane architectures, experience with Apache Kafka or Redpanda is a plus
Experience designing and operating systems that enforce access control, policy, and auditability at scale
Self-directed, strong written communicator, comfortable collaborating asynchronously in GitHub, Slack, and Google Docs
Excellent interpersonal and communication skills. Able to lead without authority across engineering teams and engage credibly with technical customers and external audiences

Nice To Haves

Familiarity with AI safety, alignment, or responsible AI research as it applies to autonomous systems in enterprise environments
Experience with protocol buffers, protoreflect and codegen
Background in security, identity, or data governance systems relevant to enterprise environments
Experience running Kafka/Redpanda and/or Postgres at scale
Experience with Kubernetes and cloud-native infrastructure across AWS, GCP, or Azure
Prior experience in an early-stage or growth-stage startup, especially in a category-defining product area

Responsibilities

Track emerging research, frameworks, and industry developments in the agentic AI space, and translate the most relevant signals into concrete engineering proposals and product bets
Identify, prototype, and deliver high-leverage emerging capabilities for ADP, in partnership with the broader team
Define architectural patterns for new ADP capabilities, ensuring they integrate cleanly with the existing runtime, policy, and observability layers
Engage directly with strategic customers and the field to stress-test new capabilities against real enterprise deployment scenarios
Serve as a technical ambassador for ADP externally, publishing research, writing engineering blog posts, and speaking at conferences to establish Redpanda's credibility at the frontier of agentic infrastructure
Participate in on-call and help drive down operational toil, improving system observability and self-healing capabilities over time