Principal Software Engineer, Prisma Access

Palo Alto Networks•Santa Clara, CA

9d•$147,000 - $237,500•Onsite

About The Position

The Team Engineering - Our engineering team is at the core of our products and connected directly to the mission of preventing cyberattacks. We are constantly innovating — challenging the way we, and the industry, think about cybersecurity. Our engineers don’t shy away from building products to solve problems no one has pursued before. We define the industry instead of waiting for directions. We need individuals who feel comfortable in ambiguity, excited by the prospect of a challenge, and empowered by the unknown risks facing our everyday lives that are only enabled by a secure digital environment. Your Career As a Principal Engineer on the Prisma Access team, you’ll design and build the distributed backend services that form the backbone of our platform. You’ll think broadly about all system components, weigh trade-offs for every design decision, and work with cutting-edge cloud technologies — integrating service APIs across multiple cloud providers (AWS, GCP, Azure, OCI) to deliver an optimal customer experience. This is an AI-native engineering role. You won’t just use AI as a productivity tool — you’ll build LLM-powered and agentic features directly into the product, design systems that detect and defend against AI-era threats, and operate with AI-assisted development workflows (Cursor, Claude Code, Copilot, internal agents) as a daily standard. You’ll also shape our monitoring infrastructure, optimize data collection pipelines, analyze system disruptions, and develop solutions that drive measurable improvements in reliability. This is a rare opportunity to take ownership of a new product architecture and build it from the ground up, defining how AI changes what a cloud security platform can do.

Requirements

B.S. or M.S. in Computer Science, Electrical Engineering, or a related technical field — or equivalent practical experience
5+ years of professional software engineering experience building production-grade backend systems
Proficiency in one or more of: Go, Python, Java, or C++
Strong fundamentals in data structures, algorithms, operating systems, networking, and distributed systems
Hands-on experience architecting services on at least one major public cloud — AWS, GCP, Azure, or OCI
Working knowledge of networking protocols (TCP/IP, HTTP/HTTPS, TLS) and cloud network architectures (VPCs, load balancers, firewalls, NAT, peering)
Proven experience developing and scaling complex distributed systems, including large-scale distributed databases (e.g., Cassandra, DynamoDB, Spanner, CockroachDB) and messaging/streaming systems (e.g., Kafka, Pub/Sub, SQS, NATS)
Hands-on experience integrating LLMs into production services — API-based (OpenAI, Anthropic, Gemini) or self-hosted — including prompt engineering, structured outputs, function/tool calling, and basic eval design
Fluency with AI-assisted development workflows — daily use of tools like Cursor, Claude Code, GitHub Copilot, or similar; able to apply them effectively without over-trusting them
Strong written and verbal communication — able to author technical proposals, design specs, and architecture diagrams, and present them to both technical and non-technical audiences

Nice To Haves

Production experience with agentic AI frameworks — LangChain, LangGraph, LlamaIndex, CrewAI, Semantic Kernel, or equivalents
Experience designing RAG architectures over enterprise data, including chunking strategies, embedding models, and vector databases (Pinecone, Weaviate, pgvector, Milvus, OpenSearch k-NN)
Familiarity with the Model Context Protocol (MCP) and patterns for exposing internal systems as tools to LLMs and agents
Experience building evaluation pipelines for LLM systems — offline evals, A/B testing, golden datasets, regression suites, LLM-as-judge patterns

Responsibilities

Analyze requirements, design, develop, and support highly scalable software features and infrastructure on our next-generation security platform — taking features from inception to production, ready for cloud-native deployment
Architect and build distributed microservices that process traffic and configuration at scale for thousands of enterprise customers
Write clean, testable, readable, and maintainable code that performs reliably under production load
Build and automate monitoring, observability, and alerting infrastructure to ensure platform reliability and rapid incident response
Participate in on-call rotations for production services and contribute to a culture of operational excellence
Design and ship LLM-powered and agentic product features — including policy authoring assistants, intelligent troubleshooting agents, automated RCA, anomaly detection, and natural-language admin experiences
Build production-grade RAG pipelines, tool-using agents, and multi-agent workflows over security telemetry, configuration data, and threat intelligence
Develop evaluation harnesses, golden traces, and CI/CD quality gates for AI features — measuring accuracy, latency, cost, and safety with the same rigor applied to traditional services
Implement guardrails, grounding, and human-in-the-loop controls to minimize hallucinations and ensure trustworthy behavior in security-critical contexts
Apply ML/AI techniques to networking and orchestration problems: traffic classification, capacity prediction, smart routing, drift detection, and zero-day anomaly identification
Collaborate cross-functionally with Product Management, Development, QA, SRE, and Customer Support to deliver the roadmap and improve customer outcomes
Actively guide testing strategy for critical components — including AI-powered test generation and autonomous QA workflows — balancing performance, supportability, and maintainability
Mentor junior engineers, lead design reviews, and contribute to engineering best practices, including responsible adoption of AI-assisted development workflows
Drive a results-oriented culture with a strong focus on execution, quality, and speed