Principal AI Engineer - AI Ops

Palo Alto Networks • Office - USA - CA - Headquarters, CA

2d • $156,400 - $253,000 • Onsite

About The Position

Palo Alto Networks is hiring a Principal AI Engineer to lead the architecture, development, and deployment of enterprise-grade AI applications, real-time agentic workflows, and data-driven reasoning platforms. The role will define the long-term technical vision for enterprise AI, set standards for autonomous agent systems, guide architectural transformation, and bridge advanced AI capabilities with mission-critical production systems.

The engineer will shape enterprise AI and LLM deployment architectures, multi-agent systems, streaming and batch data and embedding pipelines, hybrid search and RAG architectures, and vector indexing systems. They will establish standards for AI evaluation, observability, cost and performance optimization, safety, hallucination reduction, and security compliance. The role also includes mentoring engineering teams and collaborating with executive leadership, Product, Security, Cloud Architecture, and cross-functional engineering teams.

The team uses technologies including Python, Java, Dataflow, BigQuery, Bigtable, Pub/Sub, Kubernetes, vector search infrastructure, and enterprise AI orchestration frameworks, integrated with Google Cloud Platform services such as Vertex AI, GKE, and Dataflow. The position requires the employee to be located in the San Francisco Bay Area for on-site collaboration. Palo Alto Networks describes its general work model as primarily office-based, with flexibility when needed.

Requirements

• At least 9 years of software engineering experience.
• At least 4 years of hands-on experience designing and delivering production-grade AI/LLM features, RAG platforms, and agentic architectures.
• Extensive hands-on experience with Google Cloud Platform services, including Vertex AI, BigQuery, Dataflow, Bigtable, and GKE.
• Deep expertise in agentic architectures, including multi-agent orchestration, human-in-the-loop controls, task planning, state management, and long-term memory systems.
• Experience using agentic frameworks such as LangChain/LangGraph, LlamaIndex, AutoGen, or custom agent engines.
• Proven data engineering background, including building streaming and batch pipelines with Dataflow, Apache Beam, or Spark.
• Experience designing and fine-tuning vector search architectures, such as Vertex AI Vector Search, Pinecone, PGVector, or Milvus.
• Mastery of Python and Java/Scala, with a focus on high-throughput, latency-sensitive, fault-tolerant system design.
• Expert knowledge of distributed event streaming platforms such as Google Pub/Sub and Kafka, and asynchronous event-driven architecture.
• Hands-on experience with MLOps/LLMOps pipelines, continuous evaluation CI/CD, model monitoring, tracing, and deployment patterns.
• Experience with tools such as Jenkins, GitLab CI/CD, Vertex AI Pipelines, LangSmith, Phoenix, or OpenTelemetry.
• Deep understanding of data privacy, prompt injection prevention, access control, encryption, and governance in enterprise cloud AI environments.
• Advanced knowledge of Kubernetes/GKE, service meshes, and microservices architecture.
• Must be located in the San Francisco Bay Area for on-site collaboration.

Responsibilities

• Define and execute technical strategy for enterprise AI, LLM deployment architectures, and multi-agent systems across the engineering organization.
• Design scalable multi-agent systems, autonomous decisioning engines, tool-use frameworks, and complex event-driven agentic workflows.
• Architect end-to-end streaming and batch data and embedding pipelines, hybrid search and RAG architectures, and vector indexing systems for enterprise LLM features.
• Set enterprise standards for AI evaluation, observability, cost-performance optimization, safety guardrails, hallucination reduction, and security compliance.
• Mentor Senior Staff and Senior Engineers, establish code quality standards, and foster continuous AI innovation and research-to-production execution.
• Partner with Product, Security, and Cloud Architecture teams to align technology roadmaps with long-term business goals.
• Collaborate with executive leadership and cross-functional engineering teams to develop reliable products from AI initiatives.

Benefits

• The compensation may include restricted stock units and a bonus.
• A description of employee benefits is available through the company; specific benefits are not listed in the posting.

What This Job Offers

• Job Type: Full-time
• Career Level: Principal
• Education Level: No Education Listed