Principal Engineer Software, Dev Infrastructure (Chronosphere)

Palo Alto Networks•San Francisco, CA

17h•Remote

About The Position

The Team Chronosphere, a Palo Alto Networks company, is the observability platform built for control in the modern, containerized world. Chronosphere empowers customers to focus on the data and insights that matter by reducing complexity, optimizing costs, and remediating issues faster. Chronosphere reduces data volumes and associated costs by 84% on average while saving developers thousands of hours. Recognized as a leader by major analyst firms, Chronosphere is trusted by the world's most innovative brands, including DoorDash, Affirm, and Zillow. We're looking for an Infrastructure Engineer to build developer tooling that enables developer velocity and reliability for the entire engineering organization. Our team owns the full end-to-end development lifecycle, from local development tools, CI/CD, dynamic testing environments, deployments, and releases. You’ll be working with the latest cloud-native technologies as you build an infrastructure platform that enables the engineering organization to deliver observability for some of the largest technology companies.

Requirements

8+ years of relevant experience.
Strong experience in at least one backend language (e.g., Go, Java, Python, or Rust).
Strong proficiency in at least one backend language (e.g., Go, Java, Python, or Rust).
Solid understanding of cloud-native concepts and experience working with cloud providers like AWS or GCP.
Comfortable navigating Kubernetes and container-level logic.
Deep knowledge of Linux internals, process management, and resource isolation.
Understanding of the OSI model, service meshes, load balancing, and "zero-trust" security architectures.
Experience building and debugging systems that deal with CAP theorem trade-offs, eventual consistency, and distributed tracing.
A track record of completing assigned tasks/tickets reliably and estimating work effectively within a sprint.
The ability to debug your own code efficiently using logs and tests, while proactively identifying edge cases (like nulls or limits) during the design phase.
Strong communication skills to keep teammates informed, raise blockers early, and contribute meaningfully to code reviews and design discussions.
A proactive approach to learning new tools, processes, and libraries.
Experience or interest in using AI coding assistants (like Cursor or Claude) to improve productivity and automate boilerplate tasks.

Responsibilities

Design and maintain high-scale developer tooling and backend services that improve productivity and reliability across a distributed cloud environment.
Define, deploy, and manage entire environments using declarative IaC (Terraform), ensuring our platform is reproducible and version-controlled.
Identify and eliminate systemic bottlenecks in the software development lifecycle (SDLC) through architectural changes or advanced tooling.
Ensure our infrastructure remains resilient under massive traffic loads while optimizing for performance, cost-efficiency, and near-real-time telemetry processing.
Define platform standards and reference architectures that span a 1–3 year horizon, balancing feature velocity with long-term technical debt.
Act as the "glue" across teams, consulting on infrastructure best practices and up-leveling the organization through mentorship.