Principal Software Engineer - Observability

Snowflake•Menlo Park, CA
•$264,000 - $379,500

About The Position

Snowflake is seeking a hands-on, product-minded Principal Software Engineer to help build an AI-driven observability platform for customers and engineers. The Observability team develops telemetry, analytics, and diagnostic capabilities that help people understand workload and service health, detect problems early, and resolve them quickly. The role combines distributed-systems engineering with a strong focus on customer and developer needs. The engineer will shape technical direction and lead initiatives from concept through customer launch across engineering, product, and design. The work includes building a platform where AI agents and engineers can collaborate to detect, diagnose, and resolve issues across Snowflake workloads and services.

Requirements

  • 12+ years of software engineering experience, including technical leadership of complex backend, infrastructure, or data-platform initiatives spanning multiple teams.
  • Deep experience designing, building, and operating large-scale distributed systems, including understanding of concurrency, consistency, caching, failure recovery, storage and query efficiency, and performance.
  • Strong computer science fundamentals and hands-on proficiency in Java, Go, C++, or a comparable backend language.
  • Experience with AWS, Azure, or Google Cloud.
  • Active use of AI throughout the software lifecycle, with the ability to demonstrate improvements to execution, problem-solving, and quality.
  • A track record of shipping products to external customers, partnering with product and design, delivering solutions to ambiguous problems, making scope and quality trade-offs, and driving adoption, reliability, and customer satisfaction after launch.
  • Clear written communication, low-ego collaboration, strong ownership, and experience mentoring senior engineers.
  • A degree in Computer Science or a related field, or equivalent practical experience.

Nice To Haves

  • Experience building observability, monitoring, or analytics platforms involving logs, metrics, traces, or alerting.
  • Experience with OpenTelemetry.
  • Experience with time-series or columnar storage.
  • Experience with AIOps or agentic diagnosis systems.
  • Experience building developer-facing data products.

Responsibilities

  • Design and build scalable distributed infrastructure and pipelines for ingesting, storing, and querying high-volume logs, metrics, traces, and events, with attention to cost, query latency, fault isolation, and data freshness.
  • Partner with product management and design to define and deliver customer-facing observability capabilities, including telemetry collection, alerting, dashboards, and diagnostics.
  • Use customer feedback and usage data to inform product decisions and own outcomes through launch and adoption.
  • Develop AI-driven workflows that correlate signals, surface anomalies, explain likely root causes, and recommend or perform remediation.
  • Ground AI capabilities in high-quality telemetry and rigorous evaluation to support customer and engineer trust.
  • Use AI throughout software design, implementation, testing, and operations; establish evaluation, security, and reliability standards for AI-powered capabilities and encourage AI-first practices across the team.
  • Translate ambiguous customer and developer needs into practical designs and milestones.
  • Deliver intuitive APIs, SDKs, and self-service experiences based on open standards such as OpenTelemetry, and guide incremental migrations from existing tooling.
  • Write and review production code, diagnose complex failures, and own operational outcomes.
  • Mentor engineers and influence teams through technical depth, clear communication, and sound judgment.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service