Software Engineer, Data Intake

Observe, Inc.
2dHybrid

About The Position

Observe is an AI-powered observability platform engineered for scale. Built on an open, scalable data lake, Observe ingests and stores telemetry at dramatically lower cost while allowing reuse through open data formats like Apache Iceberg. Its Knowledge Graph tracks relationships between objects in the system—and how they change over time—providing essential context for investigations. Pairing agentic workflows with context from the Knowledge Graph, Observe’s chat-based AI SRE accelerates root cause analysis and resolution. Engineering teams at Capital One, Topgolf, and Dialpad ingest hundreds of terabytes daily and troubleshoot 3x faster at 60% lower cost, depending on Observe to maintain reliability at scale. Team: We’re hiring a Software Engineer for our Data Intake team, which builds the agent, ingestion infrastructure, and in-product onboarding experience that connects customer systems to Observe. We’re looking for engineers who enjoy deep systems work and care about how that work is experienced by users. This is a hybrid role combining: Agent and data pipeline development (OpenTelemetry-based, Go). Infrastructure and configuration (Helm, Kubernetes, cloud templates). Backend APIs and data models that power user-facing onboarding and troubleshooting experiences. The team’s goal is to make data ingestion self-serve, transparent, and diagnosable, so customers can understand exactly where their telemetry is flowing or failing.

Requirements

  • Strong software engineering fundamentals, with experience in backend or systems development
  • Proficiency in Go or another statically typed language, with willingness to work deeply in Go
  • Experience with distributed systems, data pipelines, or agent-based software
  • Familiarity with Kubernetes and cloud environments (AWS, GCP, or similar), or strong interest in developing this expertise
  • Comfort reading and reasoning about open-source code and external systems
  • Enjoy debugging distributed systems and data pipelines, especially ones you didn’t originally write
  • Are comfortable moving between configuration, infrastructure, and code to understand real-world system behavior
  • Care about making complex systems observable, diagnosable, and explainable
  • Like turning messy production reality into clear signals and APIs that others can rely on

Nice To Haves

  • Experience with OpenTelemetry, observability agents, or telemetry pipelines
  • Experience building or operating control planes for distributed systems
  • Prior work on onboarding, configuration management, or user-facing diagnostics for infrastructure products
  • Background in platform, infrastructure, or SRE-adjacent teams

Responsibilities

  • Work on the Observe Agent (OTel collector distribution), including custom receivers, processors, and exporters.
  • Build Config- and infra-heavy systems (OTel config, Helm charts, Kubernetes, AWS templates).
  • Control-plane APIs and data models that represent ingestion state, errors, and events.
  • Debug complex ingestion issues across agents, pipelines, and backend systems
  • Scale systems that translate low-level distributed behavior into clear user-facing signals
© 2024 Teal Labs, Inc
Privacy PolicyTerms of Service