Staff Backend Engineer - Grafana Enterprise | US | Remote

Grafana Labs

3h•$174,986 - $209,983•Remote

About The Position

Grafana Labs is a remote-first, open-source powerhouse with over 20 million users of its visualization tool, Grafana. The company supports over 3,000 companies in managing their observability strategies with the Grafana LGTM Stack, which can be run fully managed with Grafana Cloud or self-managed with the Grafana Enterprise Stack. Grafana Labs is experiencing rapid growth while maintaining its core values of an open-source legacy, a global collaborative culture, and a passion for meaningful work. The team operates in an innovation-driven environment characterized by transparency, autonomy, and trust. This is a remote position open to candidates in the US and Canada. The Grafana Enterprise team focuses on developing innovative solutions for large-scale customer challenges, enhancing security, robustness, flexibility, multitenancy, and interoperability. They collaborate across the organization and with external customers, including major cloud service providers and global enterprises, to address demanding distributed systems challenges. The company is remote-only and global, valuing diverse experiences and backgrounds. Meetings typically occur between 14:00 and 17:00 UTC, with flexibility to accommodate engineer and customer needs. The company measures itself by openness, helpfulness, and shared successes. The role requires experienced software engineers passionate about distributed systems and building reliable, scalable backend infrastructure. The backend is primarily Go, and engineers contribute to open-source communities. The company invests in developer productivity, offering AI coding assistants with a funded usage budget and access to frontier models. Development is encouraged to be pragmatic and AI-assisted, always paired with strong code review and quality standards.

Requirements

Deep professional experience writing production services, from ideation through to production operations at scale
Strong distributed systems fundamentals: replication, consistency models, partitioning, fault tolerance, and the trade-offs that come with operating at scale
Demonstrated experience designing and operating systems for large-scale, high-traffic, high-availability, or multi-tenant environments, ideally in the context of infrastructure, observability, or software delivery platforms
Professional experience building and consuming gRPC/protobuf APIs and designing clean service contracts across service boundaries
Strong database skills, such as PostgreSQL and/or MySQL; including schema design, query optimisation, and schema migrations at scale
Experience with large-scale CI/CD systems and build tooling, designing, operating, or integrating with continuous delivery pipelines that serve large engineering organisations or external operators at scale
Comfort working with Kubernetes and containerised deployment environments, including patterns for operating stateful workloads and multi-tenant clusters
Experience with observability tooling: OpenTelemetry, Prometheus metrics, structured logging, and distributed tracing
Familiarity with dependency injection patterns (e.g., Google Wire) and clean, testable service architecture
You work well as a communicative member of a team of engineering professionals.
You earn trust by saying what you mean and doing what you say.
You are customer focused and especially attuned to the needs of large-scale operators who rely on Grafana as critical infrastructure. You start with their needs and work backwards.
You insist on the highest standards and work to develop the skills and knowledge of your fellow team members.
You take on complex distributed systems challenges, break them down into digestible problems, and leverage your team and organization to deliver.
You design modular solutions, deliver minimum loveable products, gather data and feedback, and then progress iteratively.

Nice To Haves

Experience with TypeScript and React for contributing to frontend features and collaborating closely with frontend engineers
Experience with Grafana's LGTM+ observability stack (Loki, Mimir, Tempo, Pyroscope, Alloy)
Prior experience at or building for large-scale cloud service providers, IaaS providers, or global enterprises with demanding SLA requirements
Experience designing or operating large-scale build infrastructure artifact registries, distributed build caches, hermetic build systems (e.g., Bazel), or developer platform tooling

Responsibilities

Earning the trust of our large-scale operator customers to further Grafana's "big tent" philosophy of data accessibility and to meet clear business objectives
Designing and leading the development of backend services, distributed systems, and enterprise features at scale
Driving continuous improvement of our engineering culture through words and actions
Driving projects from initial ideation through the development lifecycle to production
Contributing to the scalability, reliability, security, and multi-tenancy of the Grafana platform trusted by some of the world's largest operators
Owning the operational health of our platform by participating in weekday 12h x 5d and separate weekend 24h x 2d on-call rotations.
Hiring and developing the best engineers to build the future of Grafana
Developing your skills as a thought leader to drive continuous improvement of engineering and operational practices across Grafana Labs
Design and build the backend systems powering Grafana Enterprise — the platform trusted by the world's largest operators to run their observability and software delivery infrastructure.
Hiring and developing the best engineers we can to deliver the future of Observability.
Architecting and implementing distributed backend services in Go, with a focus on correctness, observability, and performance at scale
Designing APIs and service contracts used by thousands of enterprise operators and cloud service providers
Collaborating with Product and UX to shape features and partnering with frontend engineers to ship complete, end-to-end solutions
Driving scalability and reliability improvements that matter to large-scale operators running Grafana in regulated, high-availability environments
Engaging directly with large enterprise customers and cloud service providers to understand their requirements and translate them into robust engineering solutions
Advocating for our customers at every stage of the development lifecycle