Staff Backend Engineer - Mimir Query, Databases | USA | Remote

Grafana Labs

3d•$174,986 - $209,983•Remote

About The Position

Grafana Labs is seeking a Staff Backend Engineer for their Mimir Query, Databases team. This is a remote position for candidates in USA time zones. The role involves working on Grafana Cloud, a composable observability platform that integrates metrics, logs, traces, and profiles. The Databases team specifically manages the telemetry databases (Mimir for metrics, Loki for logs, Tempo for traces, and Pyroscope for profiles) offered as hosted services in Grafana Cloud and as on-premise solutions. These are multi-tenant distributed systems implemented in Go, operating at scale on Kubernetes across major cloud providers. The Mimir squad focuses on optimizing the efficiency and resilience of processing, storing, and querying metrics at large volumes, with performance being key to the offering's competitiveness. Engineers on this team are likely engaged in larger projects, incorporating performance and reliability improvements, writing and reviewing PRs and design documents, shepherding automated release rollouts, and participating in the on-call rotation.

Requirements

Motivated self starter with a bias towards action
Customer focused
Good time management skills
Pragmatism: able to take on complex challenges and break them down to achieve short feedback loops: to analyze, design, and build modular solutions, deliver MVPs, gather data and feedback and then progress iteratively
Collaboration and communication skills
Solid experience with at least one programming language (Go preferred, but Python, C, C++, Rust or similar translates well)
Experience with leading and delivering projects from gathering requirements, brainstorming ideas all the way to shipping a product to the customer’s hands in a self-driven and collaborative way
Experience with developing software that runs in the Cloud or experience with systems engineering
Experience with being on-call and following the DevOps model
Experience writing clean, robust, and performant software that is easily maintained by others
Familiarity with observability systems, know when to use metrics, logs, traces, to debug a problem.
Comfortable with microservices architecture and distributed systems
Experience and comfort developing with LLMs like Claude and Codex on your day to day

Nice To Haves

Experience working with Kubernetes
Experience working with queue systems, e.g. the Kafka protocol
Been a user of Grafana and Prometheus in operational roles (including on-call for your team at a previous employer or just using these tools on hobby/homelab projects)
Familiarity with the concept of infrastructure as code

Responsibilities

Take an active role in influencing our roadmap and your own career objectives
Work with your team to deliver new features, then use the results to iterate and improve.
Drive projects from initial idea all the way to operations once it is in the hands of customers
Embrace our open-source culture and contribute to other projects that may not directly fall within your team’s scope
Design, build, operate, and maintain critical systems, owning the reliability, performance, and availability
Be a part of your team’s follow-the-sun on-call rotations and take ownership of the services you’re running
Support other team members, participate in design discussions and collaborate with the team
Learn new skills by gaining a deeper understanding of our cloud product and our customers and getting to know the codebase of a large distributed system