Backend Engineer | Mimir | USA | Remote

Grafana Labs

11d•$127,651 - $153,180•Remote

About The Position

Grafana Labs is a remote-first, open-source powerhouse. We are looking for candidates in the United States time zones. Grafana Cloud is our composable observability platform that integrates metrics, logs, traces, and profiles with Grafana. The Databases team owns and operates the telemetry databases that are Mimir for metrics, Loki for logs, Tempo for traces, and Pyroscope for profiles. These databases are offered as a hosted service in Grafana Cloud, and additionally as on-premise solutions with Grafana Enterprise Metrics, Grafana Enterprise Logs, and Grafana Enterprise Traces. They are multi-tenant distributed systems implemented in Go and operating at scale on Kubernetes across all major Cloud service providers (AWS, GCP, Azure). The Mimir squad has 3 sub-squads, Ingest, Storage, and Query, which together maintain the Mimir OSS project, and additionally own and operate Grafana Cloud Metrics across 3 major cloud providers. Engineers on the team focus on optimizing the efficiency and resilience of processing, storing, and querying metrics at large volumes. These services operate at a large scale and performance is key to keeping the offering competitive and running smoothly. A Mimir engineer has various work streams. They are likely engaged in a larger project with another engineer, and they are also incorporating some performance and reliability improvements discovered through operating the system in production. They are also responsible for writing and reviewing PRs and design documents from other engineers in the squad, shepherding automated release rollouts, and participating in the on-call rotation for their systems.

Requirements

You are a motivated self starter with a bias towards action.
You are customer focused. We build everything with our users in mind.
You have a passion for building intuitive products that fit customers’ needs.
You have good time management skills, which you leverage to work on the right things at the right time.
Pragmatism: You have a bias towards action, taking direction and building a plan of action to analyze, design, and build modular solutions, deliver MVPs, gather data and feedback and then progress iteratively
Collaboration and communication: The smallest unit we have is a squad. You’ll be working with your teammates in a fully remote setup. Good communication and time management skills are a must
AI: some experience using LLMs for day to day coding tasks and understanding a codebase.
Experience with at least one programming language. We use Go, but if you have familiarity with Python, C, C++, Rust or similar then that translates well
Some experience with delivering projects as a member of a larger team. Your experience includes gathering requirements and brainstorming ideas, all the way to shipping features to the customer’s hands.
Some experience with developing software that runs in the Cloud or some experience with systems engineering
Some experience with being on-call and following the DevOps model
Experience writing clean, robust, and performant software that is easily maintained by others
Some familiarity with observability systems, know when to use metrics, logs, traces, to debug a problem.

Nice To Haves

Experience working with Kubernetes
Experience working with queue systems, e.g. the Kafka protocol
Been a user of Grafana and Prometheus in operational roles (including on-call for your team at a previous employer or just using these tools on hobby/homelab projects)
Exposure to microservices architecture and distributed systems, or a desire to learn
Familiarity with the concept of infrastructure as code

Responsibilities

Take an active role in influencing our roadmap and your own career objectives
Work with your team to deliver new features, then use the results to iterate and improve.
Help your team drive projects from initial idea all the way to operations once it is in the hands of customers
Embrace our open-source culture and contribute to other projects that may not directly fall within your team’s scope
Build, operate, and maintain critical systems, owning the reliability, performance, and availability
Be a part of your team’s follow-the-sun on-call rotations and take ownership of the services you’re running
Support other team members, participate in design discussions and collaborate with the team
Learn new skills by gaining a deeper understanding of our cloud product and our customers and getting to know the codebase of a large distributed system