Distributed Systems Engineer

Cloudflare • San Francisco, CA

3d • Onsite

About The Position

We are looking for a talented Distributed Systems Engineer to join the Data Localization team. The team builds the infrastructure that enforces where customer data is stored, processed, and decrypted across one of the largest globally distributed edge networks in the world. The problem is not simply building fast, resilient distributed systems; it is building them with provable geographic boundaries that hold under failure, at the scale and reliability Cloudflare's customers depend on. You will work across the full stack in Go and Rust, from low-level policy enforcement and cryptographic key routing at the edge to customer-facing APIs and dashboards, on top of Cloudflare's existing infrastructure: the edge fleet, globally distributed key-value storage, Workers and Durable Objects, PostgreSQL, Kubernetes, and regional ClickHouse. Features ship end-to-end: you will own the design, implementation, rollout, and production operation of the systems you build. This is a good fit if you are drawn to problems where compliance correctness is a hard constraint and not just a quality goal, and where the failure modes you reason about have real consequences for customers operating under regulatory scrutiny.

Requirements

• 3+ years of professional experience designing, building, and operating production distributed systems at scale.
• Strong proficiency in at least one system or backend language such as Go, Rust, or C/C++, and a willingness to work in others as the codebase demands.
• Solid grasp of distributed systems fundamentals, including: Consistency and consensus models (strong, sequential, eventual; Paxos / Raft at a conceptual level).
• Replication, sharding, and partitioning strategies, and their tradeoffs against availability and latency.
• Failure modes: partial failure, network partitions, split brain, clock skew, and how these show up in real systems.
• Idempotency, retries, backpressure, timeouts, circuit breakers, and rate limiting as first-class design concerns.
• Health checking, failure detection, leader election, and graceful degradation.
• Observability: metrics, logs, and tracing as design inputs, not afterthoughts.
• Practical experience with API design (REST or gRPC), relational databases, and asynchronous messaging or event streaming systems, with a clear understanding of transactional and consistency boundaries.
• Comfortable with AI-assisted development tooling, with the judgment to use it to accelerate work while remaining accountable for correctness, security, and design quality
• Track record of production ownership: on-call, incident response, post-mortems, and continuous investment in reliability and performance.
• Strong written and verbal communication skills; ability to write clear design documents and collaborate effectively across time zones.

Nice To Haves

• Experience building compliance-driven, security-sensitive, or multi-region systems.
• Familiarity with cryptography basics — envelope encryption, key management, HSMs, or PKI.
• Exposure to edge, CDN, or L4/L7 proxy platforms, or to large-scale globally distributed storage or key-value systems.
• Experience contributing to or driving multi-team, multi-quarter engineering programs with cross-functional dependencies across platform, security, and product teams.

Responsibilities

• 3+ years of professional experience designing, building, and operating production distributed systems at scale.
• Strong proficiency in at least one system or backend language such as Go, Rust, or C/C++, and a willingness to work in others as the codebase demands.
• Solid grasp of distributed systems fundamentals, including: Consistency and consensus models (strong, sequential, eventual; Paxos / Raft at a conceptual level).
• Replication, sharding, and partitioning strategies, and their tradeoffs against availability and latency.
• Failure modes: partial failure, network partitions, split brain, clock skew, and how these show up in real systems.
• Idempotency, retries, backpressure, timeouts, circuit breakers, and rate limiting as first-class design concerns.
• Health checking, failure detection, leader election, and graceful degradation.
• Observability: metrics, logs, and tracing as design inputs, not afterthoughts.
• Practical experience with API design (REST or gRPC), relational databases, and asynchronous messaging or event streaming systems, with a clear understanding of transactional and consistency boundaries.
• Comfortable with AI-assisted development tooling, with the judgment to use it to accelerate work while remaining accountable for correctness, security, and design quality
• Track record of production ownership: on-call, incident response, post-mortems, and continuous investment in reliability and performance.
• Strong written and verbal communication skills; ability to write clear design documents and collaborate effectively across time zones.

Benefits

• We’re not just a highly ambitious, large-scale technology company. We’re a highly ambitious, large-scale technology company with a soul.
• Fundamental to our mission to help build a better Internet is protecting the free and open Internet.
• Project Galileo : Since 2014, we've equipped more than 2,400 journalism and civil society organizations in 111 countries with powerful tools to defend themselves against attacks that would otherwise censor their work, technology already used by Cloudflare’s enterprise customers--at no cost.
• Athenian Project : In 2017, we created the Athenian Project to ensure that state and local governments have the highest level of protection and reliability for free, so that their constituents have access to election information and voter registration. Since the project, we've provided services to more than 425 local government election websites in 33 states.
• 1.1.1.1 : We released 1.1.1.1 to help fix the foundation of the Internet by building a faster, more secure and privacy-centric public DNS resolver. This is available publicly for everyone to use - it is the first consumer-focused service Cloudflare has ever released.
• We don’t store client IP addresses never, ever.
• We will continue to abide by our privacy commitment and ensure that no user data is sold to advertisers or used to target consumers.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed