Senior Distributed Systems Engineer

Cloudflare • San Francisco, CA

1d • Onsite

About The Position

We are looking for a talented Distributed Systems Engineer to join the Data Localization team. The team builds the infrastructure that enforces where customer data is stored, processed, and decrypted across one of the largest globally distributed edge networks in the world. The problem is not simply building fast, resilient distributed systems; it is building them with provable geographic boundaries that hold under failure, at the scale and reliability Cloudflare's customers depend on. You will work across the full stack in Go and Rust, from low-level policy enforcement and cryptographic key routing at the edge to customer-facing APIs and dashboards, on top of Cloudflare's existing infrastructure: the edge fleet, globally distributed key-value storage, Workers and Durable Objects, PostgreSQL, Kubernetes, and regional ClickHouse. Features ship end-to-end: you will own the design, implementation, rollout, and production operation of the systems you build. This is a good fit if you are drawn to problems where compliance correctness is a hard constraint and not just a quality goal, and where the failure modes you reason about have real consequences for customers operating under regulatory scrutiny.

Requirements

• 5+ years of professional experience designing, building, and operating production distributed systems at scale.
• Strong proficiency in at least one system or backend language such as Go, Rust, or C/C++, and a willingness to work in others as the codebase demands.
• Solid grasp of distributed systems fundamentals, including: Consistency and consensus models (strong, sequential, eventual; Paxos / Raft at a conceptual level).
• Replication, sharding, and partitioning strategies, and their tradeoffs against availability and latency.
• Failure modes: partial failure, network partitions, split brain, clock skew, and how these show up in real systems.
• Idempotency, retries, backpressure, timeouts, circuit breakers, and rate limiting as first-class design concerns.
• Health checking, failure detection, leader election, and graceful degradation.
• Observability: metrics, logs, and tracing as design inputs, not afterthoughts.
• Practical experience with API design (REST or gRPC), relational databases, and asynchronous messaging or event streaming systems, with a clear understanding of transactional and consistency boundaries.
• Comfortable with AI-assisted development tooling, with the judgment to use it to accelerate work while remaining accountable for correctness, security, and design quality
• Track record of production ownership: on-call, incident response, post-mortems, and continuous investment in reliability and performance.
• Strong written and verbal communication skills; ability to write clear design documents and collaborate effectively across time zones.

Nice To Haves

• Experience building compliance-driven, security-sensitive, or multi-region systems.
• Familiarity with cryptography basics — envelope encryption, key management, HSMs, or PKI.
• Exposure to edge, CDN, or L4/L7 proxy platforms, or to large-scale globally distributed storage or key-value systems.
• Experience contributing to or driving multi-team, multi-quarter engineering programs with cross-functional dependencies across platform, security, and product teams.

Responsibilities

• Design, build, and operate backend services that enforce regional boundaries on where customer data is stored, processed, and decrypted, across a globally distributed edge network.
• Contribute to end-to-end feature delivery — write technical designs and RFCs, implement services and libraries, add tests and observability, drive rollout, and own the resulting systems in production.
• Partner with engineers on adjacent platforms, cryptography, networking, data, and product teams to integrate residency guarantees into new and existing surfaces without regressing performance or reliability for other customers.
• Reason carefully about failure modes, consistency, and blast radius; design systems that fail closed for compliance while degrading gracefully for availability, and make these tradeoffs explicit in design docs.
• Participate in on-call for your services, lead incident response and post-mortems, and continually invest in reliability, capacity planning, and cost.
• Raise the engineering bar through code review, mentorship, and clear written communication; contribute to the team's engineering standards and tooling.

Benefits

• Cloudflare was named to Entrepreneur Magazine’s Top Company Cultures list and ranked among the World’s Most Innovative Companies by Fast Company.
• Project Galileo : Since 2014, we've equipped more than 2,400 journalism and civil society organizations in 111 countries with powerful tools to defend themselves against attacks that would otherwise censor their work, technology already used by Cloudflare’s enterprise customers--at no cost.
• Athenian Project : In 2017, we created the Athenian Project to ensure that state and local governments have the highest level of protection and reliability for free, so that their constituents have access to election information and voter registration. Since the project, we've provided services to more than 425 local government election websites in 33 states.
• 1.1.1.1 : We released 1.1.1.1 to help fix the foundation of the Internet by building a faster, more secure and privacy-centric public DNS resolver. This is available publicly for everyone to use - it is the first consumer-focused service Cloudflare has ever released. Here’s the deal - we don’t store client IP addresses never, ever. We will continue to abide by our privacy commitment and ensure that no user data is sold to advertisers or used to target consumers.
• Cloudflare is proud to be an equal opportunity employer. We are committed to providing equal employment opportunity for all people and place great value in both diversity and inclusiveness. All qualified applicants will be considered for employment without regard to their, or any other person's, perceived or actual race, color, religion, sex, gender, gender identity, gender expression, sexual orientation, national origin, ancestry, citizenship, age, physical or mental disability, medical condition, family care status, or any other basis protected by law. We are an AA/Veterans/Disabled Employer.
• Cloudflare provides reasonable accommodations to qualified individuals with disabilities. Please tell us if you require a reasonable accommodation to apply for a job. Examples of reasonable accommodations include, but are not limited to, changing the application process, providing documents in an alternate format, using a sign language interpreter, or using specialized equipment. If you require a reasonable accommodation to apply for a job, please contact us via e-mail at [email protected] or via mail at 101 Townsend St. San Francisco, CA 94107.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed