Senior Software Engineer, Data Infrastructure Compute

Roblox•San Mateo, CA

3d•$243,290 - $295,250•Onsite

About The Position

Roblox’s Data Platform Compute team owns the Kubernetes (EKS) clusters that run the company’s Spark workloads and the software that enables engineers, analysts, and ML practitioners to submit and run those workloads reliably and cost-effectively. The Senior Software Engineer will build and operate the systems between users and the cluster, including the Spark job submission service, internal Spark libraries, interactive notebook compute, and cluster lifecycle automation. The role offers ownership of technically unsettled problems and a part in deciding which technologies Roblox adopts. The engineer will build core compute platform services and APIs; deliver Spark Connect for notebook workflows and manage the EKS compute behind them; automate safe cluster fleet management; develop Spark job tuning tools; operate Kubernetes batch scheduling; evaluate emerging technologies with benchmarks on Roblox workloads; and improve reliability, observability, and cost. The role also partners with data engineering, ML, analytics, and infrastructure teams to understand requirements, communicate changes, and support migrations. For roles based at Roblox’s headquarters in San Mateo, California, the stated annual base salary range is $243,290–$295,250. The position is onsite Tuesday through Thursday, with optional office presence Monday and Friday, unless otherwise noted. Full-time employees are eligible for equity compensation and benefits.

Requirements

At least 5 years of experience building and operating production-grade distributed systems or infrastructure platforms.
Hands-on experience operating Apache Spark on Kubernetes as a platform, including executor and driver behavior, resource tuning, and troubleshooting failed or slow jobs.
Strong experience operating production Kubernetes clusters at scale, including upgrades, node lifecycle, autoscaling, and operators or controllers.
Strong proficiency in at least one of Go, Java, Scala, or Clojure, with a record of writing clean, well-tested, maintainable backend code.
Experience with AWS services such as EKS, EC2, IAM, and S3, or GCP equivalents, and infrastructure-as-code.
Experience owning a system from design through production operation, including on-call, incident response, and observability.
Ability to influence design decisions across teams, mentor engineers, and drive alignment on cross-cutting changes.
B.S. in Computer Science or equivalent practical experience.

Nice To Haves

Python proficiency; much of the tuning and analysis tooling uses Python.
Experience with YuniKorn, Volcano, Kueue, or other Kubernetes batch schedulers.
Experience with Spark Connect, Livy, Kyuubi, or other interactive and gateway layers for Spark.
Experience with the Spark Operator, Karpenter, Helm, Terraform, or Argo.
Experience building tools that use Spark event logs or metrics for tuning, cost attribution, or anomaly detection.
Experience maintaining internal Spark builds or driving Spark version upgrades.
Familiarity with native execution engines such as Comet, Gluten, or Velox, or JVM performance tuning.
Experience migrating workloads from YARN/Hadoop to Kubernetes.
Contributions to open-source projects in the data or Kubernetes ecosystem.
Experience operating at consumer-internet scale (100M+ users).

Responsibilities

Design, build, and operate core compute platform services behind Spark job submission, including APIs, scheduling integration, and user-facing libraries.
Deliver Spark Connect for notebook workflows, including provisioning and managing supporting EKS compute with isolation, resource limits, and cost controls.
Build EKS fleet management automation, including blue-green rollouts that validate AMIs and Kubernetes versions before production deployment while minimizing disruption to running jobs.
Develop an auto-tuner that analyzes Spark history and event logs to recommend or apply configuration changes that improve performance and reduce cost.
Operate and upgrade YuniKorn-based scheduling on Kubernetes, including queue design, quotas, tenant fairness, and version upgrades.
Evaluate emerging technologies such as native Spark acceleration, benchmark them on Roblox workloads, and make data-backed recommendations.
Define SLOs and metrics, build observability, and reduce toil, incidents, and compute spending across a multi-tenant platform.
Partner with data engineering, ML, analytics, and infrastructure teams to understand needs, communicate changes, and migrate users to improved workflows.