Staff Backend Software Engineer - Activations

LiveRamp•San Francisco, CA

About The Position

LiveRamp’s Activations Backend team builds and operates the large-scale data processing platform behind its primary activation product. The platform processes more than 100,000 batch jobs per day, from gigabytes to over 100 terabytes, and distributes data to hundreds of destinations. It processes multiple exabytes annually and provides monitoring, statistics, error recovery, and resiliency for major customers. The role will lead the design and evolution of a petabyte-scale activation platform toward a cloud-forward, multi-regional, delta-first, cache-aware, and cost-efficient architecture. This includes shaping technical strategy across matching and delta computation, job orchestration, and delivery pipelines; building and improving large-scale data pipelines; and delivering reliable, high-throughput, low-latency activation. The work may combine SingleStore-backed state and delta computation, Spark/Dataproc for large batch workloads, and streaming systems for event-driven and incremental deliveries. The engineer will design smarter workflow orchestration and scheduling, including job classification, multi-queue scheduling, capacity pools, destination-specific rate limits and SLAs, and methods to avoid duplicated computation. The role also entails production ownership, performance and cost optimization, incident response, postmortems, and ongoing improvements to reliability and observability. The engineer will collaborate across partner teams, mentor other engineers, represent Activations Backend in architecture discussions, and evaluate AI-enhanced tools for software development and operations.

Requirements

At least 5 years of experience writing and deploying high-quality production code in a modern language such as Java, Go, or Scala, including ownership of complex production systems.
Experience leading the design and delivery of large-scale distributed or big data systems with clear business impact.
Strong data engineering and SQL skills, including data modeling, complex queries on very large tables, and reasoning about performance, correctness, and cost.
Deep experience owning end-to-end data pipelines, including ingestion, transformation, orchestration, failure handling, and observability.
Comfort working in a cloud environment, ideally GCP, and with containerized workloads such as Kubernetes/GKE or similar.
Ability to define and drive technical strategy, decompose multi-quarter problems, evaluate tradeoffs, align stakeholders, and deliver incremental value.
Excellent communication and collaboration skills, including influencing across teams and building consensus on complex technical decisions.
Demonstrated ability to mentor and grow engineers, give effective feedback, and create opportunities for others to contribute.
Comfort with ambiguity, curiosity, and the ability to turn questions into concrete experiments and system changes.
AI-enabled development experience or strong interest in learning and using AI-enhanced development tools and helping others use them effectively.

Nice To Haves

Experience with Google Cloud Platform services such as GCS, Dataproc, GKE, Pub/Sub, BigQuery, and IAM.
Experience with Temporal or Cadence, or Airflow or similar systems for long-running, failure-prone workflows.
Experience with Apache Spark or Dataproc for large-scale batch processing.
Experience with data warehouses such as SingleStore, BigQuery, Snowflake, or similar.
Experience with Kafka, Redpanda, Pub/Sub, or equivalent streaming platforms, especially for high-volume or incremental processing.
Experience designing multi-tenant systems that enforce rate limits, fairness, and SLAs across customers and destinations.
A strong background in performance and cost optimization for large-scale data workloads.
Experience in advertising, marketing, data activation, or other systems where data correctness, timeliness, and scale are critical.

Responsibilities

Lead the design and evolution of a petabyte-scale activation platform toward a delta-first, cache-aware, cost-efficient, cloud-forward, and multi-regional architecture.
Set technical strategy for major areas of Activations Backend, including matching and delta computation, job orchestration, and delivery pipelines, from design through rollout and long-term maintenance.
Architect and build big data pipelines using technologies such as Apache Spark/Dataproc, SingleStore, Kubernetes/GKE, and streaming systems.
Use workflow engines such as Temporal and Cadence to orchestrate complex, long-running workflows with robust retry, compensation, and observability, and define reusable engineering patterns.
Design job classification, queueing, capacity pools, and rate-limit-aware scheduling to balance latency-sensitive work with large backfills and bulk workflows.
Improve performance and cost efficiency for large-scale workloads by reducing duplicate processing, improving cache hit rates, tuning cluster sizing and autoscaling, and setting and tracking SLOs.
Own critical production services, coordinate incident response and postmortems, and drive structural fixes that reduce operational load and risk.
Evaluate and adopt AI-enhanced tools for coding, design exploration, data analysis, and operational debugging, and help establish best practices.
Collaborate with partner teams to improve activation reliability, speed, and observability.
Mentor and develop engineers through design and code reviews, pairing, and technical guidance.
Represent Activations Backend in cross-team architecture forums and external venues.