Staff Engineer, Platform (R6019)

Shield AI•San Mateo, CA

17d•Onsite

About The Position

We are looking for a Staff Platform Engineer to contribute to the Platform solutions and infrastructure that power Forge, the AI Factory. These components provide distributed runtime capabilities that enable teams to reliably orchestrate workloads, process data, and underpin the workflows of building an AI Pilot. The Forge Platform Engineering team provides Kubernetes-native capabilities that support autonomy development, simulation, testing, training, evaluation, deployment, and operational workflows across commercial cloud, on-premises infrastructure, sovereign deployments, edge environments, and fully air-gapped systems. This is a hands-on technical leadership role. You will define platform architecture, implement production software, establish reusable operational patterns, and partner with teams across autonomy, ML, simulation, test, infrastructure, and product engineering. Success requires balancing developer productivity, reliability, extensibility, performance, portability, and long-term operational maintainability.

Requirements

Significant experience designing and operating production distributed systems, cloud-native platforms, backend infrastructure, or data-intensive services.
Strong software engineering skills and a record of delivering production systems in Go and Python.
Deep understanding of distributed-systems fundamentals, including failure handling, idempotency, consistency tradeoffs, retries, ordering, delivery semantics, backpressure, partitioning, state management, and fault tolerance.
Experience designing or operating workflow orchestration, distributed job execution, asynchronous processing, event-driven systems, or long-running service workflows.
Ability to define architecture and technical standards while remaining hands-on in implementation, production troubleshooting, performance analysis, and reliability improvement.
Experience working across multiple teams to turn recurring infrastructure needs into reusable, well-documented platform capabilities.
Clear technical communication and the ability to make complex distributed-systems architecture understandable to both specialists and downstream users.

Nice To Haves

Kubernetes controllers, operators, Custom Resource Definitions, admission control, scheduling extensions, KubeRay, or workload-management systems.
Distributed execution and workflow technologies such as Ray, Temporal, Argo Workflows, Flyte, Dagster, Airflow, Prefect, Kubernetes Jobs, or comparable systems.
Durable messaging and event-streaming technologies such as NATS JetStream, Kafka, Redpanda, Pulsar, RabbitMQ, SQS/SNS, or comparable systems.
ETL/ELT, batch processing, event-driven data pipelines, CDC, schema evolution, data validation, artifact processing, or large-file transfer workflows.
Service networking technologies and practices such as Envoy, service meshes, Kubernetes networking, and CNI plugins.
Observability software such as OpenTelemetry, Prometheus, Grafana, Loki, Tempo, Jaeger, distributed tracing, structured logging, SLOs, service-level indicators, alerting, and incident-management practices.
Terraform, Helm, ArgoCD, GitOps, Kubernetes package management, repeatable platform distribution, and Infrastructure as Code.

Responsibilities

Build Kubernetes-native platform services: Develop and operate Kubernetes-based services, controllers, operators, deployment patterns, and runtime integrations that support distributed workloads across multiple environments.
Develop distributed orchestration capabilities: Design and build reusable primitives for authoring, scheduling, and scaling pipeline work.
Build reliable data-processing infrastructure: Develop platform capabilities for data storage, ingestion, validation, transformation, and governance.
Develop highly extensible platform components: The Forge Platform base provides standardized tooling around authentication, authorization, observation, networking, routing, secret management, and more to the services that are integrated on top of the ecosystem.
Create reference architectures: Establish recommended deployment patterns, operating profiles, capacity guidance, benchmarks, reliability practices, and distribution approaches across cloud providers, on-prem, edge, and air-gapped environments.
Advance observability and operability: Establish end-to-end metrics, logs, traces, structured events, dashboards, alerting, service-level objectives, operational diagnostics, and runbooks for workflows, pipelines, event streams, and platform services.
Partner with downstream teams: Work directly with autonomy, ML Ops, simulation, test, infrastructure, product, and customer-facing teams to turn recurring distributed-systems problems into reusable platform capabilities.