Senior DevOps Engineer, AI Platform

FloQast•San Jose, CA

3d•$186,000 - $282,000

About The Position

This role exists because FloQast's AI products have outgrown the infrastructure patterns the rest of the platform runs on. Transform, AI Matching, and AutoBuilder are customer-facing products carrying real accounting workloads, and they behave nothing like a CRUD service. They call foundation models in multiple regions, execute generated code in sandboxes, spend money per token rather than per request, and fail in ways a 500-rate dashboard never catches. Today, DevOps engineers carry this work alongside the wider fleet. We are making it someone's whole job. You will embed with the Transform and Close AI pods and own the AI runtime the way our other embedded DevOps engineers own their business unit's platform. This is a DevOps role with an AI infrastructure specialization, not a research or modeling role. You will not train models or tune prompts for accuracy. You will make the systems that serve them fast, observable, multi-region, cost-bounded, and auditable. The products you'll support include Transform, FloQast's data transformation and analytics product, which is a monorepo of containerized services on AWS, including an agentic LLM thread runtime, a natural-language-to-SQL service, and queue-driven workflow workers. You'll own the foundation-model runtime across our US, EU, and AU regions, sandbox isolation for AI-executed code, autoscaling for the worker fleet, and cost attribution for model spend. AI Matching is the automated reconciliation matching stack in our Close product line, featuring LLM-backed match scoring, a matching copilot service family, and a regression harness that guards matching quality. You'll own throughput and unit economics at close-cycle peak, the sandbox for AI-generated code, and keeping the eval harness running in CI so a model or prompt change cannot ship blind. AutoBuilder is the Transform capability that generates transformation workflows and scripts for users instead of making them hand-build each one. You'll own generation-queue health and backpressure, triage that distinguishes a model failure from an infrastructure failure, scale-to-demand behavior for spiky load, and the latency budget for a user waiting on a generated artifact.

Requirements

5+ years in DevOps, SRE, platform, or infrastructure engineering, including production systems you were on call for.
Deep, hands-on AWS: ECS/Fargate, Lambda, SQS, S3, IAM, VPC and networking, ALB/NLB.
Terraform at production scale: modules, state management, multi-region, multi-account.
CI/CD and container ownership: GitHub Actions preferred, Docker, image supply chain, scaling policies.
Production AI infrastructure: you have operated at least one LLM-backed or ML-serving workload in production and can speak concretely about tokens, latency, throttling, and cost. Managed foundation-model APIs (Bedrock, Vertex, Azure OpenAI, Anthropic), serving platforms (SageMaker, KServe, Ray Serve, vLLM, Triton, TrueFoundry), agent or tool-execution runtimes, and eval harnesses in CI all count.
Observability practice: Grafana, Prometheus, or OpenTelemetry, distributed tracing, and the judgment to define SLOs that measure user experience rather than server aliveness.
Working fluency in Python or TypeScript/Node.js, at the level of reading a service, debugging it, and sending a real PR, plus the ability to read the other.

Nice To Haves

Multi-region infrastructure under data-residency constraints (US/EU/AU). The product requires this; prior experience with it moves you to the front of the line.
AI-specific cost and performance work: token accounting, prompt caching, batching, model routing, right-sizing serving capacity.
Sandboxed execution of untrusted or generated code.
Terraform orchestration layers (Atmos or comparable) and monorepo build systems (NX, Turborepo, Bazel).
Progressive delivery with feature flags (Harness or similar).
FinOps tooling (CloudZero or similar) and per-tenant cost attribution.
Data infrastructure exposure: MongoDB, PostgreSQL, Snowflake, EMR/Spark.
Audit evidence you produced for SOC 2 / ISO 27001, or a cloud-cost reduction you delivered and can explain, including what you traded to get it.
Prior work in a regulated or audited SaaS domain: fintech, accounting, healthcare.

Responsibilities

Own our AWS Bedrock and Bedrock AgentCore footprint across US, EU, and AU: model access, provisioned versus on-demand throughput, cross-region inference, quotas and throttles, and region-appropriate model availability. Data residency drives the multi-region design as much as latency does. Define the standard way product teams call a model, retry it, budget it, log it, and trace it, so they stop hand-rolling those primitives.
Operate the execution environments for AI-generated code, with session lifecycle limits, network controls, and least-privilege IAM.
Write and review Terraform across a multi-account, multi-region AWS estate. Every AI resource lands as code. Nothing is hand-made in a console.
Extend our Grafana platform with the signals AI systems need: token consumption, per-model and per-region latency distributions, throttle and retry rates, tool-call failure taxonomy, sandbox session outcomes, generation success rate, and end-to-end agent traces. Define SLOs against critical user journeys, because an AI SLO that only measures HTTP health measures nothing.
Treat AI spend as a first-class cost line: model inference, serving capacity, sandbox compute, and the telemetry volume these systems generate. Tag everything so per-business-unit, and ideally per-tenant, attribution shows up in our FinOps tooling.
Build GitHub Actions pipelines for Node/TypeScript and Python services in NX monorepos. Make model and prompt changes releasable like code: versioned, gated on eval suites, feature-flagged, reversible. A prompt change that silently degrades matching accuracy should be caught by a pipeline, not by a customer.
Join the DevOps on-call rotation, with runbooks you help write for AI-specific failure modes: region-wide model throttling, sandbox exhaustion, silent degradation without errors, poison messages in generation queues. Plan capacity against the accounting calendar; month-end close is the load test.
Hold the AI stack to our SOC 2 and ISO 27001/42001 evidence bar: audit logging, encryption, least-privilege access, patching, and asset inventory, for model endpoints and sandboxes too. Enforce tenant isolation on every AI path, including prompts, retrieved context, generated code, and logs.