Staff AI Engineer - AI Infrastructure & Agentic Platform

ShiftKey

1d•Remote

About The Position

ShiftKey is a healthcare workforce marketplace operating in a regulated (HIPAA) environment, and we are building toward being an AI-native company. We are past the experimentation phase but nowhere near done - and that middle stage is exactly why this role exists. The Staff AI Engineer owns ShiftKey’s AI infrastructure layer: the organizational knowledge platform, the infrastructure for hosting and operating AI agents in the cloud, and the retrieval layer that powers both internal engineering tools and customer-facing capabilities. A retrieval stack is already in production, serving real workflows today - this is not a from-scratch build. The next layer of the platform is in active development. And the most ambitious parts - richer retrieval over organizational knowledge, agentic infrastructure, the platform API surface, and the cost-governance model that keeps running within budget in a regulated environment - are yours to architect and own. You will not be maintaining someone else’s finished platform, and you will not be starting from a blank page. You will inherit serious retrieval foundations and own architecture of everything that comes next. You will have high autonomy and organization-wide influence, partnering directly with the platform team, AI ambassadors across engineering, and senior engineering leadership.

Requirements

7+ years of professional software engineering experience, with at least 2 years building and operating production AI/LLM application systems - not research, not prototyping, not demos.
Retrieval engineering beyond the basics. Our stack already includes re-ranking and HyDE; we need someone who has worked at or above that level: hybrid search, re-ranking, query transformation, context-window management, and evaluation of retrieval quality in production.
Working experience with agentic frameworks and multi-step reasoning loops - tool use, iteration control, cost governance, and model routing trade-offs.
Production-grade software engineering fluency (strict typing, testing, async/concurrency, modern toolchain) in Go, Python, or TypeScript, with the ability to ramp into another quickly.
Hands-on experience operating AI workloads on a managed cloud AI platform (AWS Bedrock or Azure AI Foundry), including the identity/secrets model and model access governance. Bedrock preferred given our AWS stack.
Hands-on Terraform experience - able to author and provision new infrastructure independently, not just modify existing modules.
Familiarity with production observability for AI systems: metrics, structured logging for model spend and latency, and evaluation harnesses to detect regressions.
Understanding of HIPAA-style data-handling requirements in a regulated SaaS environment.

Nice To Haves

Prior healthcare experience is a plus, not a requirement.

Responsibilities

Evolving the AI knowledge platform - taking the retrieval, indexing, and synthesis layer (currently semantic RAG + re-ranking + HyDE) to an organization-wide platform serving both internal engineering tools and customer-facing capabilities.
Architecting and operating agentic infrastructure on AWS - multi-step, tool-using AI systems that plan, retrieve, and act on complex queries and operational events, with cost guardrails and observability built in from day one.
Designing and building graph-based, relationship-aware retrieval across the organization's data sources, enabling multi-hop queries and letting agents accumulate organizational knowledge over time. This is on our roadmap, not in production - you will define the approach.
Partnering with product engineering to define the AI platform API surface, translating infrastructure primitives into developer-ready abstractions.
Building reference agent implementations on the platform - operational-incident triage, customer support, and future agentic use cases - grounding each agent's reasoning in institutional knowledge.
Owning the AI infrastructure cost model: monitoring compute, model, and storage spend, flagging anomalies, and proposing guardrails to keep workloads within defined budgets.