Senior Applied AI Engineer

Alterion • San Francisco, CA

3d • Onsite

About The Position

Alterion is building the control plane for the agentic enterprise - enabling organizations to safely deploy, monitor, and scale autonomous AI agents with full visibility and governance. Our platform unifies semantic observability, dynamic policy enforcement, and distributed monitoring to ensure agent ecosystems remain trustworthy, compliant, and efficient at scale. As part of this mission, we are building an Intelligence Platform that is responsible for understanding, scoring, and optimizing agent behavior across the enterprise. Including accuracy, security, cost, drift, anomalous behavior, and runtime dynamics. We are looking for a Senior Applied AI Engineer to join our on-site Research & Intelligence team in San Francisco. This role is for an experienced engineer who enjoys working close to research problems but is primarily motivated by building, shipping, and improving production systems. You will work on applied GenAI problems including agent evaluation, observability, and safety; translating proven techniques into scalable, reliable components within Alterion’s platform. This is not a pure research role. Success is measured by what ships and what improves in production, not by publications.

Requirements

• 5+ years of experience in software engineering, ML engineering, or applied AI roles.
• Strong Python skills and experience building production services or data pipelines.
• Hands-on experience working with LLMs (APIs or open-source models) in real systems.
• Familiarity with model evaluation, monitoring, and performance tradeoffs.
• Experience using ML frameworks such as PyTorch or similar.

Nice To Haves

• Experience with GenAI evaluation, observability, or safety tooling.
• Familiarity with embeddings, vector search, or semantic similarity systems.
• Exposure to fine-tuning, PEFT, or cost-aware inference strategies.
• Experience working in early-stage or fast-moving product teams.

Responsibilities

• Build and maintain pipelines for model evaluation, scoring, and monitoring in production environments.
• Apply fine-tuning, optimization, and lightweight model adaptation techniques where appropriate.
• Develop embeddings, classifiers, and scoring services that support natural language analysis.
• Run structured experiments to evaluate model performance, tradeoffs, and regressions.
• Partner with engineering and product teammates to translate ideas into working systems.
• Contribute to internal tooling, dashboards, and prototypes that improve agent visibility and control.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed