Senior Software Engineer, AI

ImmunityBio•El Segundo, CA

1d•Hybrid

About The Position

The Senior Software Engineer – AI is a hands-on, systems-level role integrating data engineering, software development, and AI agent architecture, with close collaboration across platform engineering, data science, security, and product teams. The Senior Software Engineer – AI will serve as a subject matter expert (SME) in AI agent development and will work with cross-functional teams for successful implementation of production-grade AI systems. This role supports Engineering leadership in driving appropriate change control, maintaining project schedules and quality standards, designing and programming AI agent systems, and providing technical support for daily AI-powered operations — including the specification, deployment, and continuous improvement of agent frameworks and supporting infrastructure.

Requirements

Bachelor's degree in Computer Science, Software Engineering, or a related field with 7+ years of relevant software development experience is required.
3+ years of hands-on Python development experience, with strong proficiency in Python 3.12+ and modern language features (type hints, async/await, etc.) is required.
3+ years of hands-on Typescript development experience is required.
Experience with at least one AI agent framework (LangGraph, LangChain, CrewAI, or LlamaIndex) in a production or near-production setting is required.
Experience building and running containerized applications with Docker, including multi-stage builds and foundational DevOps practices, is required.
Excellent interpersonal skills and ability to work effectively in a cross-functional team environment.
Excellent technical writing, communication, and organizational skills, including the ability to produce concise READMEs, architecture documents, and runbooks.
Strong knowledge of engineering principles required to support secure, production-grade AI services and infrastructure.
Deep technical knowledge and hands-on experience with AI agent frameworks, orchestration patterns, and context-management strategies.
Strong leadership skills, with the ability to mentor junior engineers and drive technical standards.
Strong knowledge and proficiency in data analytics, monitoring, and observability tooling (metrics, logging, tracing, dashboards) for production AI services.
Solid understanding of API design principles, error handling patterns, and security best practices for backend services.
Familiarity with AI safety and guardrail patterns, including prompt hardening and content filtering methodologies.

Nice To Haves

Experience with complex troubleshooting and system optimization in a regulated or enterprise environment is preferred.
Experience working within an organization with formal change control, compliance, or audit requirements is preferred.

Responsibilities

Subject Matter Expert for the design, development, maintenance, and deployment of AI agent systems, including one or more of LangGraph StateGraph patterns, CrewAI multi-agent orchestration, LlamaIndex data/agent workflows, or related frameworks.
Design, build, and maintain FastAPI-based agent servers, including async/await endpoints, streaming responses, and health check endpoints.
Implement comprehensive testing strategies — unit, integration, edge-case, and performance — for agents and services to ensure production reliability.
Build and enforce guardrails for AI interactions, including input/output validation, safety checks, prompt hardening, policy enforcement, and robust errors handling in alignment with organizational security and compliance standards.
Design secure tool execution patterns adhering to least-privilege access, allowlists, secrets management, audit logging, and validation controls against prompt injection/data exfiltration.
Implement agent observability to include per-step traces, tool-call telemetry, cost/latency budgets, and SLO-based alerting.
Design and build agent evaluation sets, adversarial tests, regression tests, and monitoring for safety/quality drift.
Design and build RAG/retrieval pipelines using vector databases such as Chroma, Milvus, Weaviate, and Qdrant.
Consult and collaborate with SMEs across Platform Engineering, Data Science, Security, Quality, and Product to ensure agent systems meet technical and operational requirements.
Lead the creation, modification, and maintenance of AI system documentation, including architecture decision records (ADRs), runbooks, API specifications, network topology diagrams, data flow diagrams, and best-practice guides, maintained in a compliant and auditable state.
Influence functional groups regarding operability, technical feasibility, engineering design, security posture, maintainability, and project schedule and documentation requirements.
Assist with the planning and implementation of AI agent-based projects and solutions in collaboration with cross-functional teams, including determination and delivery on cost, timeline, and quality to ensure sustained production operations.
Lead containerized deployment, start-up, commissioning, and release qualification activities using Docker multi-stage builds, dependency management, and container health check best practices.
Provide technical mentoring and oversight to less experienced engineers responding to and troubleshooting AI agent and service-related issues.
Create, edit, and adhere to Standard Operating Procedures (SOPs), process improvements, and standardization of engineering templates for agent development.
Monitor agent performance and reliability, collect metrics and logs, and use data to identify and prioritize optimization opportunities.
Perform ad-hoc and cross-functional projects assigned to support business needs and provide developmental opportunities.
Create, edit and adhere to Standard Operating Procedures (SOPs), process improvements, and standardization of templates.
Performs ad-hoc and cross-functional duties and/or projects assigned to support business needs and provide developmental opportunities.

Benefits

Medical, Dental and Vision Plan Options
Health and Financial Wellness Programs
Employer Assistance Program (EAP)
Company Paid and Voluntary Life/AD&D, Short-Term and Long-Term Disability
Healthcare and Dependent Care Flexible Spending Accounts
401(k) Retirement Plan with Company Match
529 Education Savings Program
Voluntary Legal Services, Identity Theft Protection, Pet Insurance and Employee Discounts, Rewards and Perks
Paid Time Off (PTO) includes: 11 Holidays
Exempt Employees are eligible for Unlimited PTO
Non-Exempt Employees are eligible for 10 Vacation Days, 56 Hours of Health Pay, 2 Personal Days and 1 Cultural Day