Principal AI Systems Engineer

Iovance Biotherapeutics

8d • $175,000 - $200,000 • Remote

About The Position

This is a senior role responsible for the hands-on design, build, validation, and deployment of Artificial Intelligence (AI) systems at Iovance Biotherapeutics. This role is focused on execution — turning approved use cases into working, validated, production AI systems that deliver measurable business value. This role directly supports Iovance’s strategy to improve overall operational productivity. The ideal candidate is a deeply technical, hands-on senior engineer with demonstrated production experience designing and shipping Large Language Model (LLM) applications, Retrieval-Augmented Generation (RAG) systems, and modern AI integrations. This position works closely with IT leads, business process owners across Manufacturing, Quality, Regulatory Affairs, Clinical, Commercial, G&A, IT Security, Privacy, and Quality Assurance to deliver AI capabilities that respect Iovance’s regulated environment. This role executes against a roadmap and prioritized backlog, while contributing technical input to refinement, scoping, and sequencing.

Requirements

• 10+ years of progressive software and/or AI/ML engineering experience, with the most recent 1+ years dedicated primarily to LLM-based application development.
• Demonstrated track record of shipping production AI systems that real users depend on.
• Deep, hands-on production experience with Retrieval-Augmented Generation (RAG) including chunking strategies, embedding models, vector stores, retrieval and reranking, and grounding for high-accuracy use cases.
• Working production experience with Model Context Protocol (MCP) or equivalent agent/tool integration patterns.
• Practical working experience across multiple frontier model families (e.g., Anthropic Claude, OpenAI GPT, Google Gemini, leading open-weight models), with the judgment to select among them based on task fit, accuracy, cost, latency, safety properties, and data-handling commitments.
• Awareness of model capability changes and how to evaluate new model releases.
• Strong software engineering fundamentals: production Python (and ideally one other language); modern Agile delivery; version control (Git); CI/CD; containerization; cloud platforms (preferably AWS, with working familiarity with S3, Redshift, and Bedrock); MLOps tooling and practices.
• Hands-on, builder disposition; this role spends the majority of its time writing code, designing systems, evaluating models, and producing technical artifacts — not in meetings.
• Working production experience with AWS data and AI services is strongly preferred (S3, Redshift, Bedrock, IAM).
• Working knowledge of enterprise search and retrieval services such as OpenSearch, Elasticsearch, or equivalent, including the design of hybrid retrieval patterns (vector plus lexical/BM25) that improve grounding and relevance in production RAG systems

Nice To Haves

• Relevant cloud and AI/ML certifications (AWS, Azure, etc.)
• Working knowledge of US regulatory requirements applicable to AI in regulated life sciences, including 21 CFR Part 11, GxP, HIPAA, and emerging FDA expectations on AI/ML in pharmaceutical manufacturing and drug development
• Bachelor’s or Master’s degree in Computer Science, Data Science, Engineering, or related quantitative discipline.

Responsibilities

• Design, build, and ship AI systems against the approved Iovance AI roadmap, including end-to-end ownership of architecture, retrieval pipelines, prompts, evaluation, integrations, and deployment for assigned use cases.
• Implement production-grade Retrieval-Augmented Generation (RAG) systems on Iovance’s AWS infrastructure (S3, Redshift, Bedrock), including chunking strategies, embedding selection, vector storage, retrieval and reranking, grounding, and citation handling appropriate to high-accuracy use cases.
• Build, maintain, and run evaluation harnesses for AI systems, including held-out test sets, accuracy and grounding metrics, hallucination detection, adversarial inputs, and regression testing across model and prompt changes; treat evaluation as a first-class engineering deliverable, not an afterthought.
• Design and implement integrations between AI systems and Iovance enterprise systems using Model Context Protocol (MCP), APIs, and event-driven patterns, applying least-privilege access principles and partnering with IT Security on integration approval.
• Own and maintain LLM security controls for production AI systems, including input and output guardrails, prompt injection and jailbreak defenses, sensitive data redaction (PII, PHI, Iovance Confidential Information and Intellectual Property), content moderation, and abuse monitoring, working in partnership with IT Security.
• Design, develop, deploy, and maintain AI agents (multi-step reasoning systems that combine LLMs with tools, retrieval, and planning) appropriate for use in a regulated life sciences environment, including bounded scope, defined human oversight, traceability of agent decisions, and safe handling of write-back actions to systems of record
• Establish and uphold modern engineering practices for AI development including version control for code, prompts, and evaluation sets; CI/CD pipelines; environment separation (dev, test, production); and reproducible builds.
• Conduct hands-on technical evaluation of AI vendors, tools, and Foundation Models when build-vs-buy decisions are under consideration; produce concise, fact-based recommendations to the IT function lead and AI Governance Committee, including proof-of-concept results where appropriate.
• Implement and operate technical controls for production AI systems including audit logging, access management, prompt and model change control, model registry, ongoing performance monitoring, and incident detection, in alignment with Iovance policies.
• Author technical documentation appropriate to the system risk tier, including architecture diagrams, data flow diagrams, evaluation reports, runbooks, and validation deliverables; contribute to the Iovance AI Validation Playbook.
• Mentor junior engineers who collaborate on AI projects; stay current on rapid advances in AI tooling, models, and engineering best practices, and bring technical recommendations forward.

Benefits

• The annual base salary we reasonably expect to pay is listed. Individual pay decisions depend on various factors, such as primary work location, complexity and responsibility of the role, job duties/requirements, and relevant education, experience and skills.
• Pay Transparency

What This Job Offers

• Job Type: Full-time
• Career Level: Senior