Sr. Technical Analyst, Agentic AI

Johnson & Johnson Innovative Medicine•Raritan, NJ

1d•$94,000 - $151,800•Hybrid

About The Position

We are searching for the best talent for a Sr. Technical Analyst, Agentic AI to be located in Raritan, NJ. MedTech R&D IT – Data and AI, Lead Analyst – Agentic AI (Document Generation & Cognitive Search). Position Summary: We are seeking a Product Lead Analyst with deep expertise in agentic AI systems to drive the design, delivery, and continuous improvement of autonomous document-generation and cognitive-search platforms. This role operates at the intersection of product analysis, AI/ML solution delivery, and enterprise content workflows. The ideal candidate is an agentic AI expert who understands how to orchestrate large language models, retrieval-augmented generation pipelines, and multi-step reasoning agents to produce consistent, traceable, review-ready outputs at scale.

Requirements

5–8+ years in product/business analysis within technology, Life Sciences, or regulated environments
Demonstrable expertise in agentic AI systems – designing, deploying, or managing multi-step autonomous AI workflows
Hands-on experience with LLMs, RAG pipelines, prompt engineering, and at least one agentic framework (LangGraph, CrewAI, AutoGen, or equivalent)
Strong understanding of semantic search, vector databases, and retrieval strategies
Proven ability to translate complex AI/ML capabilities into clear business requirements, user stories, and acceptance criteria
Experience with AI evaluation and quality metrics (hallucination rate, citation accuracy, retrieval precision/recall)
Agentic AI & Orchestration: Multi-agent frameworks (LangGraph, CrewAI, AutoGen, or equivalent), tool-use patterns, chain-of-thought reasoning, planning/reflection loops, agent memory and state management
LLM & NLP: Large Language Models (GPT, Claude, open-source), prompt engineering, fine-tuning, Retrieval-Augmented Generation (RAG), semantic chunking, embedding models
Search & Retrieval: Vector databases (Pinecone, Weaviate, OpenSearch), semantic search, hybrid search (BM25 + vector), re-ranking, knowledge graphs
Cloud & Infrastructure: AWS (Bedrock, S3, Lambda, Step Functions, SageMaker), or equivalent cloud AI services; API design and integration patterns
Document & Content Platforms: PLM systems (e.g., Windchill), document management, content extraction/parsing, regulatory submission workflows
Evaluation & Observability: LLM evaluation frameworks (RAGAS, DeepEval, or equivalent), tracing (LangSmith, Arize), hallucination detection, citation validation

Nice To Haves

Experience with document-generation or intelligent document processing (IDP) solutions
Familiarity with PLM systems, enterprise content management, or regulated document workflows
Background in AWS cloud services (Bedrock, SageMaker, Step Functions, Agentcore) or similar cloud AI platforms
Knowledge of responsible AI practices, model governance, and AI safety frameworks
Certification or training in AI/ML (AWS ML Specialty, Stanford AI courses, or equivalent)

Responsibilities

Own the product backlog and roadmap for agentic document-generation solutions that autonomously draft technical, regulatory, and compliance documents from source evidence.
Design and refine reusable prompt templates, agent workflows, and orchestration logic that drive source-document ingestion, multi-step reasoning, and output composition with full section-level traceability.
Define acceptance criteria ensuring generated drafts are consistent, citation-accurate, and traceable to source documents for human expert review and approval.
Collaborate with domain stakeholders to onboard new document-generation use cases, validate output quality, and expand platform coverage.
Own the product backlog for agentic cognitive-search platforms that go beyond keyword lookup – interpreting intent, reasoning across documents, and synthesizing cited answers.
Define and validate natural-language query experiences where every statement in the synthesized answer is cited to its source document.
Manage requirements for modular search capabilities including evidence retrieval, lifecycle intelligence, and domain-specific search agents.
Ensure role-based access governance so synthesized answers respect the permissions held on underlying source documents.
Drive end-to-end delivery of agentic AI features including sprint planning, release readiness, and go/no-go criteria.
Translate business requirements into user stories, detailed specifications, and acceptance criteria for engineering teams building agent pipelines.
Coordinate cross-team dependencies, conduct change-impact analysis, and manage risk/milestone tracking across multiple agentic workstreams.
Monitor output quality of AI-generated content and search answers; define feedback loops, guardrails, and continuous-improvement metrics.
Support incident and problem management for agent orchestration, LLM performance, retrieval accuracy, and hallucination detection.
Contribute to observability improvements including citation-accuracy scoring, traceability audits, prompt-performance benchmarking, and agent-step telemetry.
Ensure agentic AI outputs adhere to applicable quality-system, validation, and regulatory requirements.
Maintain audit-ready traceability artifacts demonstrating every generated section is linked to its source evidence.
Support responsible-AI practices including bias monitoring, content safety, and ethical use of generative models.
Collaborate with business, regulatory, clinical, quality, and engineering teams to expand agentic platform adoption.
Drive onboarding of new document-generation and cognitive-search use cases across business units.

Benefits

medical
dental
vision
life insurance
short- and long-term disability
business accident insurance
group legal insurance
consolidated retirement plan (pension)
savings plan (401(k))
Vacation –120 hours per calendar year
Sick time - 40 hours per calendar year (56 hours for Washington State residents)
Holiday pay, including Floating Holidays –13 days per calendar year
Work, Personal and Family Time - up to 40 hours per calendar year
Parental Leave – 480 hours within one year of the birth/adoption/foster care of a child
Condolence Leave – 30 days for an immediate family member: 5 days for an extended family member
Caregiver Leave – 10 days
Volunteer Leave – 4 days
Military Spouse Time-Off – 80 hours