Senior Manager, AI Engineer/ Solutions & Engineering

Bristol Myers Squibb•Princeton, NJ
•$153,770 - $186,327

About The Position

Bristol Myers Squibb is seeking a hands-on AI Engineer to build and deploy AI solutions that address business needs, especially enterprise AI assistants and agents used by commercial teams. The role designs how models are prompted, grounded, equipped with tools, evaluated, and governed, and develops production solutions using coding agents. It requires understanding model and agent-harness behavior, including when approaches are likely to work or fail, and designing accordingly. The position spans LLM evaluation and quality engineering, coding-agent-assisted development, applied LLM engineering, model and harness fundamentals, and engineering practices such as LLMOps and security. The engineer partners with marketing, commercial, and IT teams to turn business needs into technical solutions, enable business users to understand AI’s capabilities and limitations, and improve team practices through reusable patterns, peer reviews, and awareness of new models and platform capabilities. The position is based in Princeton, New Jersey. The listed starting compensation range is $153,770–$186,327 for full-time employees. Eligible employees may receive discretionary cash and stock incentives and participate in company benefit plans.

Requirements

  • Bachelor’s degree in Computer Science, Engineering, Data Science, or a related field, or equivalent experience.
  • At least 5 years of software or ML engineering experience, including at least 1 year building LLM- or agent-based solutions.
  • A portfolio of shipped work, such as assistant extensions, connectors, agent workflows, or an evaluation suite.
  • Strong communication skills, including the ability to explain trade-offs to business partners.
  • Experience designing evaluations for non-deterministic systems, including golden datasets, rubric- and LLM-as-judge graders, human review, and regression testing.
  • Ability to evaluate retrieval, tool use, multi-step agent trajectories, and final outputs separately.
  • Ability to calibrate LLM graders against human labels and identify grader bias.
  • Experience running evaluations in CI to measure changes to prompts, tools, skills, or model versions before release.
  • Ability to use evaluation results for error analysis, failure categorization, and targeted improvements.
  • Current, hands-on daily use of coding agents such as Claude Code, Codex, Cursor, or similar to design, build, test, and refactor software.
  • Experience with spec-driven workflows, including writing specifications and acceptance tests, breaking work into agent-sized tasks, and running agents in parallel or on long tasks.
  • Experience configuring coding agents with project instructions, skills, MCP servers, hooks, and permissions.
  • Ability to critically review agent output and explain and defend shipped code.
  • Proven experience delivering LLM solutions in production, rather than prototypes alone.
  • Strong Python skills; TypeScript is also desirable. Experience with API-driven development, Git, CI/CD, and a cloud platform such as AWS, Azure, or GCP.
  • Advanced prompt and context engineering, including system prompts, few-shot patterns, structured outputs, reasoning strategies, and long-context strategies.
  • Retrieval and grounding knowledge, including chunking, hybrid search, reranking, metadata filtering, and understanding when connectors or direct file access can replace a vector database.
  • Experience with agentic patterns such as tool and function calling, multi-step workflows, and MCP, and judgment about when not to use an agent.
  • Experience working with unstructured and semi-structured data, including documents, PDFs, and content assets.
  • Understanding of transformer fundamentals, including tokenization, attention, context windows, and sampling.
  • Understanding of model training approaches, including pretraining, instruction tuning, RLHF/RLAIF, and how reasoning models differ.
  • Understanding of agent harnesses, including the agent loop, tool exposure, context management, and permissions.
  • Knowledge of failure modes such as hallucination, sycophancy, position bias, prompt injection, and tool misuse, and ways to mitigate them.
  • Ability to reason about cost, latency, and quality from first principles.
  • Knowledge of LLMOps, including tracing, observability, prompt and model versioning, and cost and usage management.
  • Knowledge of security fundamentals, including prompt injection through tools and documents, data leakage, PII, and least-privilege access.

Nice To Haves

  • Hands-on experience with enterprise AI assistant platforms such as Claude Desktop (Chat, Cowork, Code, skills, projects, plugins, or connectors), ChatGPT Enterprise, Gemini Enterprise/Gemini for Workspace, or similar platforms.
  • Experience building or configuring MCP servers and integrating enterprise systems as connectors.
  • Experience with agent SDKs or API-based agent development for use cases that exceed assistant-platform capabilities.
  • Experience in pharma, life sciences, or other regulated industries, including awareness of MLR, GxP, or 21 CFR Part 11.
  • Experience with commercial pharma data and systems such as Veeva, PromoMats, or IQVIA, or with digital marketing, content supply chains, or HCP/DTC engagement.
  • Experience with enterprise AI rollout, including adoption programs, workshops, champion networks, or usage governance.
  • Experience with multi-agent or sub-agent orchestration, such as LangGraph.
  • Experience with multimodal or document-understanding workloads.
  • Experience with fine-tuning or distillation, including LoRA or PEFT, and open-weight models such as Hugging Face, Llama, or Mistral.

Responsibilities

  • Build and ship LLM-powered solutions—including assistant extensions, connectors, agentic workflows, and custom applications—from prototype through production.
  • Define quality criteria for each use case and create golden datasets, graders, and regression suites; use evaluations to guide prompt, tool, and model changes.
  • Use coding agents as development tools by writing specifications, directing agents, and reviewing and testing their output while owning correctness and quality.
  • Design solutions to work with AI assistant platform harnesses, including their agent loops, tool calls, context assembly, permissions, and sandboxing.
  • Engineer system prompts, reusable instructions, memory, retrieval, structured outputs, and long-context strategies.
  • Select an appropriate build pattern, such as a platform extension, connector, custom application, or API-based agent, and determine when not to use an agent.
  • Select models by weighing capability, cost, latency, and risk, using evaluation results as evidence.
  • Implement guardrails and human-in-the-loop controls suitable for regulated pharmaceutical content, including MLR review, compliance, auditability, and data boundaries.
  • Monitor and improve quality, cost, latency, and adoption of deployed solutions using production traces and feedback.
  • Translate business needs into technical solutions in partnership with marketing, commercial, and IT stakeholders.
  • Explain AI capabilities, limitations, and value to business stakeholders.
  • Share reusable practices, review peers’ work, and track new models and platform capabilities.

Benefits

  • Comprehensive benefit plans, including wellbeing support, retirement and financial protection benefits, and medical, dental, vision, life, and disability insurance offerings, subject to applicable plan terms and eligibility.
  • Flexible Time Off without a set accrual limit for eligible U.S.-based exempt employees, subject to manager approval.
  • Eleven paid company holidays each year.
  • For eligible non-exempt employees, RayzeBio employees, and employees located in Puerto Rico: 160 hours of paid vacation annually for new hires, subject to manager approval, 11 paid company holidays, and 3 optional holidays.
  • Depending on eligibility, additional time-off benefits may include paid sick leave, up to two paid volunteer days per year, summer-hours flexibility, and leaves for medical, personal, parental, caregiver, bereavement, or military needs.
  • Eligible employees may receive an annual Global Shutdown between Christmas Day and New Year’s Day.
  • May be eligible for discretionary incentive cash and stock opportunities.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service