Generative AI Developer

Marsh McLennan•Conyers, GA

1d•$83,200 - $124,800•Remote

About The Position

Marsh McLennan Agency (MMA) is seeking a Gen AI Developer who can build, test, and deploy applications powered by large language models and machine learning tools. This role involves reviewing system performance, prompt engineering, integrating APIs, building data processing pipelines, fine-tuning models, collaborating with product and engineering teams, running evaluations for safety and bias, optimizing costs, and deploying updates. MMA provides business insurance, employee health & benefits, retirement, and private client insurance solutions. With offices across North America, MMA combines the personalized service of a local consultant with the global resources of Marsh.

Requirements

Designing and delivering AI-enabled applications and agentic systems in production
Building multi-agent architectures, including planner, executor, reviewer, critic, and supervisor patterns
Developing tool-calling, function-calling, and memory-enabled agents
Using LangChain, LangGraph, and similar frameworks for orchestration and workflow design
Applying DSPy, prompt engineering, and optimization techniques to improve quality and repeatability
Building production-grade AI services in Python
Integrating with leading LLM platforms such as Anthropic, OpenAI, and open-source models
Designing RAG solutions using vector databases
Integrating AI systems with enterprise platforms through APIs, workflows, events, and data services
Implementing observability, evaluation, and optimization for AI systems, including tracing, decision logging, and performance monitoring
Working within governance, security, privacy, and compliance frameworks, ideally in regulated industries
Leading complex technical delivery while collaborating effectively across engineering, product, business, risk, and compliance teams
Overall 10+ years of technology experience, including delivery of scalable, resilient digital products

Responsibilities

Build, test, and deploy applications powered by large language models and machine learning tools.
Review system performance, checking logs for API errors, latency issues, and output quality from deployed models.
Refine and test system prompts to improve response accuracy and reduce hallucinations.
Connect large language models (like GPT-4 or Claude) with internal databases using retrieval-augmented generation (RAG) frameworks.
Write Python code to process data, embed documents, and manage vector databases (such as Pinecone or Milvus).
Adjust model weights or train smaller open-source models on domain-specific company data.
Meet with product managers and software engineers to align AI features with user needs.
Test the application for safety, bias, and edge-case failures using automated testing tools.
Review token usage and API spending to keep cloud infrastructure efficient.
Push new code updates to production environments using CI/CD pipelines.