AI Developer

Cobalt Benefits Group•Boston, MA

About The Position

Cobalt Benefits Group (CBG), a healthcare third-party administrator, is seeking an AI Developer to design, build, test, and deploy governed AI solutions that improve operational efficiency and member experience. This hands-on role turns approved use cases into production-ready retrieval-augmented assistants, document-extraction agents, and AI-enabled workflows using custom development and low-code Microsoft tools. The developer partners with business owners, DevOps, IT Security, Compliance, and AI governance stakeholders to integrate solutions securely with core systems and establish reusable patterns for responsible AI delivery. The position works with technologies including Microsoft 365 Copilot, Copilot Studio, Power Automate, Power Apps, Teams, Azure OpenAI/Azure AI Foundry, Azure AI Search, Azure API Management, Entra ID, Key Vault, Azure SQL, Cosmos DB, Azure Data Lake Storage, Blob Storage, Dataverse, GitHub, and CI/CD pipelines. Specific technologies vary by project and risk tier. Solutions must meet CBG governance and technical security requirements, with privacy, security, and human oversight proportionate to risk. Automated claim denials without human review and use of PHI in public generative AI models are prohibited. Work aligns with HIPAA, HITRUST CSF, the NIST AI Risk Management Framework, and applicable OWASP guidance. Success measures include completing governance onboarding and shipping an enhancement through Production Readiness Review in the first 90 days; releasing an approved AI solution and enabling monitoring dashboards within six months; and meeting approved business KPIs while maintaining current solution records within 12 months.

Requirements

Bachelor's degree in Computer Science, Information Systems, a related field, or equivalent practical experience.
At least 3 years of professional software development experience.
Hands-on generative AI development and evaluation experience, including RAG, context engineering, tool/function calling, and at least one production or production-pilot solution.
Strong Python and/or C#/.NET development skills.
Experience with production APIs, REST integration, OAuth or similar authorization, and secure secrets/configuration management.
Ability to review, test, debug, and maintain AI-generated code.
Working knowledge of SQL, data modeling, and data pipelines.
Experience handling HIPAA-regulated information, PHI masking, and access controls.
Experience with AI regression testing, monitoring, model version management, release controls, cost optimization, and guarded multi-step agents.
Ability to document technical decisions, follow governance and change control, conduct business requirements sessions, and recommend delaying or stopping solutions that fail evaluation thresholds.
Collaborative communication skills to translate business processes into technical requirements and work across business, engineering, security, and compliance teams.
Analytical, detail-oriented problem-solving skills and an appreciation for measurable quality, documentation, and continuous improvement.
Sound judgment and accountability in privacy, fairness, human oversight, and responsible AI delivery.
Adaptability and hands-on building skills, including selecting between low-code and custom engineering approaches.

Nice To Haves

Experience with Microsoft Copilot Studio, Power Automate, Power Platform, Azure OpenAI, and Azure AI Search.
Demonstrated improvement of retrieval or extraction systems using before-and-after evaluation results.
Experience with AI coding agents and Model Context Protocol (MCP).
Familiarity with healthcare, third-party administration, employee benefits, claims, eligibility, plan documents, or EDI.

Responsibilities

Design, prototype, build, test, and deploy AI applications and agents through production; select appropriate Copilot Studio, workflow, agent, custom-service, vendor, or non-AI approaches under Build-vs-Buy guidance.
Develop retrieval-augmented generation (RAG) solutions, including document ingestion, chunking, indexing, metadata, retrieval tuning, and grounded responses with source citations.
Integrate with internal and third-party systems through authenticated REST and GraphQL APIs, and collaborate on data pipelines for documents and structured data.
Apply least-privilege permissions, single sign-on, secrets management, PHI/PII protections, output validation, and appropriate logging and retention controls in collaboration with IT Security.
Create evaluation harnesses using golden test sets, subject-matter expert rubrics, and production sampling; monitor accuracy, citation support and completeness, latency, and transaction cost, including results by plan type.
Implement telemetry for prompts, retrieval, tool calls, and outputs; maintain runbooks, hand-off documentation, cost controls, and operational monitoring.
Advance solutions through CBG AI Solution Delivery Playbook decision gates and required artifacts, including risk assessments, requirements, evaluation plans, and transition documentation.
Partner with the AI Risk Manager, Compliance, Data Protection Officer, Security, and AI Ethics Committee as appropriate; share reusable components, templates, and approved prompt patterns.
Complete AI governance onboarding, ship an enhancement through Production Readiness Review, and establish a golden-test-set evaluation harness within the first 90 days.
Release a new approved AI solution, publish reusable components adopted by another project, and enable monitoring dashboards within the first six months.
Meet approved business KPIs, demonstrate reusable delivery patterns, and maintain current inventory, risk classification, evaluation, and monitoring records within the first 12 months.

Benefits

Medical, dental, and vision insurance for eligible full-time employees after a waiting period.
Twice-annual employer HSA contributions covering 50% of the HDHP plan’s annual deductible.
Company-provided Basic Life and AD&D insurance.
Company-paid short-term and long-term disability.
Flexible Spending Accounts.
401(k) retirement plan with up to a 6% employer match, fully vested after 3 years.
10+ paid holidays.
Fully paid half-day Summer Fridays.
Paid vacation and sick time.
Annual paid volunteer day.
Annual tuition reimbursement.
Annual health and wellness reimbursement.
Company events.