AI Engineer

Alaffia Health•New York, NY

12d

About The Position

Alaffia Health is a healthcare AI startup focused on reducing improper payments and administrative waste in health plans through expert clinicians and transparent AI. Its platform uses generative AI and machine learning to improve accuracy, efficiency, and compliance in medical billing and auditing. The AI Engineer will build and improve the intelligence behind Alaffia’s production AI agents, which review real claims. This model-focused role spans agent design, LLM development and fine-tuning, retrieval-augmented generation (RAG), and reinforcement learning. The engineer will take ideas from experiments through production systems that reason over claims, medical records, policies, contracts, clinical evidence, and coding guidelines. The role works closely with AI engineers, the MLOps team, clinical subject-matter experts, and—in production integration—Software and DevOps teams. The position requires current authorization to work in the United States; visa sponsorship is not available.

Requirements

At least 3 years of experience in applied ML or AI engineering; ideally including production agentic systems, LLM fine-tuning, and RAG pipelines.
Strong software engineering fundamentals and proficiency in Python.
Extensive experience developing AI agents and agentic workflows, including tool calling, orchestration, and multi-step reasoning.
Hands-on experience with LLM fine-tuning, including SFT and PEFT/LoRA, and training pipelines using frameworks such as PyTorch and Hugging Face.
Proven experience building production RAG systems, including vector databases, embedding models, and retrieval optimization.
Practical experience applying reinforcement learning to LLMs, such as RLHF, DPO, PPO, or reward modeling.
Strong understanding of LLM evaluation methodologies and experiment tracking.
Current authorization to work in the United States; the company is unable to sponsor work visas.

Nice To Haves

Experience with agent frameworks such as LangGraph, LangChain, LlamaIndex, or similar.
Familiarity with LLM provider APIs and self-hosted model serving, such as vLLM or TGI.
Experience with distributed training and GPU infrastructure.
Working knowledge of AI tooling such as MLFlow, LangFuse, and Weights & Biases.
Experience with healthcare data, including claims, EHRs, or medical coding such as ICD-10/CPT, or experience in other regulated domains.

Responsibilities

Design and build single- and multi-agent systems for claim review, including planning, tool use, memory, and multi-step reasoning across claims data, medical records, and coding guidelines.
Build and optimize RAG pipelines over policies, contracts, clinical guidelines, and medical records, including chunking, embeddings, hybrid search, re-ranking, and grounded, cited outputs.
Fine-tune open-source and proprietary LLMs on healthcare and claims data using methods such as SFT, LoRA/QLoRA, and instruction tuning; manage dataset curation, training runs, and model versioning.
Develop reinforcement-learning and alignment methods, including RLHF, DPO, and reward modeling, using clinical auditor feedback to improve agent accuracy, reasoning, and decision quality.
Define evaluation frameworks, benchmarks, and metrics for agents, fine-tuned models, and RAG systems, including hallucination, faithfulness, retrieval quality, and task-level accuracy; run and track experiments.
Collaborate with Software and DevOps teams to move models and agents from prototype to production, optimizing latency, cost, and reliability at scale.
Ensure models and agents handle PHI securely and comply with HIPAA; build guardrails for accurate, explainable, and auditable outputs.