AI Engineer

Alaffia Health•New York, NY

12d

About The Position

Alaffia Health is a healthcare AI startup using generative AI and machine learning to improve medical billing, claims review, and data automation. The AI Engineer will build and improve the intelligence behind production AI agents that review real claims, making them more accurate and better grounded in policies, contracts, clinical evidence, and medical records. This model-focused role spans agent development, LLM development and fine-tuning, retrieval-augmented generation (RAG), and reinforcement learning. The engineer will take ideas from early experiments through production and help shape how Alaffia builds, trains, and improves its AI systems. The role works closely with AI engineers, the MLOps team, clinical subject-matter experts, and—on production integration—the Software and DevOps teams. Models and agents must handle protected health information securely and comply with HIPAA.

Requirements

At least 3 years of experience in applied machine learning or AI engineering; experience developing agentic systems, fine-tuning LLMs, and building production RAG pipelines is preferred within this requirement.
Strong software engineering fundamentals and proficiency in Python.
Extensive experience developing AI agents and agentic workflows, including tool calling, orchestration, and multi-step reasoning.
Hands-on experience with LLM fine-tuning, including SFT and PEFT/LoRA, and training pipelines using frameworks such as PyTorch and Hugging Face.
Proven experience building production RAG systems, including vector databases, embedding models, and retrieval optimization.
Practical experience applying reinforcement learning to LLMs, such as RLHF, DPO, PPO, or reward modeling.
Strong understanding of LLM evaluation methodologies and experiment tracking.
Current authorization to work in the United States; the company is unable to sponsor work visas at this time.

Nice To Haves

Experience with agent frameworks such as LangGraph, LangChain, LlamaIndex, or similar.
Familiarity with LLM provider APIs and self-hosted model serving, such as vLLM or TGI.
Experience with distributed training and GPU infrastructure.
Working knowledge of AI tooling such as MLFlow, LangFuse, and Weights & Biases.
Experience with healthcare data, including claims, EHRs, or medical coding such as ICD-10/CPT, or experience in other regulated domains.

Responsibilities

Design and build single- and multi-agent systems for claim review, including planning, tool use, memory, and multi-step reasoning across claims data, medical records, and coding guidelines.
Build and optimize RAG pipelines over policies, contracts, clinical guidelines, and medical records, including chunking, embeddings, hybrid search, reranking, grounding, and citations.
Fine-tune open-source and proprietary LLMs on domain-specific healthcare and claims data using methods such as supervised fine-tuning, LoRA/QLoRA, and instruction tuning; manage dataset curation, training runs, and model versioning.
Develop reinforcement learning and alignment methods, such as RLHF, DPO, reward modeling, and related approaches, using clinical auditor feedback to improve agent accuracy, reasoning, and decision quality.
Define evaluation frameworks, benchmarks, and metrics for agents, fine-tuned models, and RAG systems, including hallucination, faithfulness, retrieval quality, and task-level accuracy; run and track experiments rigorously.
Collaborate with Software and DevOps teams to take models and agents from prototype to production, optimizing latency, cost, and reliability at scale.
Ensure models and agents handle PHI securely and comply with HIPAA; build guardrails for accurate, explainable, and auditable outputs.