AI Engineer

PCORI•Washington, DC

1d•$125,000 - $145,000

About The Position

The Patient-Centered Outcomes Research Institute (PCORI) is an independent nonprofit organization authorized by Congress in 2010. Its mission is to fund research that will provide patients, their caregivers and clinicians with the evidence-based information needed to make better-informed healthcare decisions. PCORI is committed to continually seeking input from a broad range of stakeholders to guide its work PCORI is seeking a highly motivated and innovative AI Engineer to design, build, and deploy advanced AI Capabilities. Examples include Natural Language Processing (NLP) and Machine Learning (ML) models that automate portfolio coding and classification tasks. This role will focus on developing text classification models to analyze and code Letters of Intent (LOIs) and research plans against PCORI’s taxonomy framework—improving efficiency, accuracy, and scalability of manual coding processes. We are also looking for an AI engineer who can support in deploying employee productivity tools (e.g., Microsoft Copilot). The AI Engineer will be part of the Data & Technology Infrastructure team reporting to the Director of the unit and will partner and collaborate with all teams at PCORI to strategize, define, and implement artificial intelligence and machine learning solutions that solve complex business challenges. In this role, you will work closely with cross-functional teams to leverage data, optimize algorithms, and deliver intelligent systems that enhance decision-making and automation in PCORI

Requirements

Master’s degree in computer science, Business Analytics, Data Science, Engineering, Applied Mathematics, or related field
Proven experience in Machine Learning and NLP, particularly in text classification and multi-label problems
Strong programming skills in Python (PyTorch/TensorFlow, Hugging Face Transformers), R or Java, and frameworks like TensorFlow, PyTorch, or Scikit-learn
Familiarity with end-to-end software development for AI models, from training to deployment
Understanding of MLOps tools (MLflow, Weights & Biases, or similar)
Experience with cloud platforms (Azure preferred; AWS or GCP acceptable).
Solid understanding of algorithms, statistics, and model evaluation techniques
Strong problem-solving skills and ability to work in a team-oriented environment

Nice To Haves

Experience working with LLMs (GPT models, Microsoft OpenAI Service)
Knowledge of retrieval-augmented generation (RAG) and semantic search techniques
Proficiency in data augmentation techniques to expand small datasets
Exposure to LangChain, agent workflows, or enterprise AI frameworks

Responsibilities

Design, build, and deploy AI/ML models to address business needs
Collect, preprocess, and analyze structures and unstructured data
Collaborate with product, engineering, PMO, security, and business team to identify AI opportunities
Develop and maintain scalable data pipelines and model training environments
Monitor model performance, retrain, and improve algorithms as needed
Stay up to date with the latest AI/ML research, frameworks and tools
Ensure AI systems comply with ethical, security, and regulatory requirements
Deploy AI solutions (e.g., Microsoft Copilot)
Design, develop, and optimize NLP models (e.g., BERT, PubMedBERT, GPT-family) for multi-label classification of research documents
Align AI models with taxonomy domains and coding frameworks, ensuring outputs meet organizational standards
Build systems to generate annotated outputs that highlight coded sections of research documents for enhanced human review.
Evaluate model performance against gold-standard datasets using accuracy, precision, recall, and F1-score.
Deploy and manage models in production using MLOps practices (experiment tracking, monitoring, retraining).
Collaborate with data engineers, analysts, and domain experts to refine models and integrate them into operational workflows.
Explore advanced approaches such as LLM integration, retrieval-augmented generation (RAG), and semantic search.
Work with Azure (preferred) or other cloud platforms to build scalable and secure deployment pipelines.