Data Scientist (Databricks, Mosaic AI)

OpenDataJobs • Alexandria, VA

2d • $110,000 - $140,000 • Remote

About The Position

Peregrine Advisors is hiring a full-time, remote Data Scientist to build, evaluate, and productionize machine learning and generative AI models for federal agencies. The initial project will likely involve taking models and AI agents from experimentation to production on Azure Databricks and producing evidence that they work. The role may include developing and training models, tracking experiments and model versions with MLflow, building reusable features and serving models, developing retrieval and agent applications, evaluating model and agent quality, monitoring production performance, and documenting model use and limitations. The successful candidate will help deliver trustworthy, explainable models and reusable pipelines that teams can retrain and rerun. Responsibilities may evolve across clients and assignments as the firm grows.

This is a W-2 position with an annual salary of $110,000 to $140,000. The initial engagement requires United States citizenship; dual citizens and green card holders are excluded under the contract terms. The position also requires the ability to satisfy suitability, security, and access requirements.

Requirements

• At least 4 years of data science or machine learning engineering experience, including deploying models to production.
• Bachelor's degree.
• Python for data science and machine learning, including pandas, scikit-learn, and PySpark.
• Hands-on Azure Databricks experience, including MLflow for experiment tracking and model management.
• Experience deploying and monitoring production models, including model serving, versioning, and drift monitoring.
• Experience building LLM applications, including retrieval-augmented generation (RAG).
• Experience evaluating model and LLM outputs against defined metrics, test sets, and human review.
• SQL and Git-based version control.
• Delivery experience in a federal government environment, including its security, access, and review processes.
• Proficiency in writing, PowerPoint, and Excel.
• United States citizenship; dual citizens and green card holders are prohibited by the contract terms.
• Ability to satisfy required suitability, security, and access requirements.

Nice To Haves

• Experience on a federal system accredited at Impact Level 5 (IL-5) or higher.
• Experience with Agent Bricks, Databricks AI Search (formerly Vector Search), or MLflow 3 evaluation for generative AI.
• Experience with another managed machine-learning platform, such as Azure Machine Learning, Amazon SageMaker, or Google Vertex AI.
• Experience with Azure OpenAI or other models in Microsoft Foundry.
• Databricks certification or a Microsoft Azure AI or data science certification.
• Model risk management or model validation experience, including preparing documentation requested for model review.
• Familiarity with AI-assisted developer tooling.

Responsibilities

• Develop and train machine learning models on Azure Databricks, tracking experiments, models, and versions in MLflow.
• Build reusable features in Databricks Feature Store and serve models through Databricks Model Serving.
• Build retrieval and agent applications with Agent Bricks and Databricks AI Search, using Azure OpenAI models through Microsoft Foundry.
• Evaluate model and agent quality using MLflow evaluation, including LLM judges, labeled test sets, and human review.
• Monitor production models for drift and quality, and document models for review.
• Measure model quality before and after release, record intended uses and limitations, and build pipelines teams can retrain and rerun.
• Work across assignments and clients and contribute to the firm's capabilities, tools, and lines of business.

Benefits

• Medical, dental, and vision insurance with the employee premium fully paid and half of dependent premiums paid.
• Employer-paid life, accidental death, short-term disability, and long-term disability insurance.
• 401(k) with employer match of up to 4% of salary.
• Unlimited paid time off.
• Sponsored professional certifications and continuing education.
• Extensive onboarding and opportunities for development across assignments and clients.
• Rotation across functions to expand skills and perspective.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level