Principal AI Engineer

Vertex Inc.•Remote - PA, PA

2d•$159,600 - $207,500

About The Position

The Principal Engineer, AI Model Training & Data Strategy owns how Commercial AI (CAI) products train, fine-tune, and evaluate models, and how the data behind those models is sourced, curated, stored, and governed. This is primarily a model-training role with a strong secondary focus on the data management and pipelines that make high-quality training possible. The role defines the enterprise training strategy and the standards for how and where training data from Commercial AI products is stored, versioned, and reused.

Requirements

Strong hands-on experience training and fine-tuning both traditional AI/ML models and LLMs in production
Deep experience with parameter-efficient fine-tuning (QLoRA, LoRA, PEFT), quantization, and the tradeoffs versus full fine-tuning
Proficiency with ML/DL frameworks and libraries (e.g., PyTorch, Hugging Face Transformers/PEFT/TRL, scikit-learn)
Experience building and operating large-scale data pipelines and platforms (e.g., Spark, Ray, dbt, or equivalents)
Strong grasp of data management: dataset storage architecture, versioning, lineage, governance, and PII handling
Experience with experiment tracking and reproducible ML (e.g., MLflow, Weights & Biases)
Understanding of distributed training and GPU/compute optimization
Ability to define strategy and standards while remaining hands-on in code
Strong stakeholder collaboration and problem-solving skills
Bachelor’s degree in Computer Science, Engineering, or related discipline
12 or more years of experience in AI/ML engineering, applied ML, or data engineering, with significant hands-on model training and fine-tuning

Nice To Haves

advanced degree in ML, AI, or Data Science preferred

Responsibilities

Define and own the end-to-end model training strategy across CAI products, spanning traditional AI/ML models and large language models
Fine-tune large language models using parameter-efficient techniques (e.g., QLoRA, LoRA, PEFT) and full fine-tuning where warranted
Train, evaluate, and tune traditional AI/ML models (classification, regression, ranking, clustering, and similar)
Work with large volumes of data – design and optimize pipelines for ingestion, cleaning, labeling, and feature engineering
Define standards for how and where training data from Commercial AI products is stored, versioned, and accessed (data lakes/warehouses, feature stores, dataset registries)
Establish data governance, lineage, quality, licensing/consent, and PII-handling practices for training data
Build reproducible training pipelines and experiment tracking (datasets, hyperparameters, checkpoints, and metrics)
Define evaluation methodology and benchmarks for model quality, including offline evaluation and regression testing
Curate and clean training, validation, and test datasets, including synthetic data generation where appropriate
Optimize training cost and compute utilization (GPU efficiency, distributed training, quantization)
Partner with product and platform teams to operationalize and hand off trained and fine-tuned models to production
Mentor engineers and raise model-training and data-quality maturity across teams