Principal AI/ML Engineer

Fidelity Investments • Durham, NC

16h • Onsite

About The Position

Designs and develops advanced Machine Learning (ML) and trustworthy AI systems that support large-scale, enterprise-wide platforms. Focuses on secure model development, AI safety, multi-agent orchestration, and cloud-scale ML infrastructure, enabling reliable, transparent, and high-assurance deployment of AI across critical business functions. Builds robust pipelines for model training, inference, and continuous monitoring, ensuring compliance and resilience under real-world conditions. Designs model lifecycle operations (MLOps) infrastructure including SageMaker and Kubernetes for reproducible, scalable, and secure ML deployment. Responsible for research and prototyping of cutting-edge AI methodologies, including federated learning, adversarial robustness, and multi-agent safety mechanisms.

Requirements

• Bachelor’s degree in Computer Science, Engineering, Information Technology, Information Systems, or a closely related field (or foreign education equivalent) and five (5) years of experience as a Principal AI/ML Engineer (or closely related occupation) developing ML platform applications for Cloud infrastructures (Amazon Web Services (AWS), Azure, Google, and IBM) using agile methodologies.
• Master’s degree in Computer Science, Engineering, Information Technology, Information Systems, or a closely related field (or foreign education equivalent) and three (3) years of experience as a Principal AI/ML Engineer (or closely related occupation) developing ML platform applications for Cloud infrastructures (Amazon Web Services (AWS), Azure, Google, and IBM) using agile methodologies.
• Demonstrated Expertise (“DE”) delivering scalable, secure, and distributed applications with robust Identity and Access Management (IAM) and encryption (Knowledge Management System (KMS)), by architecting and deploying enterprise-scale AI/ML systems and auto-ML infrastructure through Infrastructure as code (IAC) and using Cloud-native platforms (Terraform, AWS, Azure, or GCP) and container orchestration frameworks (Kubeflow).
• DE architecting and engineering high-performance big data applications (AWS Glue, EMR, Kinesis, Athena, and Dynamo DB) and autonomous multi-agent systems; designing batch processing jobs and Extract, Transform, Load (ETL) pipelines to support predictive analytics using Hadoop, MongoDB, AWS, and PostgreSQL; developing agentic workflows using frameworks including Strands, CrewAI, LangGraph, and OpenAI Swarm with protocols -- Model Context Protocol (MCP) Server and Accelerated Graphics Port (AGP); and driving context aware predictive analytics and optimization models by implementing multi threaded, asynchronous solutions in Python and Java, supported by short term and long term memory management architectures for Retrieval Augmented Generation (RAG) pipelines using vector databases, OpenSearch, and high performance caching solutions (Redis or Memcached).
• DE automating end-to-end application deployment and MLOps via workflow orchestration tools (Airflow and AWS Step Functions) and CI/CD pipelines (Jenkins and Git); performing model metadata processing to deliver lineage tracking, version control, auditability, reproducibility, and real time analysis of model performance, parameters, and deployment history across distributed ML workflows, using AWS DynamoDB, AWS RDS, MLflow, and AWS Athena; enabling reproducible builds, integrity checks, and scalable CI/CD by securely handling, versioning, and distributing container images, ML models, and software dependencies using JFrog Artifactory; and accelerating model development, monitoring, drift detection, and interpretability within Agile environments by engineering bridge solutions, using Go and FastAPI.
• DE implementing secure AI through adversarial robustness, federated learning, and governance across distributed ML systems using AI Generative Adversarial Network (AIGAN), Google Federated Learning Framework, SageMaker Clarify, and MLflow; enhancing low latency, high throughput model serving and maximizing central processing unit (CPU) or graphics processing unit (GPU) utilization through deployment on accelerated inference servers, using Deep Java Library (DJL), Triton, and Flask; and evaluating ML model inference performance using statistical analysis, monitoring tools (CloudWatch, Datadog, and Splunk), and dashboards including Streamlit and Gradio.

Nice To Haves

• Experience and/or expertise may be gained during doctoral program.

Responsibilities

• Architects and implements AI/ML systems that support model training, inference, observability, and continuous monitoring across distributed cloud environments.
• Develops secure and trustworthy AI frameworks, including adversarial robustness pipelines, anomaly detection models, and model governance mechanisms that ensure compliance, transparency, and risk mitigation.
• Builds and optimizes agentic AI workflows that automate data pipelines, model lifecycle operations, and system self-diagnostics.
• Supports research and prototyping of advanced AI/ML methodologies, including hybrid neural architectures, federated learning, adversarial learning, and multi-agent AI safety mechanisms.
• Conducts performance, reliability, and robustness evaluations of AI systems under real-world constraints, high-throughput workloads, and adversarial conditions.
• Collaborate with cybersecurity, cloud engineering, and data science teams to integrate AI safety and model assurance into enterprise architectures.
• Collaborates with cross-functional teams to integrate AI safety and governance into enterprise architectures while mentoring engineering teams on advanced ML algorithms and secure development practices.
• Delivers technical guidance and mentorship to cross-functional engineering teams on advanced ML algorithms, infrastructure patterns, and secure development practices.

What This Job Offers

• Job Type: Full-time
• Career Level: Principal