Senior Agentic AI Engineer - Core Platforms

American Express•New York, NY
•$123,000 - $215,250

About The Position

The Senior Agentic AI Engineer designs, develops, and maintains evaluation frameworks for AI and machine learning solutions, including LLM, RAG, agentic, and multimodal applications. The role translates business, product, risk, and governance requirements into measurable success criteria; builds automated and human-in-the-loop evaluation pipelines; and works with cross-functional teams to ensure AI systems meet enterprise standards for quality, safety, reliability, compliance, latency, and cost. The position also designs and operates scalable production AI systems and leads development of capabilities such as inference services, retrieval pipelines, orchestration layers, and evaluation frameworks. It supports integration, deployment, monitoring, and maintenance of models and AI systems while upholding enterprise governance, security, and compliance standards. The role mentors junior engineers and communicates evaluation findings, tradeoffs, and remediation priorities to engineering and business stakeholders.

Requirements

  • At least 5 years of software development experience, including hands-on experience in machine learning, AI technologies, and agentic solutions.
  • Bachelor's degree in computer science, machine learning, data science, or comparable experience.
  • Knowledge of Python and data-processing technologies such as Hadoop and Spark.
  • Knowledge of NLP techniques such as fuzzy matching and models such as BERT and transformer models, including real-world applications.
  • Advanced knowledge of machine-learning techniques and performance optimization.
  • Strong understanding of AI system architectures, including LLM ecosystems, retrieval systems, orchestration frameworks, and evaluation methodologies.
  • Experience in software development focused on machine learning, generative AI, Big Data, and prompt engineering.
  • Experience working with LLMs, especially GPT-3.5/4 or similar models.
  • Experience with Python, R, Java, or C++.
  • Experience deploying AI models and pipelines using MLOps tools.
  • Experience with machine-learning algorithms and applying them to real-world problems.
  • Experience with CI/CD pipelines and containerized deployments.
  • Experience building and maintaining data pipelines and ETL processes.
  • Experience delivering end-to-end AI systems in production, including design and operational ownership.
  • Experience with distributed systems design, scalable APIs, and cloud-native AI infrastructure, including monitoring and optimization.
  • Advanced expertise in a technical domain and moderate expertise in a different technical domain.
  • Experience building evaluation harnesses and automated test pipelines for LLM, RAG, or agentic applications using production-representative datasets.
  • Experience designing human-in-the-loop and model-based evaluation workflows, including calibration, error analysis, and validation of evaluation quality.
  • Experience integrating evaluation results into CI/CD, release readiness, production monitoring, and governance evidence.
  • Eligibility to work in the United States; American Express will not pursue visa sponsorship.

Nice To Haves

  • An advanced degree is preferred.

Responsibilities

  • Mentor junior AI engineers and colleagues, supporting professional development and fostering excellence and innovation.
  • Integrate AI models into production systems in collaboration with cross-functional teams.
  • Maintain adherence to model governance, security, and compliance standards.
  • Design and implement AI and machine-learning frameworks for data processing, including prompt engineering and generative AI technologies.
  • Collaborate with engineers to deliver scalable AI solutions that meet technical and business requirements.
  • Translate business requirements into actionable AI-driven technical solutions, models, and algorithms.
  • Develop reusable Big Data pipelines and AI-based systems for efficient, consistent data processing and model deployment.
  • Research, experiment with, and deploy NLP models to classify and analyze business-related text.
  • Create end-to-end MLOps pipelines for scalable model deployment, monitoring, and maintenance.
  • Partner with product and business teams to align technology initiatives with business objectives.
  • Design and operate scalable production AI systems that integrate ML models, LLMs, and intelligent services across components.
  • Lead development of AI capabilities, including inference services, retrieval pipelines such as RAG, orchestration layers, and evaluation frameworks.
  • Ensure AI systems meet enterprise standards for reliability, safety, explainability, observability, and compliance, and mentor others on best practices.
  • Define evaluation strategies, success criteria, benchmarks, and scorecards for LLM, RAG, agentic, and multimodal AI across quality, safety, reliability, latency, and cost.
  • Build automated evaluation pipelines using curated datasets, synthetic test generation, human review, model-based grading, and deterministic checks; validate evaluators for consistency and bias.
  • Establish regression testing, release gates, and continuous evaluation to detect quality degradation, drift, unsafe behavior, and changes caused by models, data, tools, or orchestration.
  • Partner with product, engineering, risk, and governance teams to turn use-case requirements and policy expectations into measurable acceptance criteria and traceable evidence.
  • Analyze failures and communicate actionable findings, tradeoffs, and remediation priorities to engineering and business stakeholders.

Benefits

  • Competitive base salaries
  • Bonus incentives
  • 6% company match on retirement savings plan
  • Free financial coaching and financial well-being support
  • Comprehensive medical, dental, vision, life insurance, and disability benefits
  • Flexible working model, with hybrid, onsite, or virtual arrangements depending on role and business need
  • 20+ weeks of paid parental leave for all parents, including for pregnancy, adoption, or surrogacy
  • Free access to global onsite wellness centers staffed with nurses and doctors, depending on location
  • Free and confidential counseling support through the Healthy Minds program
  • Career development and training opportunities
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service