AI Engineer (Agentic AI / AWS Bedrock)

Match Point Solutions•Los Angeles, CA

1d•Hybrid

About The Position

MatchPoint Solutions is seeking an experienced AI Engineer to build, deploy, and monitor production-grade AI agents and Generative AI solutions using AWS technologies. The role focuses on developing agents with Claude Agent SDK and Python, deploying them on Amazon Bedrock AgentCore Runtime, automating deployments with GitHub Actions, and using Langfuse for observability and ongoing evaluation of agent traffic. It combines AI engineering, cloud deployment, data pipeline development, and LLM performance monitoring. The position is listed as a contract role in Los Angeles, California, with a hybrid work arrangement and a pay range of $70–$80 per hour. The engineer will develop autonomous agents and production-ready AWS architectures, implement observability and continuous evaluation, build document-processing and RAG pipelines, and automate CI/CD and deployment processes.

Requirements

6–8+ years of software engineering, cloud engineering, or AI/ML development experience.
Strong Python development experience.
Hands-on experience building Generative AI or Agentic AI applications and deploying and supporting AI agents in production.
Experience with Claude Agent SDK, AI agent development, agent orchestration, and tool calling.
Experience with Amazon Bedrock, Bedrock AgentCore Runtime, AWS Step Functions, Amazon Textract, and Amazon Comprehend.
Experience deploying Langfuse and working with LLM tracing, monitoring, and performance analysis.
Experience with continuous agent evaluation, LLM-as-a-Judge, evaluation metrics, and automated testing.
Experience with GitHub Actions, CI/CD pipelines, Git, and deployment automation.
Experience with vector databases, embeddings, chunking strategies, indexing, and knowledge base integrations for RAG.
Strong understanding of AWS cloud services, LLM architectures, and automated deployment processes, including AWS architecture, IAM, CloudWatch, security, and deployment practices.
Ability to independently design, implement, troubleshoot, and optimize AI agent solutions.

Nice To Haves

Experience with LangChain, LangGraph, or other agent orchestration frameworks.
Experience with RAGAS, DeepEval, or similar LLM evaluation frameworks.
Familiarity with Amazon OpenSearch, Pinecone, FAISS, or pgvector.
Experience with Docker, Terraform, or infrastructure as code.
Knowledge of Model Context Protocol (MCP).
Experience with prompt engineering, guardrails, and AI governance.
Experience deploying enterprise Generative AI solutions in production environments.

Responsibilities

Design, develop, and deploy Python AI agents using Claude Agent SDK, including agents that execute complex workflows and interact with external tools, APIs, and knowledge bases.
Develop agent orchestration workflows with error handling, retry mechanisms, and guardrails, and integrate agents with enterprise applications and data sources.
Deploy and manage AI agents using Amazon Bedrock AgentCore Runtime and build scalable, secure, production-ready agent architectures on AWS.
Integrate agents with Amazon Bedrock foundation models and enterprise knowledge bases; implement authentication, access controls, logging, and runtime monitoring.
Deploy, configure, and maintain Langfuse; integrate it with agents to capture traces, prompts, responses, tool calls, and execution metrics.
Monitor agent performance, latency, token consumption, and operational costs, and troubleshoot production issues using distributed tracing and monitoring dashboards.
Design continuous evaluation frameworks for production agent traffic, evaluating accuracy, relevance, hallucinations, task completion, and tool execution quality.
Build automated evaluation pipelines using Langfuse, LLM-as-a-Judge, and custom metrics; establish quality benchmarks, feedback mechanisms, and regression detection.
Develop AWS Step Functions workflows integrating Amazon Textract and Amazon Comprehend for intelligent document processing.
Build document ingestion, preprocessing, and transformation pipelines, including vector database solutions for chunking, embedding generation, and indexing.
Integrate agents with RAG pipelines and enterprise knowledge bases, optimizing retrieval performance and contextual accuracy.
Build and maintain GitHub Actions CI/CD pipelines to automate builds, testing, deployment, configuration management, and agent evaluations.
Implement deployment workflows for Amazon Bedrock AgentCore Runtime and apply DevOps practices for version control, release management, and production monitoring.