Agentic AI Engineer in Test

Santander • Miami, FL

1d

About The Position

Santander is a global leader and innovator in the financial services industry and is evolving from a high-impact brand into a technology-driven organization. Our people are at the heart of this journey and together, we are driving a customer-centric transformation that values bold thinking, innovation, and the courage to challenge what’s possible. This is more than a strategic shift. It’s a chance for driven professionals to grow, learn, and make a real difference. If you are interested in exploring the possibilities We Want to Talk to You! The Agentic AI Engineer designs, builds, and deploys autonomous AI agents powered by large language models (LLMs) capable of reasoning, planning, using tools, and executing multi-step tasks. This role combines software engineering, machine learning, LLM integration, and workflow orchestration to create intelligent systems capable of taking multi-step actions with minimal human supervision. This role blends software engineering, AI/ML integration, orchestration, and secure tool-access protocols (such as MCP) to deliver safe, reliable, and scalable agentic systems for enterprise environments. The engineer collaborates with data, platform, security, product, and SDET teams to transform business workflows into intelligent, AI-driven processes, to deliver reliable, safe, and scalable agentic solutions.

Requirements

• Bachelor's Degree or equivalent work experience: Computer Science, Software Engineering or equivalent field. - Required.
• 0+ Years Experience with test strategies, test plans, and test cases. - Required.
• Experience in writing SQLs to query the backend during the test execution.
• Expertise in testing web based applications and web services based systems with minimal to no user interface.
• Experience building test automation frameworks for AI/ML or distributed systems.
• Understanding of LLM evaluation techniques (BLEU, factuality checks, grounding scores, custom metrics).
• Ability to debug agent reasoning errors, hallucinations, planning failures, and tool misuse.
• Knowledge of testing RAG systems (retrieval correctness, latency, vector search quality).
• Experience with structured testing for tool calls, MCP tools, API integrations, and orchestration chains.
• Strong skills in observability, log analysis, and test data management.
• Ability to apply responsible AI and safety testing practices.
• Experience integrating AI with internal tools, data sources, and enterprise services.
• Cloud engineering experience (AWS/Azure/GCP) and containerization (Docker, Kubernetes).
• Understanding of AI governance, responsible AI, and compliance requirements.
• Experience with JIRA/ALM/Xray or similar tools.
• Strong analytical, communication, documentation, and problem-solving skills.
• Ability to manage workload, collaborate in Agile teams, effectively, and work independently.

Nice To Haves

• Established work history or equivalent demonstrated through a combination of work experience, training, military service, or education.
• Experience in Microsoft Office products.

Responsibilities

• Design and implement agentic systems with planning, reasoning, memory, tool use, and multi-step workflows.
• Build LLM-powered pipelines using frameworks such as LangChain, AutoGen, Llama Index, or custom runtimes.
• Implement MCP (Model Context Protocol) or similar protocols for secure and governed tool/API access.
• Integrate agents with APIs, microservices, workflows, distributed systems, and enterprise platforms.
• Develop retrieval pipelines (RAG), embeddings, memory stores, and prompt strategies, frameworks so as LLM/RAG test automation solutions.
• Implement observability, telemetry, and monitoring for agent decisions and tool interactions.
• Build CI/CD, automation, and MLOps practices for agent deployment.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level