Senior Software Engineer

KMM Technologies•Columbia, MD

2d•Hybrid

About The Position

The client needs an experienced SDET/Automation Engineer who specializes in validating AI/ML and Generative AI systems, including testing LLM outputs, prompts, and RAG implementations. Ideal candidate will have strong technical expertise including writing complex SQL queries, automation (Selenium/BDD), and hands-on experience with cloud platforms - specifically AWS (SageMaker/Bedrock) and the Salesforce ecosystem. The candidate must have a background in the healthcare claims industry and understand critical AI challenges like model drift, hallucination detection, and bias evaluation. This role involves collaborating with cross-functional teams to build test strategies, monitor chatbots, and ensure quality delivery during a Snowflake-to-AWS migration. Excellent communication skills and adaptability are essential to clearly articulate AI testing findings to diverse stakeholders and keep up with evolving Gen AI tools.

Requirements

SDET/Automation Engineer background.
Mandatory healthcare claims industry experience.
Deep expertise in testing AI/ML and Gen AI systems (LLMs, RAG, prompt engineering).
Strong AWS (SageMaker/Bedrock) skills.
Salesforce skills.
SQL/automation skills.
Excellent communication skills.
Adaptability.

Nice To Haves

Experience with Databricks.
Experience with Snowflake.
Experience with CloudBees.
Experience with Jenkins.

Responsibilities

Validating AI/ML and Generative AI systems, including testing LLM outputs, prompts, and RAG implementations.
Writing complex SQL queries.
Automation testing (Selenium/BDD).
Hands-on experience with cloud platforms - specifically AWS (SageMaker/Bedrock) and the Salesforce ecosystem.
Understanding critical AI challenges like model drift, hallucination detection, and bias evaluation.
Collaborating with cross-functional teams to build test strategies.
Monitoring chatbots.
Ensuring quality delivery during a Snowflake-to-AWS migration.
Clearly articulating AI testing findings to diverse stakeholders.
Keeping up with evolving Gen AI tools.