Software Engineering SMTS

Salesforce•San Francisco, CA

3d•Hybrid

About The Position

Salesforce is seeking an enthusiastic Senior Member of Technical Staff (SMTS) with an AI-first approach to join the Product Led Growth team within the Self-Service and Growth organization. This role focuses on building and maintaining the validation of AI-driven code to ensure it is production-ready for customers. The team is working on initiatives like Onboarding Workflows, Agentic Getting Started, and Slack UI, aiming to help new customers self-service and quickly realize value from Salesforce's Customer 360. The SMTS will play a key role in defining and implementing quality strategies, collaborating with cross-functional teams, and leveraging AI tools to accelerate and improve testing accuracy.

Requirements

Strong technical fundamentals of Java, particularly in data structures and algorithms.
Hands-on experience evaluating Large Language Models (LLMs) for quality response, including familiarity with GPT-like models and AI evaluation frameworks is a big plus.
Proven track record of delivering automation solutions in a fast-paced CI/CD environment, supporting rapid deployment schedules without compromising quality.
Excellent communication skills, with the ability to work cross-functionally with product managers, engineers, AI/ML engineers, and data scientists.
Strong analytical and problem-solving abilities with a focus on identifying quality gaps and driving improvement.

Nice To Haves

A software engineer with a strong coding background and understanding of delivering software at pace in Agentic PDLC.
Strong experience with white-box testing and fixing code is a huge plus.
Experience working with public cloud infrastructure (AWS/Azure/GCP) and/or foundational knowledge of the Salesforce platform (Salesforce Certifications are a plus).
Prior experience testing autonomous agents, conversational UIs, or generative AI workflows.

Responsibilities

Design, build, and maintain scalable, AI-first testing frameworks and automation suites to validate complex PLG workflows and agentic capabilities.
Translate high-level quality strategies into actionable automation plans, focusing heavily on validating LLM integrations, prompt accuracy, and deterministic system behaviors.
Dig deep into the application code to understand system architecture, identify quality gaps, perform white-box testing, and actively contribute to fixing code and test flakiness.
Implement and support evaluation processes for subjective quality dimensions of our AI features, including response relevance, user safety, and contextual accuracy.
Partner closely with developers, product managers, and engineers within the PLG ecosystem to ensure quality is integrated early and continuously throughout the agile delivery lifecycle.