Salesforce Application Engineer

Phizenix•Menlo Park, CA

About The Position

We are seeking an experienced Salesforce Application Engineer to design and build a scalable quality assurance and evaluation framework for AI-enabled employee support experiences delivered through Salesforce. As AI handles a growing volume of employee interactions, the organization requires a systematic mechanism to evaluate response quality, detect regressions, identify hallucinations, validate policy compliance, and monitor overall AI performance at scale. This role will develop the technical infrastructure, workflows, integrations, dashboards, and automation required to continuously assess AI-generated and human-assisted responses across Salesforce-based employee service channels. The engineer will work closely with AI, Product, Employee Experience, Compliance, Data Science, and Quality Engineering teams to establish measurable quality standards and operational controls.

Requirements

Bachelor's degree in Computer Science, Engineering, Information Systems, or a related discipline.
5+ years of enterprise application development experience.
3+ years of hands-on Salesforce development experience.
Strong experience with Apex, Lightning Web Components, Salesforce Flow, SOQL, REST APIs, Salesforce security and data models.
Experience building applications on Salesforce Service Cloud or Employee Service platforms.
Experience designing automated testing or quality assurance frameworks.
Strong knowledge of software engineering, integration, and CI/CD practices.
Experience working with AI, machine learning, conversational AI, or large language model applications.
Strong analytical, troubleshooting, and communication skills.

Nice To Haves

Experience building evaluation infrastructure for generative AI or conversational AI systems.
Knowledge of AI evaluation concepts such as groundedness, relevance, factuality, hallucination detection, and model regression.
Experience with Salesforce Einstein, Agentforce, Data Cloud, or comparable AI-enabled Salesforce capabilities.
Experience integrating Salesforce with enterprise knowledge bases and AI platforms.
Familiarity with Python, SQL, data pipelines, analytics platforms, and automated evaluation libraries.
Knowledge of prompt management, retrieval-augmented generation, model observability, or LLM operations.
Experience supporting HR, employee service, legal, compliance, or policy-driven applications.
Salesforce Platform Developer or Application Architect certification is preferred.

Responsibilities

Design and implement a comprehensive evaluation framework for AI-generated employee support responses.
Establish automated and human-in-the-loop evaluation workflows.
Develop mechanisms to assess response accuracy, completeness, relevance, clarity, tone, groundedness, policy compliance, and employee experience quality.
Build configurable evaluation scorecards and quality thresholds.
Enable evaluation across multiple use cases, channels, employee groups, and support domains.
Maintain versioned evaluation datasets, benchmarks, and test scenarios.
Design, configure, and develop Salesforce solutions supporting AI quality assurance and evaluation.
Build custom objects, Apex services, Lightning Web Components, flows, validation rules, and automation.
Develop reusable services for capturing AI prompts, responses, source references, confidence scores, and evaluation outcomes.
Extend Salesforce Service Cloud and related employee support capabilities.
Implement secure, scalable, and maintainable solutions aligned with Salesforce engineering standards.
Support sandbox, development, testing, staging, and production environments.
Create automated evaluation pipelines for AI-generated and human-assisted responses.
Compare responses against approved knowledge sources, expected answers, and business policies.
Develop scoring logic for factual accuracy, relevance, completeness, and actionability.
Capture evaluator feedback and convert it into structured quality metrics.
Build workflows for sampling and reviewing high-risk or low-confidence interactions.
Enable trend analysis by use case, model version, region, policy area, and interaction type.
Develop mechanisms to identify unsupported, fabricated, or inconsistent AI responses.
Validate whether responses are grounded in approved enterprise knowledge sources.
Capture and assess citations, source references, retrieval results, and confidence indicators.
Flag responses that contain unverifiable claims or contradict enterprise policy.
Route suspected hallucinations for human review and remediation.
Partner with AI and data science teams to improve retrieval, prompting, and model behavior.
Build automated regression suites for AI-enabled Salesforce capabilities.
Maintain benchmark prompts, expected responses, edge cases, and negative test scenarios.
Compare response quality across model, prompt, knowledge base, workflow, and application releases.
Detect quality degradation before production deployment.
Integrate AI evaluation tests into CI/CD and release-management pipelines.
Establish release gates based on defined quality and compliance thresholds.
Translate employee support policies, procedures, and regulatory requirements into executable evaluation rules.
Build automated checks for prohibited content, sensitive data handling, required disclosures, and escalation requirements.
Ensure AI responses comply with applicable HR, privacy, security, legal, and corporate policies.
Maintain audit trails for evaluations, overrides, approvals, and corrective actions.
Support compliance reviews, audits, and evidence collection.
Implement access controls and data-retention standards for evaluation data.
Design reviewer interfaces and queues for human evaluation within Salesforce.
Enable quality analysts and subject-matter experts to score, annotate, and classify responses.
Support blind reviews, consensus scoring, adjudication, and reviewer calibration.
Create task-routing logic based on risk, business domain, language, and evaluator expertise.
Capture structured reviewer feedback for model and process improvement.
Monitor reviewer agreement and evaluation consistency.
Build dashboards and reports for AI quality and operational performance.
Track metrics such as response accuracy rate, hallucination rate, policy compliance rate, regression failure rate, human escalation rate, reviewer agreement, employee satisfaction, and resolution effectiveness.
Provide drill-down capabilities by model, release, use case, interaction type, and policy category.
Develop alerts for quality threshold breaches and emerging failure patterns.
Provide stakeholders with actionable insights and remediation recommendations.
Integrate Salesforce with AI platforms, large language models, knowledge systems, data warehouses, and analytics tools.
Develop secure REST, event-driven, batch, and middleware-based integrations.
Ingest conversation logs, model Clientdata, retrieval context, evaluation scores, and reviewer feedback.
Ensure data quality, lineage, traceability, and reconciliation.
Optimize data models and processing pipelines for high-volume evaluation workloads.
Protect employee and enterprise data through appropriate security and privacy controls.
Develop unit, integration, regression, performance, and security tests.
Support user acceptance testing, release readiness, deployment, and hypercare.
Troubleshoot Salesforce, integration, data, workflow, and evaluation-processing issues.
Lead root-cause analysis for production incidents and quality failures.
Implement preventive controls and technical improvements.
Maintain operational documentation, runbooks, design specifications, and support procedures.
Partner with AI Engineering, Product Management, Data Science, Employee Experience, HR, Compliance, Legal, Security, and Quality teams.
Translate business quality expectations into technical requirements and evaluation criteria.
Participate in architecture reviews, sprint planning, backlog refinement, and release governance.
Communicate risks, quality trends, technical decisions, and remediation plans.
Provide technical leadership and knowledge transfer to engineering and support teams.