Software Development Engineer in Test

Sofico • Dallas, TX

14h • Remote

About The Position

SOFICO Intelligence is hiring an SDET for its AI team to own quality for non-deterministic, multi-agent systems. The role focuses on making the company’s autonomous AI platform trustworthy by designing and building an evaluation harness that supports AI governance and confidence in releases. This includes testing both the software and the quality of the AI’s outputs in a production system.

Approximately 85% of the role is focused on agentic AI evaluation and testing, including grounding and faithfulness, multi-agent orchestration, security, guard rails, performance, and automated tests for the platform’s Go microservices, data/SQL tools, and APIs. The remaining approximately 15% involves strengthening automated testing across the broader SOFICO Intelligence platform and, where relevant, the wider SOFICO engineering environment. The SDET partners with AI and data engineers to define quality bars and acceptance criteria and contributes to the supervisor review-and-retry loop.

The role works with a production multi-agent AI platform built on Go microservices, LLM orchestration, and fleet data. Testing and development tools and technologies include Go, Python, JavaScript, Playwright, REST Assured, PyTest, JMeter, PostgreSQL, Redis, RabbitMQ, Databricks, Docker, Kubernetes, REST and WebSocket/streaming APIs, Postman, Jira, TestRail, Git, and CI/CD pipelines. The team operates remotely.

Requirements

• Proven experience spanning both software development and quality assurance or quality engineering.
• Strong coding skills and confidence programming in at least one language; the team primarily works in Go and Python.
• Hands-on experience with open-source automated testing tools and frameworks.
• Solid understanding of RESTful APIs and ability to make HTTP requests from the command line or a tool such as Postman.
• Comfort working from the command line and reasoning about distributed, concurrent systems.
• Hands-on experience testing, evaluating, or building with LLM-based or agentic systems, including direct work with model APIs, tool-calling, or multi-agent workflows and an understanding of how they fail.
• Demonstrated ability to evaluate non-deterministic AI output against a measurable quality bar, such as building or running evaluation harnesses, scoring grounding and faithfulness, detecting quality regressions, or red-teaming for prompt injection and jailbreaks.
• Working knowledge of LLM quality concepts, including prompting, context windows and token limits, hallucination and grounding, and sources of non-determinism in model output.
• A strong desire to ensure trustworthy, bug-free software ships every day.
• Fluent English for interaction with international teams.
• Comfort with ambiguity, collaborative problem-solving, prioritizing work independently, giving and receiving constructive criticism, and applying a structured, analytical approach to scalable evaluation systems.

Nice To Haves

• Depth in AI evaluation methodology, such as designing golden datasets, LLM-as-judge scoring, supervisor/critic patterns, or automated regression detection for agentic pipelines.
• Experience with Go, data warehouses or SQL analytics such as Databricks, event-driven systems, or real-time streaming.
• Familiarity with enterprise back-office or ERP environments, including Java back ends and relational databases such as Oracle, MSSQL, or PostgreSQL.
• Exposure to automotive finance, leasing, fleet, or mobility.

Responsibilities

• Design and build an evaluation harness for agentic AI, including golden datasets, automated scoring, faithfulness and grounding checks, and regression detection across prompt, model, and agent changes.
• Verify that AI insights and numbers trace back to real data, and build automated checks to prevent hallucinations from reaching production.
• Test multi-agent orchestration end to end, including tool-calling loops, supervisor-specialist pipelines, autonomous background jobs, and synthesis of findings into reports.
• Red-team the system for prompt injection, jailbreaks, and tool misuse, and prove multi-tenant data isolation.
• Validate AI guard rails, including iteration limits, agent-recursion depth, and token and cost ceilings.
• Pressure-test performance and scale, including real-time streaming, high-concurrency conversations, latency, and cost under realistic and adversarial load.
• Build automated tests for Go microservices, data/SQL tools, and APIs, and integrate them into CI/CD for continuous evaluation.
• Partner with AI and data engineers to define quality bars and acceptance criteria, and help evolve the supervisor review-and-retry loop.
• Strengthen automated testing across the broader SOFICO Intelligence platform and, where relevant, the wider SOFICO engineering environment.
• Help raise quality engineering standards across the combined group.

Benefits

• Health care, including medical, dental, and vision coverage for US employees.
• Flexible PTO policy.
• Conferences, advanced training, and certifications to support professional growth.

What This Job Offers

• Job Type: Full-time
• Education Level: No Education Listed