Software Test Engineer

Bigbear.ai • UNAVAILABLE, UNAVAILABLE

1d

About The Position

Ask Sage, a BigBear.ai company, is hiring a Software Engineer Test Engineer to strengthen automated validation across our AI platform. We have a growing backlog of backend and platform work that requires deeper regression coverage, stronger system-level validation, and more scalable test execution. This is not a manual QA role, and it is not a DevSec role. We are looking for a software engineer who specializes in quality engineering: someone who can understand backend code, reason through distributed system behavior, write high-quality automated tests, and improve the infrastructure that gives engineers reliable release signal. You will partner with engineers to define acceptance tests that prove a fix works and fail reliably if the fix is reverted, and your work will directly improve release velocity, platform reliability, and confidence in complex AI workflows.

Requirements

• Strong software engineering background with deep experience in automated test development.
• 2-5 years of Software Testing experience
• Experience testing backend services, APIs, distributed systems, and database-backed applications.
• Experience designing smoke, unit, integration, and end-to-end testing strategies.
• Strong judgment around edge cases, non-happy paths, adversarial inputs, regression risk, and failure isolation.
• Experience improving CI test reliability, execution time, parallelization, test isolation, and failure observability.
• Familiarity with testing Defense in Depth behavior: validating that multiple layers of checks work together, without this being a dedicated DevSec role.
• Strong troubleshooting, analytical, and communication skills.
• Ability to work independently and as part of a team.
• Ability to obtain a Secret clearance.

Nice To Haves

• Experience writing E2E tests for AI platforms, LLM applications, model gateways, agents, MCP tools, or tool-using systems.
• Experience designing AI E2E tests that control token usage, external provider calls, latency, and cost.
• Experience testing workflow automation systems, agentic harnesses, or no-code/low-code automation platforms (e.g., Power Automate, Zapier, Make, n8n).
• Familiarity with flagship Generative AI provider APIs (Google VertexAI, AWS Bedrock, Microsoft Azure OpenAI) and models (OpenAI GPT, Anthropic Claude, Google Gemini).
• Experience with CI/CD pipelines (e.g., GitHub Actions), Docker, Kubernetes, and observability/monitoring tooling.
• Experience with PostgreSQL and SQL for validating stored data.
• Knowledge of government compliance frameworks (FedRAMP, NIST AI RMF, CMMC 2.0).

Responsibilities

• Develop and maintain smoke tests, unit tests, integration tests, and end-to-end tests across backend and platform workflows.
• Validate backend changes for new PRs through automated test coverage, direct API testing, and user-facing workflow verification where appropriate.
• Exercise backend behavior through the UI when useful, without owning visual design, frontend UX validation, or manual UI regression testing.
• Build regression coverage for edge cases, malformed inputs, authorization boundaries, concurrency issues, failure modes, and other non-happy-path scenarios.
• Improve the scalability, determinism, execution performance, maintainability, and diagnostic quality of the test suite.
• Strengthen test fixtures, mocks, test data management, failure analysis, and CI feedback loops.
• Design end-to-end tests for AI platform workflows while minimizing unnecessary token usage, external provider calls, latency, and test cost.
• Apply AI-assisted development and analysis tools to accelerate test design, test generation, triage, and maintenance while preserving reliability, reviewability, performance, and deterministic validation standards.
• Validate backend APIs, authentication flows, billing and token behavior, model routing, AI workflow execution, file parsing, MCP/tool execution, and passthrough APIs.
• Validate AI platform E2E paths involving prompts, model responses, streaming behavior, tool calls, agents, workflow orchestration, and provider-facing API compatibility.
• Validate agentic harnesses, MCP integrations, and workflow automation systems, including concepts common to no-code and low-code workflow builders such as Power Automate, Zapier, Make, n8n, and similar platforms.
• Validate security-sensitive product behavior such as user isolation, permission boundaries, validation, sanitization, rate limits, replay prevention, safe error handling, and layered control behavior.
• Build and maintain test infrastructure that must scale with a growing platform, expanding product surface area, and active engineering team.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Education Level: No Education Listed