Information Extraction Engineer

Red Cell Partners

2h•$135,000 - $170,000•Remote

About The Position

The posting identifies the role as Information Extraction Engineer, but the role description itself describes a QA Engineer responsible for quality for an AI-enabled decision-support system. This is the first dedicated quality engineering hire, positioned on the product side and reporting through the Product Lead. The engineer will translate product-defined acceptance criteria into executable tests, build automated coverage, and independently produce evidence that delivered work meets specified requirements. The role works closely with engineering and platform teams while maintaining independence in testing and reporting. The position owns the test strategy, automation, measurement and evidence, and release governance. It includes functional, integration, performance, and load testing; CI/CD release gates; labeled evaluation sets and sampled audits; and release records that document versions, results, and rollback criteria. The role is fully remote with occasional travel of up to 25% to DEFCON AI headquarters, customer sites, and vendor facilities. US citizenship and an active US Secret clearance are required; the work takes place in a controlled government cloud environment and requires favorable investigation and CAC eligibility from the start.

Requirements

At least 5 years in software quality engineering, test automation, or software engineering with substantial dedicated test ownership.
A clear point of view on testing practices and the judgment to establish and uphold standards.
Strong coding ability in a test automation language and framework, such as Python with pytest or JavaScript/TypeScript with Playwright, Cypress, or Selenium; ability to read and reason about application code. Python is used for most of the stack.
Routine use of AI-assisted development, with judgment about its value and verification of its output.
Demonstrated experience building automated test suites and integrating them into CI/CD as release gates.
Experience owning a test plan through review and acceptance.
Experience testing against requirements rather than implementation, including working directly with product owners or end users to define correctness.
Experience with performance and load testing, including understanding what results demonstrate and do not demonstrate.
Working statistical literacy, including sampling design and understanding the confidence a measurement supports.
Experience measuring system misses, not only correct outcomes.
Willingness to report unfavorable results and keep measurements independent of the people whose work is evaluated.
Ability to gate work owned by people who do not report to the role and to earn credibility with engineers.
Ability to produce clear, audit-ready written documentation.
Willingness to hold a release and explain the reason to stakeholders.
US citizenship.
Active US Secret clearance; favorable investigation and CAC eligibility are required from the start.
Willingness to travel up to 25% to customer sites, DEFCON AI headquarters, and vendor facilities as required.

Nice To Haves

Active Top Secret clearance.
Experience in government DevSecOps, RMF, or ATO environments, including producing test evidence for formal accreditation or audit.
Experience with contract and API testing, test data management, and building extensible test infrastructure.
Experience using language models to generate and maintain test coverage, with practices that keep generated tests meaningful.
Experience with JMeter, k6, Locust, or comparable performance testing tools.
Experience validating machine-learning or model output, including groundedness and citation checking, and analyzing user disagreement with system recommendations.
Experience in domains where incorrect results carry significant cost and testing must satisfy an external reviewer.
Experience with model cards, NIST AI RMF, or comparable responsible AI practices.
Experience with Section 508 and WCAG verification.

Responsibilities

Own and maintain the formal system test plan covering functional, integration, performance, and load testing.
Translate product-defined acceptance criteria into executable tests and review criteria with product before development to identify ambiguity or unverifiable requirements.
Hold engineering, platform, and partner teams accountable to acceptance criteria, defining the evidence required before changes are considered complete.
Work closely with engineering while retaining independence to report test results.
Plan and coordinate customer user acceptance testing and track findings through resolution.
Build and maintain automated test suites, integrate them into the CI/CD pipeline as release gates, and establish and report against an automated coverage standard.
Build performance and load tests that validate the system under expected user load and maintain regression coverage across releases.
Build and maintain labeled evaluation sets in collaboration with subject-matter experts.
Design and run sampled audits to confirm automated decisions and measure and report system misses as well as correct results.
Measure and report how often users override the system.
Document how committed measures are computed, what data they use, and what evidence accompanies a release, independently of model builders.
Maintain version inventories across builds, models, configurations, and thresholds.
Produce test reports, evaluation records, and rollback criteria for each release.
Work with platform engineering to confirm that test environments adequately match production.
Prevent release without the evidence required by acceptance criteria.

Benefits

Competitive salary, bonus, and equity package.
100% employer-paid comprehensive medical, dental, and vision insurance for the employee and family.
Unlimited PTO with manager or management approval.
Flexible work environment where employees manage their work day.
14 weeks of fully paid parental leave at normal pay.
Career track opportunity with potential for rapid advancement as the firm grows.
Professional development and continued learning opportunities.
Optional 401(k), FSA, and equity incentives.
Mental health benefits through Tara Mind.
Cost-effective GLP-1 solutions through Crux.