QA Cyber

ASM Research•San Antonio, TX

About The Position

The QA Cyber evaluates the quality, accuracy, consistency, and defensibility of app review decisions. While App Review reviewers assess submitted applications for functionality, policy compliance, privacy, safety, and platform requirements, the QA Analyst evaluates the quality of those reviews — ensuring reviewers apply the correct policies and review criteria, perform sufficient testing or evidence review, document their findings clearly, and reach decisions supported by the facts of the case. The QA Analyst maintains a high and consistent review standard by identifying reviewer errors, missed risks, unsupported decisions, documentation gaps, policy interpretation issues, and recurring quality trends, and requires strong analytical judgment, policy fluency, attention to detail, technical curiosity, and the ability to distinguish reviewer error from ambiguity in policy, process, tooling, or available evidence.

Requirements

Strong analytical judgment — able to reconstruct the logic of a review and determine what the app does, what should have been evaluated, which policies apply, what evidence matters, and whether the reviewer's conclusion was justified.
Policy fluency — able to apply app review policies and platform requirements consistently across varied app types, workflows, metadata, tools, permissions, and user-facing claims, and to recognize edge cases, exceptions, and appropriate escalation paths.
Evidence discipline — able to ground findings in the case record without unsupported assumptions or speculative risk claims.
Working knowledge of QA rubric application, distinguishing critical, major, minor, process, documentation, and coaching-only defects.
Technical aptitude with web applications, APIs, integrations, MCP servers, OAuth, authentication, and test credentials sufficient to assess whether technical testing was reasonable and sufficient (software engineering background not required).
Ability to identify missed or mishandled risks involving privacy, safety, compliance, deception, sensitive data, regulated activities, user trust, and ecosystem integrity.
Strong written communication skills — able to produce concise, objective, actionable QA feedback that cites evidence, maps to policy/rubric, classifies severity, and explains outcome impact.
Calibration and consistency skills — able to recognize inconsistent treatment of similar cases and help align reviewers/QA peers around shared standards.
Root-cause thinking — able to look beyond individual errors to identify training gaps, unclear guidance, process ambiguity, tooling limitations, or emerging app behaviors.
Sound judgment and discretion handling confidential app submissions, test credentials, internal review guidance, reviewer performance data, and sensitive case evidence.

Nice To Haves

Prior experience in app review or marketplace review.
Background in trust and safety quality assurance, policy operations, or content moderation QA.
Experience in risk, compliance, or integrity operations.
Software testing or technical QA experience.
Technical support quality experience.
Privacy or data-handling review experience.
Operational excellence or quality auditing background.

Responsibilities

Audit completed app review cases against established QA standards and scoring rubrics.
Assess whether the reviewer selected and applied the correct review checks, policies, and escalation paths.
Determine whether the reviewer's final decision was accurate, complete, and supported by available evidence.
Classify quality issues by severity, including decision-impacting errors, process errors, documentation gaps, and coaching opportunities.
Identify both under-enforcement (missed policy or safety issues) and over-enforcement (unsupported rejections or unnecessary escalations).
Review submitted app metadata, developer-provided materials, reviewer notes, screenshots, testing evidence, tool information, policies, privacy disclosures, and relevant case context.
Determine whether the reviewer gathered and documented enough evidence to support their conclusions.
Identify missing, contradictory, irrelevant, or insufficient evidence.
Evaluate whether another reviewer could reasonably understand and reproduce the original decision based on the documented record.
Perform targeted validation where required by the QA workflow, without assuming every case requires a full second review.
Validate whether reviewers correctly interpreted and applied app review policies and platform requirements.
Assess reviewer handling of privacy, data access, user safety, regulated or high-impact domains, impersonation, misrepresentation, third-party access, and other trust-related risks.
Ensure reviewers evaluate the app's actual functionality and submitted evidence rather than relying on unsupported assumptions or hypothetical misuse.
Identify cases where ambiguity should have led to escalation rather than approval or rejection.
Surface unclear policy guidance or recurring interpretation gaps.
Evaluate whether reviewers sufficiently assessed the app's core workflows, tool behavior, authentication requirements, MCP server connectivity, OAuth flows, test account access, and developer-provided testing instructions.
Determine whether technical blockers were reasonably investigated before being treated as review-limiting.
Identify cases where an app was approved despite meaningful technical failures or incomplete testing.
Distinguish between true app issues, reviewer execution errors, incomplete submission materials, and tooling limitations.
Support internal and client-facing calibration sessions to align on review standards, QA scoring, defect severity, and policy interpretation.
Present QA findings, trends, and examples clearly to client stakeholders during calibration discussions.
Explain the rationale behind QA decisions in a balanced, evidence-based way, including where cases involve ambiguity or reasonable judgment.
Capture client feedback, alignment decisions, and follow-up actions from calibration sessions and incorporate them into QA guidance or reviewer coaching where appropriate.
Track recurring review errors, quality trends, and high-risk issue areas.
Identify reviewer knowledge gaps, training needs, unclear guidance, tooling limitations, and process breakdowns.
Produce QA summaries, defect analyses, calibration notes, and recommendations for operational improvement.
Partner with App Review Operations and Training teams to improve review quality and consistency over time.