Automation Program Manager, Legal and Compliance Operations

Meta•Austin, TX

9d•$97,000 - $139,000

About The Position

Legal and Compliance Operations is seeking an Automation Program Manager to help automate high-volume review across Legal and Compliance workflows. These workflows combine LLM-based decisioning, rules-based logic, and signals derived from reporter history, content matching, and behavioral patterns. Because automation handles a substantial share of enforcement volume, its quality affects legal risk and the experience of rights holders, regulators, and others who depend on its decisions. The Automation Program Manager owns the measured quality of assigned automations. The role involves improving automations through iteration on prompts, rules, and signal thresholds, as well as diagnosing and resolving declines in reliability, correctness, precision, or recall. The successful candidate is rigorous about measurement, comfortable working across different automation types, and able to distinguish genuine degradation from changes in incoming population or volume mix.

Requirements

At least 2 years of relevant experience in data annotation, labeling, quality assurance, or content review operations, including experience close to automated decision output.
Experience with guideline accuracy and inter-rater agreement, including diagnosing why qualified reviewers disagree.
Working knowledge of Python and SQL, including experience pulling datasets, computing precision and recall, and segmenting results to locate concentrated losses.
Experience reading and reasoning about conditional logic to trace how a decision was reached.
Experience working with large operational datasets in a notebook environment.
An investigative approach to metric movement, focused on identifying causes rather than settling for a plausible explanation.
Ability to communicate results clearly, including reporting when a metric has not moved or a regression remains unexplained.

Nice To Haves

Experience in adversarial or fraud-adjacent domains where the reviewed population actively shifts.
Experience with rules engines, decision tables, routing logic, or classifier thresholds.
Experience coordinating vendor or cross-functional reviewer pools.
Demonstrated ongoing AI skill development, such as prompt or context engineering and agent orchestration, and staying current with emerging AI technologies.
Demonstrated ability to integrate AI tools to optimize or redesign workflows and drive measurable impact, such as efficiency gains or quality improvements.
Experience or expertise in legal and compliance operations, or trust and safety.
Experience iterating on prompts with measured before-and-after results.
Experience adhering to and implementing responsible, ethical AI practices, such as risk assessment, bias mitigation, and quality and accuracy reviews.

Responsibilities

Improve reliability, correctness, precision, and recall of assigned automations across Legal and Compliance workflows.
Iterate on LLM components, including prompts, task decomposition, routing, and abstention and fallback behavior.
Diagnose and improve rules-based components, including coverage, ordering and precedence, rule conflicts, rules firing on the wrong population, and rules affected by policy or schema changes.
Tune signal-based components, including thresholds, signal freshness and availability, and the precision cost of upstream signal drift.
Investigate regressions to identify whether causes relate to models, rules, signals, upstream data, policy changes, or shifts in the incoming population; distinguish real degradation from volume mix shift.
Drive regressions to a measured recovery that holds through a subsequent measurement period, and document what degraded, why it was not caught earlier, and what detection would have caught it.
Build and maintain offline evaluation and gold-standard datasets, coordinating expert-review sampling with reviewers who hold the ground truth.
Build regression detection that surfaces degradation before the next reporting cycle and close measurement coverage gaps for live automations.
Build scripts, notebooks, and internal tools so program managers and operations leads can run measurement independently.
Document what was tried and what did or did not move metrics so subsequent workflows can build on prior work.