About The Position

Avra is hiring a Member of Technical Staff for its Knowledge Platform. The role owns how data is acquired and the shape it is in when it arrives at Avra. This covers two areas: building crawlers, scrapers, and browser automation to collect data from sources without APIs and maintain collection as those sources change; and building the data platform—including streams, pipelines, and a lakehouse—that turns raw captures and large relational datasets into versioned tables used by research, training, and production predictions. This is a hands-on role for an engineer with experience both building collection systems against difficult sources and moving large volumes of data through production pipelines. The position involves operating the systems built, maintaining data quality and lineage, and collecting responsibly in partnership with Security and Legal.

Requirements

  • At least 5 years of relevant engineering experience at technology- and product-driven companies.
  • Production experience in at least one of web scraping, large-scale crawlers, or browser RPA, including building and operating collection against real sources rather than only consuming APIs.
  • Data engineering experience with large volumes of data using Spark, dbt, and SQL, and columnar and table formats such as Parquet and Iceberg.
  • Production experience with messaging and streaming systems, such as Kafka and/or Pub/Sub.
  • Production coding experience in Python and JavaScript, including pipelines, crawlers, and browser automation.
  • Hands-on Kubernetes experience, including packaging with Helm.
  • Experience with infrastructure as code using Terraform and CI/CD pipelines with reliable rollback.
  • Willingness to operate built systems, including incident response when sources or pipelines break.
  • Daily use of coding agents and a considered view on using them safely.

Nice To Haves

  • Experience collecting from difficult sources involving heavy JavaScript rendering, logged-in sessions, rate limits, or proxy management, using tools such as Playwright, Puppeteer, or Scrapy.
  • Experience with workflow orchestration using Airflow, Dagster, or Temporal.
  • Experience with entity resolution and relational or graph data modeling across many sources.
  • Experience with ML datasets, including point-in-time correctness, reproducible training sets, and evaluation splits.
  • Experience with GCP and AWS data services, including GKE, BigQuery, Dataproc, and EKS.
  • Experience with observability using OpenTelemetry.
  • Experience in financial services or regulated environments, including LGPD and ISO 27001.

Responsibilities

  • Build and operate crawlers, scrapers, and browser RPA to collect data from websites and portals, including scheduling, retries, session handling, and change detection.
  • Keep collection reliable as sources change by detecting layout and access changes, tracking coverage and freshness per source, and repairing collection quickly.
  • Operate event-driven ingestion on Kafka and Pub/Sub, including ordering, replays, dead-letter queues, backpressure, and idempotent consumers.
  • Build batch and incremental pipelines in Spark, dbt, and SQL to transform raw captures into clean, typed, relational tables.
  • Own the Parquet and Iceberg lakehouse, including schema evolution, partitioning, compaction, and snapshots that make training sets reproducible.
  • Establish data quality and lineage standards, including contracts, tests, and traceability from predictions back to source records.
  • Operate systems on Kubernetes, package them with Helm, provision infrastructure with Terraform, and ship through CI/CD with reliable rollback.
  • Collect data responsibly in partnership with Security and Legal, including attention to rate limits, terms of use, LGPD, and personal-data handling.
  • Respond to incidents when a source or pipeline breaks.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service