Staff Applied AI Engineer, Agentic Systems

Beacon AI•San Carlos, CA

5d•$224,000 - $260,000•Hybrid

About The Position

At Beacon AI, we're building AI-powered systems that help aviation professionals navigate complex, high-stakes operational decisions. This role sits at the intersection of agentic AI, information retrieval, and production software engineering. We build systems that reason over diverse sources of information, use tools to investigate problems, and produce reliable, explainable results. We're looking for engineers who can think critically about how these systems should work, design their underlying information and orchestration architectures, and take them from experimental prototypes to dependable production services.

Requirements

Strong software engineering fundamentals. You write maintainable production code, reason about system architecture, and understand testing, debugging, and operational reliability.
Substantive applied AI experience. You've worked on AI systems beyond basic model API integrations and can explain their architecture, tradeoffs, limitations, and observed behavior.
Information modeling instincts. You can understand a new domain, identify its important entities and relationships, and design software abstractions that reflect how its information is actually used.
Experimental discipline. You test assumptions, measure results, investigate failure modes, and use evidence to guide technical decisions.
Independent technical judgment. You can evaluate alternative approaches, recognize unnecessary complexity, and make appropriate tradeoffs.
Clear communication. You can discuss technical decisions with both technical and non-technical stakeholders and domain experts.
Due to U.S. export control regulations, we can only hire U.S. Persons (U.S. citizens, Green Card holders, lawful permanent residents, or individuals granted asylum or refugee status). We are unable to provide visa sponsorship or support visa transfers. All work must be performed in the United States.

Nice To Haves

Substantial experience in agentic systems and context engineering: Agent orchestration patterns, planning and execution, tool-use architectures, persistent and transient memory, and context management.
Substantial experience in information retrieval and knowledge representation: Semantic and structured retrieval, information extraction, document understanding, entity and relationship modeling, and knowledge-driven tool design.
Substantial experience in AI evaluation and reliability: Evaluation methodology, benchmark and dataset construction, human and model-based assessment, statistical analysis, and systematic failure characterization.
Substantial experience in model adaptation and inference optimization: Fine-tuning, model selection, inference serving, quantization, batching, and performance optimization.
Experience with Python, LangGraph, BAML, AWS Bedrock, PostgreSQL/pgvector, and event-driven AWS infrastructure.
Aviation, aerospace, or another technically complex, safety-critical domain.
Computational linguistics, NLP, information science, or knowledge representation.
Designing systems that reason over technical documentation, regulations, or heterogeneous operational data.
Developing or operating distributed, event-driven, and containerized services.
Building AI systems in which traceability, evidence, and human oversight are important.

Responsibilities

Design and build agentic systems, including agent architectures involving tool use, planning, orchestration, state management, and memory.
Develop effective context engineering strategies, balancing relevant information, model capabilities, latency, and cost.
Make thoughtful architectural decisions about when to use autonomous agents, explicit workflows, conventional algorithms, or combinations thereof.
Design tool interfaces and structured model interactions that support reliable, inspectable behavior.
Turn complex information into usable knowledge by developing an understanding of complex document collections and data sources.
Translate domain knowledge into effective retrieval systems, structured representations, and agent-accessible tools.
Apply information retrieval, information extraction, and knowledge modeling techniques to heterogeneous structured and unstructured data.
Design strategies for handling temporal relevance, provenance, conflicting evidence, and incomplete information.
Evaluate and improve AI behavior by developing rigorous evaluations for AI-powered systems, including representative datasets, human annotations, automated assessments, and targeted failure analysis.
Investigate system-level failures across retrieval, context construction, reasoning, tool execution, and final outputs.
Experiment with alternative architectures, models, and prompting strategies, using empirical results to guide decisions.
Help establish engineering practices for measuring and improving the reliability of agentic systems.
Build systems that work in production, developing reliable services and asynchronous workflows with attention to concurrency, fault tolerance, observability, and maintainability.
Integrate model inference, retrieval systems, and external data services into production applications.
Build and operate containerized workloads and cloud infrastructure.
Collaborate with engineers and domain experts to translate complex operational requirements into working systems.