Senior AI Engineer - Agentic Systems & Data Pipelines

Collaboration.Ai • Minneapolis, MN

10d • Remote

About The Position

This role involves building the agentic systems and data pipelines that power NetworkOS's AI capabilities. This includes production agent workflows using industry-leading agent SDKs and harnesses, MCP servers, and Agent Skills standards. You will also develop the evaluation and observability layer for measuring LLM quality and create ingestion pipelines to transform diverse data sources into queryable knowledge. This is an execution-focused role where you will commit code weekly, ship agents as product features, and contribute to a roadmap focused on graph and agents for defense, healthcare, and regulated enterprise clients. The goal is to have agents in production, reliable pipelines, and honest evaluations.

Requirements

• 7+ years of professional software engineering experience.
• 3+ years of experience focused on AI/ML or data engineering.
• Production agentic/LLM application experience, including building and operating systems around LLM APIs (Anthropic, OpenAI) for real users (agents, tool-use, orchestrated LLM workflows).
• Data engineering background with experience in robust, scalable pipelines for AI/ML workloads.
• LLM operations experience, including evaluations and observability for production LLM systems (quality, cost, latency).
• Production retrieval experience with vector databases and/or search engines (OpenSearch, Elasticsearch).
• Modern Python stack proficiency, including FastAPI, Pydantic, async/await, and modern dependency management.
• Demonstrated ability to leverage AI-native workflows and tools like Claude Code/Codex to accelerate development.
• Experience with Docker, Kubernetes, and AWS.
• US citizenship required due to DoD contracting (IL4/IL5 environments) and FedRAMP compliance.

Nice To Haves

• Deep experience in the agentic ecosystem, including Agent Skills standards, custom MCP servers, and agent SDKs from major vendors.
• Advanced RAG expertise, such as GraphRAG, agentic RAG, contextual retrieval, and reranking strategies.
• Experience with graph data, including knowledge graphs, graph databases, or graph-based retrieval.
• Experience with model selection and rightsizing for domain-specific use cases, balancing quality, cost, and latency.
• Experience with streaming data (Kafka, Kinesis) for real-time knowledge base updates.
• Research background, open-source contributions, or an advanced degree in ML/IR/NLP.

Responsibilities

• Design, build, and operate agentic workflows (agent SDKs, MCP servers, Agent Skills standards) for AI-driven matching, analysis, and data intelligence.
• Build the evaluation and observability layer using tools like Langfuse, golden datasets, LLM-as-judge patterns, and FinOps-style tracking to ensure measurable quality, cost, and latency for all workflows.
• Develop robust ingestion pipelines for documents, structured data, and external sources into searchable knowledge bases, including quality validation, deduplication, and incremental updates.
• Own and improve retrieval quality through hybrid search (vector, keyword, metadata) using techniques like reranking, query expansion, and contextual compression.
• Build custom MCP tools and Agent Skills to enhance the productivity of the engineering team.
• Collaborate with full-stack engineers on AI integration points.
• Participate in incident response for AI services.
• Contribute code regularly to the team's projects.

Benefits

• Autonomy to shape how AI engineering is built.
• Opportunity to work on production agents, hybrid retrieval, and continuous evaluations.
• A roadmap focused on graph and agents.
• AI-native development environment with agentic coding tools, agent SDKs, MCP servers, and Agent Skills standards.
• Work on impactful projects in defense, healthcare, and regulated industries.
• SOC 2 and NIST compliance, FedRAMP readiness.
• Opportunity for early-stage impact with work visible from week one.
• Work with a small, senior team.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed