Senior AI Application Developer

Signature Aviation•Orlando, FL

About The Position

At Signature Aviation, the Senior AI Application Developer builds AI capabilities for business operations and customer experience, including agentic systems, LLM-powered applications, and intelligent automation on Azure, along with the retrieval, feature, and data pipelines they depend on. The role owns capabilities end to end: the agent or assistant, the data it uses, the pipelines that supply it, and the evaluation used to assess it. The position is applied engineering rather than research; statistical and forecasting models are built by data science, while this role builds systems that put their outputs to work. The role follows a forward-deployed working model. The developer embeds with business units and engineering teams, builds solutions within their environments and release processes, hands over running systems, and contributes reusable lessons and components to a shared accelerator library between deployments.

Requirements

At least 5 years of experience in software, data, or ML engineering, with recent hands-on delivery.
Experience building and deploying production-grade LLM applications or agent-based AI systems, and ability to explain how a system worked, what failed, and how its quality was assessed.
Hands-on experience with Microsoft Foundry, including Azure OpenAI model deployments, the Foundry Models catalog, and Azure AI Content Safety.
Experience with cloud-native application patterns on Azure, including containers on Azure Container Apps or AKS, Azure Functions, CI/CD through Azure DevOps or GitHub Actions, Microsoft Entra ID managed identity, and Key Vault-backed configuration.
Experience with LLM application patterns such as prompt design, RAG, function and tool calling, structured outputs, and streaming, plus at least one orchestration framework: Microsoft Agent Framework, Foundry Agent Service, Semantic Kernel, LangChain/LangGraph, or LlamaIndex.
Demonstrated data engineering experience building or substantially rebuilding ingestion, transformation, embedding, or feature pipelines and maintaining their correctness in production.
Experience with vector or hybrid search using Azure AI Search, Cosmos DB, or pgvector; Pinecone, Weaviate, or Redis Vector are accepted as equivalent technologies.
Systematic experience evaluating model or prompt quality; a self-built lightweight evaluation harness is acceptable, but anecdotal assessment is not.
Strong Python skills and working TypeScript skills.
Experience integrating AI systems with enterprise data and operational platforms.
Experience working directly with users of products or systems built, through embedded delivery, solutions engineering, or stakeholder-facing product work.
Sound judgment regarding secrets, access control, and PII, including knowing when to stop and ask.
Bachelor's degree in Computer Science, Engineering, or a related technical field, or equivalent practical experience.

Nice To Haves

Strong LangGraph and multi-agent orchestration experience.
MCP and enterprise tool integration experience.
Experience with Microsoft Foundry, Azure OpenAI, Azure AI Search, Azure Databricks, or Microsoft Fabric.
Experience with OpenAI, Claude, Gemini, Llama, or other foundation models.
Experience with Langfuse, LangSmith, Ragas, Promptfoo, or Azure AI Evaluation SDK.
MLflow and model lifecycle experience.
Experience with coding agents such as Claude Code, OpenAI Codex, or GitHub Copilot.
Experience in a Forward Deployed Engineer, solutions engineering, consulting, startup, or internal innovation role.
Experience in aviation, logistics, mobility, or other large-scale operational environments.

Responsibilities

Build and deploy production LLM applications, agentic systems, and multi-agent solutions using frameworks such as LangGraph, Microsoft Agent Framework, or Semantic Kernel.
Design stateful agent workflows using tool calling, conditional routing, checkpointing, structured outputs, human-in-the-loop controls, memory, and failure recovery; select an appropriate agentic or deterministic approach.
Integrate agents securely with enterprise systems through APIs, function calling, and MCP, with appropriate authentication, authorization, approval, and audit controls.
Apply context engineering across prompts, retrieved information, conversation state, tool results, business rules, and short- or long-term memory.
Build RAG and retrieval capabilities using technologies such as Azure AI Search, Databricks Vector Search, pgvector, and Cosmos DB, including chunking, embeddings, hybrid search, metadata filtering, and reranking.
Build and maintain ingestion, transformation, embedding, and feature pipelines for AI capabilities using Azure Databricks, Microsoft Fabric, or equivalent platforms.
Develop systematic evaluation using golden datasets, automated scoring, LLM-as-judge, and human review; assess accuracy, groundedness, hallucination, task completion, agent trajectories, safety, latency, and cost.
Instrument model calls, retrieval, agent steps, and tool calls with tracing and observability tools, and support testing for prompt injection, PII leakage, and excessive agent autonomy.
Work as a Forward Deployed Engineer with business and engineering teams, taking use cases from ambiguous problems through experimentation, production deployment, and operational handoff.
Turn deployment lessons and reusable components into shared AI patterns, libraries, and accelerators.
Work with the AI Platform Engineer to follow enterprise standards for security, model access, evaluation, observability, and cost.