Sr AI Platform Engineer

Appex Innovation • Frisco, TX

2d • Onsite

About The Position

We are seeking a Sr AI Platform Engineer to join our team. This role involves developing and operating data infrastructure for AI and ML use cases, including Retrieval-Augmented Generation (RAG), agent tooling, and feature serving. You will work with technologies such as Kafka, Redis, vector databases, and Azure Databricks. The position requires strong API development skills, experience with batch and real-time data pipelines, and a solid understanding of Delta Lake internals. You will also be responsible for CI/CD implementation and collaborating with senior architects, product managers, and stakeholders.

Requirements

• Production Python with FastAPI or comparable for API development.
• Experience with versioned REST APIs, contracts, and governance.
• Experience with Kafka or comparable streaming technologies for batch and real-time data pipelines.
• Experience with CDC or incremental batch processing.
• Production Redis or Valkey experience, including cache invalidation, TTL strategies, and hot-path serving.
• Experience with vector databases and knowledge graphs (e.g., Pinecone, Weaviate, pgvector, Neo4j).
• Hands-on experience building data infrastructure for AI and ML use cases (RAG, agent tooling, feature serving).
• Hands-on production experience with Azure Databricks, Delta Lake, and Unity Catalog.
• Understanding of Delta Lake internals: transaction log, time travel, and Change Data Feed (CDF).
• Proficiency in SQL and data modeling, understanding point-lookup vs analytical query patterns.
• Experience with CI/CD using GitLab or GitHub Actions, including automated tests for data pipelines.
• Strong communication skills to work directly with senior architects, product managers, and domain stakeholders.

Nice To Haves

• Experience with embedded analytical engines like DuckDB or comparable.
• Production experience with Microsoft Fabric, OneLake, or Power BI Semantic Models.
• Experience defining and operating SLAs and SLOs for data products or APIs.
• Familiarity with MCP-style tooling for data access for AI agents.
• Prior experience with enterprise-scale data serving infrastructure at large enterprises.

Responsibilities

• Develop and operate data infrastructure for AI and ML use cases (RAG, agent tooling, feature serving).
• Build and operate end-to-end batch and real-time data pipelines using Kafka or comparable streaming technologies, and CDC or incremental batch.
• Implement and manage caching and key-value serving solutions using production Redis or Valkey, including cache invalidation, TTL strategies, and hot-path serving.
• Work with vector databases and knowledge graphs (e.g., Pinecone, Weaviate, pgvector, Neo4j) for embeddings and retrieval patterns.
• Utilize Azure Databricks, Delta Lake, and Unity Catalog in a production environment.
• Apply knowledge of Delta Lake internals, including the transaction log, time travel, and Change Data Feed (CDF).
• Develop and maintain versioned REST APIs with FastAPI or comparable, focusing on contracts and governance.
• Implement CI/CD pipelines using GitLab or GitHub Actions with automated tests for data pipelines.
• Communicate effectively with senior architects, product managers, and domain stakeholders.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed