Senior Software Engineer, AI Platform

Kai Cyber, Inc.•San Jose, CA

About The Position

Kai is building an AI-powered cybersecurity platform to help enterprises manage vulnerabilities at scale. The AI team has developed services for analyzing container images, standardizing package data, extracting natural language filters, and assessing package maintenance, all powered by LLMs and intelligent automation. This role is for an experienced software engineer to make these systems scalable. The engineer will be the first dedicated engineering hire on the AI team, working with applied AI scientists and an AI infrastructure engineer to transform code into reliable, well-tested, and maintainable production services. This is a software engineering role focused on code quality, test coverage, CI/CD pipelines, and production reliability of services that interact with LLM APIs, Azure cloud services, and serve critical data to the cybersecurity platform.

Requirements

4+ years of professional software engineering experience with a strong backend focus.
M.S. in Computer Science, AI, Machine Learning, or a related field. PhD preferred.
Deep Python expertise, including understanding of dataclasses vs Pydantic, async/await functionality, and the impact of global variables on testing.
Experience with testing as a core discipline, including building test suites for services with external dependencies, using pytest, mocking, fixtures, and testing code that calls third-party APIs.
FastAPI or equivalent modern Python web framework experience (e.g., Django REST Framework, Flask with production patterns), including designing and maintaining REST APIs.
Azure or equivalent cloud platform experience, including working with managed container services, Kubernetes, managed databases, identity/auth systems, and CI/CD in a cloud environment.
CI/CD pipeline engineering experience, including adding test gates, lint checks, and automated quality enforcement to build pipelines (e.g., Azure DevOps Pipelines, GitHub Actions, or GitLab CI).
Docker and containerization experience, including writing production Dockerfiles, understanding multi-stage builds, and debugging container networking and configuration issues.
Strong code review and collaboration skills, with the ability to raise the engineering bar for team members with less engineering experience.

Nice To Haves

Experience working with LLM provider APIs (Anthropic, OpenAI, Azure OpenAI), including understanding token limits, prompt design, structured output parsing, and retry patterns.
Experience with structured logging (structlog), observability tools (OpenTelemetry, Prometheus, Grafana), or APM platforms.
Exposure to cybersecurity, vulnerability management, or compliance-sensitive environments.
Experience on a small engineering team at a startup, with end-to-end service ownership.
Familiarity with RAG patterns, embedding pipelines, or vector databases.

Responsibilities

Build the test suite from the ground up, designing test infrastructure including unit tests with mocked LLM responses, integration tests against staging environments, and fixtures for fast and reliable testing. Wire this into CI to ensure all code passes tests before shipping.
Harden production services by auditing and fixing security issues, implementing structured logging, and adding health checks, metrics, and traces.
Improve the CI/CD pipeline by adding quality gates to catch issues before they reach production.
Refactor for maintainability by extracting shared patterns into reusable modules, breaking apart oversized classes, and reducing code duplication across services.
Fix dependency management by introducing lock files for reproducible builds, removing unused dependencies, and resolving version inconsistencies across services.
Own the reliability and performance of AI service fleet (Python/FastAPI microservices).
Build out observability, including distributed tracing, latency dashboards, and alerting on error rates and SLA breaches.
Design and implement caching strategies, rate limiting, and circuit breakers for external API calls (Anthropic, Azure ML, package registries).
Collaborate with AI scientists on prompt engineering and output parsing, applying engineering rigor to LLM integration patterns.
Mentor mid-level engineers as the team grows.