AI Specialist

Knox Systems•New York, NY

1d•Hybrid

About The Position

The AI Specialist will play a critical role in designing, building, training, and continuously improving KnoxAI, Knox’s proprietary AI capabilities that support secure cloud operations, compliance automation, and mission-critical decision-making. This role is not limited to model training or experimentation. You will be responsible for shaping how KnoxAI evolves alongside the business, ensuring it is accurate, explainable, secure, and aligned with real customer and operational needs. You’ll work closely with Engineering, Product, Security, Compliance, and Operations to translate complex requirements into production-ready AI systems. This is a hands-on, high-ownership role suited for an exceptional AI practitioner - someone who is both deeply technical and comfortable operating in ambiguous, high-stakes environments. Knox does not build novelty AI. We build AI that must work, scale, and stand up to scrutiny.

Requirements

Strong foundation in data science, machine learning, and applied AI, with demonstrated experience building and operating models used in real production systems
Advanced proficiency in Python and modern ML / AI frameworks, with the ability to move comfortably between experimentation and production code
Working proficiency in Node.js and/or Bun.js, with experience integrating AI and ML systems into production application backends and services
Hands-on experience with data-centric AI practices, including dataset design, curation, labeling strategies, versioning, and managing data quality over time
Proven ability to train, fine-tune, and evaluate models using rigorous validation approaches, with a clear understanding of tradeoffs between accuracy, precision, recall, and operational risk
Experience designing and applying custom evaluation metrics aligned to real-world outcomes, including managing false positives and false negatives in high-stakes systems
Experience operationalizing AI models in production environments, including deployment, monitoring, performance tracking, drift detection, retraining workflows, and rollback strategies
Familiarity with MLOps practices and tooling, including model versioning, CI/CD for ML, and lifecycle management
Ability to design explainable and auditable AI systems, with experience documenting model behavior, assumptions, limitations, and decision logic for internal and external stakeholders
Strong systems-level thinking, with the ability to reason across data, models, infrastructure, security, and users to build durable, maintainable AI solutions
Proven ability to operate independently, take ownership, and drive complex work forward in fast-moving, high-accountability environments
Strong communication skills, including the ability to clearly explain complex AI concepts and tradeoffs to non-technical audiences
Due to the nature of our work with federal government clients and compliance with applicable regulations, this position requires U.S. citizenship. Candidates must be able to provide documentation verifying U.S. citizenship status as part of the background check process. Any offer of employment is contingent upon the successful completion of all required pre-employment screenings, including a background check, in accordance with applicable laws and government contract requirements.

Nice To Haves

Experience working with LLMs, retrieval-augmented generation (RAG), or applied AI systems
Experience operating in regulated, security-sensitive, or compliance-driven environments
Exposure to cloud infrastructure (AWS, Azure, or GCP) and secure system architectures
Background in government, federal contracting, cybersecurity, or compliance-adjacent domains
Strong written and verbal communication skills, especially the ability to explain complex AI concepts to non-experts

Responsibilities

Own the design, development, training, and ongoing refinement of KnoxAI models and systems, ensuring they meet evolving business, security, and compliance requirements
Develop and maintain high-quality training pipelines, including dataset selection, labeling strategies, evaluation frameworks, and continuous feedback loops
Improve model performance, reliability, explainability, and robustness through experimentation, tuning, and systematic evaluation
Partner closely with Product and Engineering to translate real-world Knox use cases into production AI capabilities, not prototypes
Collaborate with Security and Compliance teams to ensure AI systems align with federal requirements, audit-ability expectations, and risk management standards
Implement monitoring and retraining strategies to detect drift, performance degradation, or emerging risks over time
Contribute to architectural decisions around AI infrastructure, deployment, and MLOps in secure, regulated environments
Document model behavior, assumptions, limitations, and decision logic to support internal understanding and external scrutiny
Stay current on advances in AI, machine learning, and data science, selectively applying new techniques where they materially improve KnoxAI
Remain hands-on while helping establish repeatable, scalable AI development practices as Knox grows