AI/ML Lead

ISACA•Remote,

1d•$151,047 - $241,653

About The Position

The AI/ML Lead is a senior, hands-on engineer and the first member of ISACA’s new internal AI implementation team. The team will take on AI engineering work currently delivered by external vendors. Reporting to the VP, Application Engineering, the role works within the platform architecture, standards, and governance set by the Senior Director, AI Architecture, Platforms & Agentic Systems. It builds and hardens AI solutions for ISACA’s members and business, including current delivery work and future initiatives as the AI portfolio grows. The initial focus is ISACA’s personalized member experience: a production RAG and knowledge graph-based recommendation system integrating ISACA’s Body of Knowledge, certification details, workforce skills data, and member profile information. Most of the role involves implementation, including writing production code, building data and retrieval pipelines, and operationalizing engineering practices that keep AI systems reliable. Success is measured by production stability, the share of AI engineering work brought in-house, and the implementation team’s growing ability to deliver AI needs without vendor dependency. The role may require periodic onsite attendance based on business needs and occasional travel for company-sponsored events, meetings, all-hands gatherings, training, or other activities necessary to support the role.

Requirements

Bachelor’s degree in Computer Science, Data Science, Statistics, Engineering, or a related field, or equivalent professional experience.
At least 8 years of professional software, data, or application development experience, including at least 2 years of hands-on experience developing, implementing, or integrating AI, generative AI, LLM-powered, or agentic AI solutions in production environments beyond proof-of-concept work.
Experience delivering LLM-powered or AI solutions in cloud environments, including LLM integration, RAG pipeline architecture, and model evaluation beyond proof-of-concept work; Azure is strongly preferred or equivalent cloud experience.
Proficiency in Python and LLM application frameworks and SDKs; working knowledge of LLM evaluation, RAG pipeline performance tracking, and prompt versioning practices.
Hands-on experience with RAG pipelines, prompt engineering, embedding-based semantic retrieval, vector databases, and LLM orchestration.
Hands-on experience with knowledge graph architectures and ontology design.
Foundational MLOps and LLMOps capability, including CI/CD for ML and LLM pipelines, containerized deployment, model and prompt versioning, monitoring, token cost tracking, and drift detection.
Demonstrated ability to oversee AI vendor work through technical architecture review, acceptance criteria definition, and delivery validation.
Strong stakeholder communication skills, including translating AI architecture decisions, risks, and tradeoffs for non-technical audiences and product and engineering stakeholders.
Experience working in agile delivery environments with cross-functional product, engineering, data, and security teams.
Strong customer and member focus, intellectual adaptability, continuous learning orientation, problem-solving and analytical thinking, and ability to evaluate tradeoffs among innovation, delivery constraints, cost, and risk.
Excellent oral and written communication skills and a collaborative working style that builds alignment without direct authority.
Ability to operate autonomously with evolving requirements and incomplete information.
Commitment to ISACA’s mission, purpose, and values, with cultural sensitivity and awareness appropriate for a global professional membership organization.

Nice To Haves

Master’s degree in Computer Science, Machine Learning, Data Science, or Artificial Intelligence.
10+ years of professional software or data engineering experience, including 3+ years developing, implementing, or integrating AI, generative AI, LLM-powered, or agentic AI solutions; ideally including early-stage or greenfield AI systems delivered at enterprise scale.
Experience contributing to or helping shape an AI or data practice by establishing engineering practices and delivery patterns; experience helping build or scale an engineering team.
Domain literacy in IT governance, cybersecurity risk management, or professional certification ecosystems.
Experience with knowledge graph reasoning or GraphRAG patterns relevant to professional knowledge and competency ontology use cases.
Familiarity with bias and fairness evaluation for AI-driven recommendation systems, including output-level testing for demographic or professional skew.
Experience in a non-profit, professional association, standards body, or other mission-driven organization.
Experience collaborating across hybrid delivery models involving in-house staff, contractors, and external development vendors.
Proficiency with Azure AI platforms or equivalent AWS or GCP services; Azure is the primary cloud platform for this role.
Familiarity with Salesforce, Sitecore, and Microsoft Azure environments.
Working familiarity with responsible AI principles and general awareness of relevant AI regulatory trends.
Familiarity with multi-agent frameworks such as LangChain, Microsoft AutoGen, or comparable tools.
Direct Graphwise experience; working knowledge of Neo4j, Amazon Neptune, or Azure Cosmos DB is also helpful.
Familiarity with Azure Data Factory, Apache Airflow, or other data and workflow pipeline orchestration tools.
Microsoft Certified: Azure AI App and Agent Developer Associate (AI-103) or MLOps Engineer Associate (AI-300); Azure AI Engineer Associate (AI-102) or Azure Data Scientist Associate (DP-100) are also recognized.
AWS Certified Generative AI Developer – Professional or Machine Learning Engineer – Associate, or Google Cloud Professional Machine Learning Engineer.

Responsibilities

Provide hands-on engineering for development, integration, and production delivery of ISACA AI solutions, beginning with the AI-driven member experience.
Build and maintain AI components, including knowledge graph ontology and configuration, embedding and retrieval pipelines, recommendation logic, and LLM-powered and agentic application components.
Implement APIs and service contracts connecting AI capabilities to ISACA’s digital experiences and business systems; build components for reuse, internal ownership, and maintainability.
Serve as the key internal technical counterpart to external AI development vendors and contractors; participate in architecture reviews and review vendor code and pull requests.
Evaluate vendor AI designs against technical requirements and engineering standards; define acceptance criteria, support production-readiness validation, and surface technical risks or delivery misalignment.
Lead knowledge transfer to move vendor-built components to internal ownership and progressively reduce vendor dependency.
Implement and operate MLOps and LLMOps practices, including CI/CD, model and prompt versioning, evaluation, monitoring, drift detection, and token cost tracking.
Establish team engineering practices such as code review, documentation, runbooks, and reusable components; foster knowledge sharing and organizational expertise.
Implement security, responsible AI, and auditability controls defined by the Senior Director and Security.
Build and maintain data foundations, including ingestion and transformation pipelines from CRM, third-party content and learning platforms, and external workforce and market trend data.
Implement data contracts with source-system owners and data quality checks and monitoring; identify and document data architecture gaps and work with data, engineering, and product teams to address them.
Recommend roles and hiring sequence for bringing outsourced AI engineering in-house; participate in hiring and onboarding and mentor engineers and contractors.
Translate AI system decisions, technical risks, and engineering tradeoffs into actionable communication for product, engineering, and IT stakeholders.
Collaborate with product, architecture, security, and data teams to integrate AI initiatives into ISACA’s broader enterprise.
Evaluate emerging AI tools, platforms, and architectures through hands-on experimentation and proofs of concept, and provide recommendations supporting build-versus-buy and platform decisions.