Global AI/ML Engineer Director

Boston Consulting Group • Boston, MA

20h • $200,000 - $244,000 • Hybrid

About The Position

As the Director of AI/ML Engineering within BCG’s Data Layer Platform Team, you will lead and operationalize the intelligence layer that transforms BCG’s vast data landscape into AI-ready, high-value capabilities. You will direct the design, development, and optimization of AI/ML systems - particularly LLM-powered reasoning, retrieval, enrichment, data interoperability and knowledge synthesis - that enable BCG’s GenAI applications (e.g., PA GenAI tools, AI copilots, Data Products) to deliver accurate, safe, and context-aware insights at scale.

You will own the technical strategy and execution models for how data becomes usable in advanced AI workflows: from ingestion, semantic modeling, abstraction, embeddings, retrieval architecture design, and model pipelines to the construction of reusable AI services and APIs/MCPs that support multiple use cases across the firm.

You will lead an engineering team and collaborate closely with Product Owners, Data Engineers, Platform engineers, Solution Architects and domain experts to ensure our AI/ML systems are robust, scalable, secure, and aligned with enterprise architecture standards. This is a highly visible strategic role shaping BCG’s long-term AI enablement agenda.

Requirements

• 12+ years of experience in AI/ML engineering, including 3–5 years leading teams building production-grade ML/LLM systems.
• Demonstrated expertise in designing LLM-powered systems, including embedding pipelines, vector-based retrieval architectures (e.g., VectorDBs, OpenSearch), RAG patterns, and multi-agent frameworks.
• Strong software engineering foundation in Python and modern ML ecosystems, with experience building APIs/microservices and deploying solutions on cloud-native infrastructure (containers, Kubernetes).
• Deep understanding of data engineering, data modeling, and scalable pipeline design to support production ML and AI workloads.
• Proven ability to define architecture strategy, technical roadmaps, and solution blueprints for complex AI/ML systems at enterprise scale.
• Exceptional cross-functional leadership skills, with the ability to influence senior stakeholders, navigate ambiguity, and align distributed teams around a shared technical direction.
• Strong grounding in data governance, Unified Data architecture constructs (e.g. Data Mesh, Data Fabric), model evaluation, responsible AI principles, and AI Security enterprise security considerations including understanding of Data Ontologies and definition catalogs

Responsibilities

• Define and own the AI/ML engineering strategy for the Intelligence Layer, ensuring alignment with Data Layer vision and enterprise architecture principles
• Architect end-to-end AI and LLM systems (RAG pipelines, embeddings, retrieval frameworks, model orchestration) to make BCG’s data actionable and consumable by GenAI applications.
• Lead the architecture and implementation of scalable AI pipelines, including data ingestion and preprocessing, feature and embedding generation, knowledge graph construction, vector indexing, and retrieval optimization to support LLM and RAG-based applications.
• Design and operationalize AI platform components (e.g., model APIs, vector databases, semantic search services) with a focus on scalability, performance, and maintainability.
• Oversee integration of AI services into enterprise systems, ensuring high availability, security, observability, and compliance with internal data and technology standards.
• Establish best practices for AI system reliability, monitoring, and lifecycle management across development and production environments
• Partner with Product Owners, Data Engineers and Architects to ensure pipelines reflect highest standards of data quality, provenance, governance, and ML-readiness.
• Drive implementation of advanced retrieval, reasoning, and grounding systems to support BCG’s internal GenAI products.
• Collaborate with Product Owners and GenAI teams to translate user needs into technical capabilities, ensuring measurable, high-value outcomes for consulting teams and practice areas.
• Engage with TAL, PPL, and other engineering chapters to align resource strategy, scale best practices, and evolve AI/ML engineering capabilities.
• Ensure systems meet stringent performance, accuracy, stability, and resilience requirements -leveraging expertise such as prompt engineering, LLM optimization, evaluation frameworks, and multi-agent pipelines.
• Introduce emerging techniques (fine-tuning, model compression, retrieval optimization, evaluators, guardrails) to continuously improve system effectiveness.
• Lead, mentor, and develop data and AI engineers, fostering excellence in scalable ML systems, agent-based architectures, and responsible AI practices.
• Establish high engineering standards across experimentation, model development, deployment, and monitoring to ensure production-grade AI solutions.
• Contribute to AI/ML and Data Engineering communities of practice by promoting knowledge sharing, reusable patterns, and continuous capability uplift across teams.

Benefits

• Zero dollar ($0) health insurance premiums for BCG employees, spouses, and children
• Low $10 (USD) copays for trips to the doctor, urgent care visits and prescriptions for generic drugs
• Dental coverage, including up to $5,000 in orthodontia benefits
• Vision insurance with coverage for both glasses and contact lenses annually
• Reimbursement for gym memberships and other fitness activities
• Fully vested Profit Sharing Retirement Fund contributions made annually, whether you contribute or not, plus the option for employees to make personal contributions to a 401(k) plan
• Paid Parental Leave and other family benefits such as elective egg freezing, surrogacy, and adoption reimbursement
• Generous paid time off including 12 holidays per year, an annual office closure between Christmas and New Years, and 15 vacation days per year (earned at 1.25 days per month)
• Paid sick time on an as needed basis

What This Job Offers

• Job Type: Full-time
• Career Level: Director
• Education Level: No Education Listed