Technical Product Manager – AI Neocloud Software Stack

Gruve • Redwood City, CA

8h • $180,000 - $220,000 • Onsite

About The Position

Gruve is seeking a Technical Product Manager (Inbound) with experience in AI neocloud environments and modern inference software stacks. In this role, you will shape inference-related product capabilities by collaborating closely with engineering teams. You will translate customer and partner feedback into actionable priorities, guide technical trade-offs, and ensure product decisions are well understood across teams. While you won’t be implementing systems yourself, you’ll need a deep technical understanding to ask the right questions and earn the trust of engineers.

Requirements

• 2+ years of product management experience in AI neoclouds, inference platforms, or GPU-centric infrastructure products.
• Experience working closely with engineering teams on highly technical systems.
• Proven experience owning inbound product responsibilities.
• Understanding of inference stack components such as: vLLM, SGLang, TensorRT-LLM, or similar.
• Continuous batching and KV cache behavior.
• Prefill vs. decode phases.
• Model routing and request scheduling.
• Quantization (INT8, INT4, FP8).
• GPU utilization, memory constraints, and autoscaling.
• OpenAI-compatible APIs, SDKs, SLAs, and observability.
• Bachelor’s degree in Computer Science, Engineering, or related technical field (or equivalent practical experience).

Nice To Haves

• Advanced degree in a technical field.
• Hands-on experience with AI infrastructure, inference platforms, or software systems.
• Strong customer-centric mindset, technical fluency, and collaborative approach.
• Excellent communication skills, able to ask clarifying questions and earn trust with engineering teams.

Responsibilities

• Own inbound product responsibilities related to AI inference software.
• Define and refine product requirements for model serving, routing, and performance.
• Translate customer and partner feedback into actionable product priorities.
• Partner closely with engineering to guide roadmap decisions and trade-offs.
• Balance latency, throughput, cost, and usability in product decisions.
• Ensure product decisions are documented, communicated, and well understood.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level