Member of Technical Staff - GPU Performance

Gimlet Labs • San Francisco, CA

3d • $150,000 - $390,000

About The Position

Gimlet is building a multi-silicon neocloud for fast, efficient AI inference. It combines large-scale compute infrastructure with an execution platform that partitions AI workloads and maps each stage to suitable hardware, serving foundation labs, hyperscalers, and AI-native companies.

As a Member of Technical Staff focused on GPU performance, you will characterize and improve production AI inference performance across Gimlet’s heterogeneous accelerator fleet. You will work across kernel execution, compiler output, runtime behavior, memory movement, synchronization, scheduling, and multi-device execution, connecting hardware-level evidence to end-to-end latency, throughput, utilization, and cost.

You will build measurement systems and performance methodology to help the company understand new hardware, find limiting resources in real workloads, and implement improvements across kernels, compilers, runtimes, and distributed execution. In the first 12–18 months, success includes establishing performance baselines, diagnosing bottlenecks, improving execution, building automated benchmarks and regression detection, translating findings into production improvements, and developing a repeatable process for bringing up new accelerators.

Requirements

• Strong software engineering fundamentals and experience working on performance-critical systems.
• Experience profiling GPU or accelerator workloads and turning measurements into production improvements.
• Strong understanding of GPU execution, memory hierarchies, asynchronous execution, synchronization, and performance tradeoffs.
• Ability to design controlled experiments, distinguish symptoms from root causes, and communicate performance findings precisely.
• Bachelor’s degree in a relevant field, or an equivalent combination of education, training, and professional experience.

Nice To Haves

• Experience with Nsight Systems, Nsight Compute, rocprof, Omniperf, VTune, Perfetto, or hardware performance counters.
• Experience with CUDA, HIP, Triton, or another accelerator programming model, and ability to prototype low-level optimizations.
• Familiarity with roofline analysis, queueing behavior, occupancy, latency hiding, instruction throughput, and memory-bandwidth analysis.
• Experience optimizing inference runtimes, execution graphs, memory allocators, kernel dispatch, batching, or model-serving systems.
• Familiarity with collective communication, topology-aware execution, and compute-communication overlap across multiple accelerators.
• Experience building performance laboratories, benchmark suites, regression infrastructure, or hardware bring-up methodology.

Responsibilities

• Establish trustworthy performance baselines for production inference workloads across established and emerging accelerator architectures.
• Isolate bottlenecks involving compute, memory bandwidth, cache behavior, kernel launch, synchronization, host-device interaction, communication, and runtime scheduling.
• Use profiling, hardware counters, controlled experiments, and performance models to explain workload behavior and identify high-leverage optimizations.
• Improve execution through techniques such as launch reduction, graph capture, stream and event coordination, compute-communication overlap, memory reuse, batching, and scheduling.
• Build automated benchmarks and regression detection across models, tensor shapes, data types, software releases, and hardware targets.
• Translate performance findings into production improvements, partnering with kernel, compiler, ML systems, and distributed systems engineers as needed.
• Develop a repeatable methodology for evaluating and bringing up new accelerators within Gimlet’s execution platform.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level