Member of Technical Staff, GPU Kernels

San Francisco Tensor Company•San Francisco, CA

11h•Onsite

About The Position

SF Tensor is building high-performance compute technology across hardware, compilers, and cloud infrastructure. Its Kernel Optimizer automatically searches for optimized code across hardware vendors and cluster topologies, while its Model Foundry manages model runs and supports moving workloads across clouds and chips. The Member of Technical Staff, GPU Kernels will establish the performance ceiling for GPU workloads through hand-optimized kernels, then translate the resulting knowledge into structures that the compiler can search. The role spans pre-training, post-training, and inference on NVIDIA, AMD, TPU, and Trainium hardware. The team works throughout the stack, including at the ISA level, and develops custom compiler backends, performance models, microbenchmarks, and tooling. The role works alongside the formal correctness team to attach formal proofs to aggressively optimized kernels. Approximately two-thirds of the role is kernel work and one-third is converting findings into the compiler search space. The team works primarily in person at its San Francisco office; relocation assistance is offered.

Requirements

Track record of hand-writing kernels that match or outperform vendor libraries.
Comfort reading PTX, SASS, GCN/CDNA ISA, or equivalent machine-level assembly.
Fluency with low-level profiling tools such as Nsight Compute, Nsight Systems, rocprof, omniperf, or equivalents.
Solid systems programming skills in C++ and CUDA or ROCm/HIP.
Working understanding of how high-level ML operations map to hardware, including where framework layers create obstacles.

Nice To Haves

Experience with compiler backends, including MLIR, LLVM, code generation, instruction selection, or scheduling.
Research experience in superoptimization, program synthesis, formal verification, or search-based compilation.
Experience with non-NVIDIA silicon, such as AMD MI-series, TPU, Trainium, or mobile edge GPUs such as Metal or Mali.
Experience with distributed AI training.
Experience with high-speed interconnects such as NVLink, NVSwitch, InfiniBand, or RoCE.
HPC background in large-scale scientific computing, MPI, or supercomputing.
Driver development experience or a background in electrical engineering, computer architecture, or hardware design.

Responsibilities

Write and hand-optimize kernels for real workloads to establish performance ceilings before automated search.
Profile at the microarchitectural level, including SM and CU utilization, warp stalls, memory bank conflicts, register pressure, instruction throughput, and occupancy tradeoffs.
Debug kernel performance issues involving clock behavior, thermal throttling, and driver paths.
Convert optimization insights into machine-searchable structures for the compiler.
Work below PTX at the ISA level, reasoning about SASS and cubins to create schedules that PTX cannot express.
Build performance models, microbenchmarks, and tools to predict kernel behavior.
Work with the formal correctness team so aggressively optimized kernels ship with formal proofs.
Develop and optimize kernels for pre-training, post-training, and inference across NVIDIA, AMD, TPU, and Trainium hardware.