Remote | GPU Kernel Engineer — $60–$80/hour

24-Mag•New York, NY

1d•Remote

About The Position

We are sharing a specialised part-time consulting opportunity for experienced GPU and accelerator engineers with hands-on expertise in kernel development, numerical validation, performance optimisation, profiling, and low-level compute frameworks. This role focuses on evaluating GPU and accelerator kernel-development tasks for correctness, completeness, reproducibility, and performance quality. Selected engineers will review kernel implementations, benchmark methodology, compilation and runtime behaviour, numerical tolerances, and optimisation decisions across multiple accelerator ecosystems.

Requirements

3+ years of hands-on experience developing, optimising, or verifying GPU or accelerator kernels
Practical experience with at least two of CUDA, Triton, NKI, or Pallas (JAX)
Strong understanding of numerical correctness, including absolute, relative, and ULP tolerances
Experience selecting and validating appropriate reference implementations
Strong performance profiling and benchmarking experience
Familiarity with Nsight, Nsight Compute, roofline analysis, or comparable profiling tools
Strong understanding of common kernel compilation and runtime failure modes
Experience with at least three of the following: Kernel generation from specification, Framework translation or lowering, Hardware-target migration, Kernel debugging, Performance optimisation, Operator fusion
Strong written communication and ability to provide precise technical feedback

Nice To Haves

Experience across both NVIDIA GPU and custom-accelerator ecosystems is preferred
Background in compiler engineering, MLIR, or intermediate-representation lowering is advantageous
Strong understanding of memory-hierarchy optimisation is preferred
Contributions to kernel or accelerator libraries such as cuBLAS, cuDNN, Triton community kernels, or JAX/XLA custom calls are advantageous

Responsibilities

Evaluate GPU and accelerator kernels for technical correctness and completeness
Review implementations developed from specifications or reference operators
Assess whether kernels correctly implement intended mathematical behaviour
Identify implementation defects, unsupported assumptions, or incomplete solutions
Apply practical judgement grounded in hands-on kernel engineering experience
Review kernel development across frameworks such as CUDA, Triton, NKI, and Pallas (JAX)
Assess framework-specific implementation choices and execution constraints
Evaluate kernel translations or migrations between different frameworks
Identify incorrect assumptions when moving implementations across accelerator ecosystems
Compare alternative kernel implementations for correctness and technical quality
Assess kernel outputs against suitable reference implementations
Evaluate absolute, relative, and ULP-based tolerances
Determine whether numerical differences fall within acceptable limits
Review floating-point behaviour and precision-related edge cases
Identify discrepancies caused by implementation defects rather than expected numerical variation
Evaluate kernel performance using tools such as Nsight, Nsight Compute (ncu), roofline analysis, or framework-native profilers
Assess whether benchmark methodology produces fair and meaningful comparisons
Review latency, throughput, utilisation, and memory behaviour
Identify misleading benchmarking practices or inappropriate baselines
Determine whether claimed performance improvements are supported by evidence
Review optimisation strategies for compute and memory efficiency
Assess tiling, vectorisation, parallelisation, and workload decomposition
Evaluate trade-offs between arithmetic throughput and memory movement
Identify bottlenecks affecting kernel performance
Assess whether optimisations preserve numerical correctness
Review use of registers, shared memory, caches, and accelerator-specific memory resources
Evaluate shared-memory tiling, register pressure, bank conflicts, and coalescing patterns
Identify inefficient memory-access behaviour
Assess data locality and memory-bandwidth utilisation
Evaluate whether memory optimisations appropriately match the target hardware
Diagnose common kernel compilation and runtime failures
Review issues involving driver incompatibilities, out-of-memory conditions, launch configurations, shape or stride mismatches, and autotuning failures
Determine whether failures originate from kernel logic, environment configuration, or runtime assumptions
Evaluate proposed debugging approaches and corrective actions
Assess whether tasks execute reliably in their intended environment
Review kernels translated or lowered across programming frameworks
Evaluate migration between different accelerator targets
Assess whether computational semantics and performance assumptions remain valid
Identify platform-specific behaviour that requires redesign rather than direct translation
Evaluate migration quality across GPU and custom-accelerator environments
Review debugging tasks involving incorrect or unstable kernel implementations
Diagnose failures using outputs, profiler data, runtime behaviour, and source code
Evaluate operator-fusion strategies where relevant
Assess whether fused kernels preserve intended semantics
Identify optimisation decisions that introduce correctness or maintainability issues
Evaluate kernel tasks involving compiler or intermediate-representation concepts where applicable
Review transformations between high-level operators and accelerator-level implementations
Assess lowering decisions for correctness and efficiency
Apply familiarity with MLIR or comparable compiler infrastructures where relevant
Identify issues arising from compiler or code-generation assumptions
Assess assigned kernel tasks against structured technical criteria
Provide clear written explanations supporting evaluation decisions
Reference specific numerical, performance, compilation, or runtime evidence
Apply evaluation standards consistently across assignments
Distinguish valid implementation alternatives from technically flawed approaches