Remote | AWS Trainium Kernel Engineer (NKI) — $60–$80/hour

24-Mag•New York, NY

1d•Remote

About The Position

We are sharing a specialised part-time consulting opportunity for experienced kernel engineers with hands-on expertise in the Neuron Kernel Interface (NKI), AWS Trainium/Inferentia2 hardware, low-level performance optimisation, and CUDA-to-NKI migration. This role focuses on evaluating NKI kernel-development tasks for technical correctness, hardware appropriateness, numerical fidelity, and performance quality. Selected experts will review Trainium-specific implementations, migration decisions, memory-management strategies, profiling results, and cross-platform numerical behaviour while providing clear, rubric-based technical feedback.

Requirements

2+ years of hands-on experience developing or optimising kernels using the Neuron Kernel Interface (NKI)
Professional experience targeting AWS Trainium or Inferentia2 hardware
Strong understanding of tile-based computation
Deep familiarity with SBUF, PSUM, and HBM memory management
Strong knowledge of partition-dimension constraints and DMA orchestration
Demonstrated experience evaluating or performing CUDA-to-NKI migrations
Familiarity with Trainium-specific performance profiling
Experience assessing NeuronCore pipeline utilisation, tensor-engine throughput, and memory-bandwidth bottlenecks
Strong understanding of cross-platform numerical correctness and mixed-precision behaviour
Direct experience with the AWS Neuron SDK, Neuron Compiler internals, or NKI kernel libraries is preferred
Prior CUDA or Triton kernel development experience is advantageous
Familiarity with NeuronCore-v2 architecture and supported numerical formats is preferred
Experience benchmarking ML workloads on Trn1 or Trn2 instances is advantageous
Strong written communication and ability to provide precise technical feedback

Nice To Haves

Direct experience with the AWS Neuron SDK, Neuron Compiler internals, or NKI kernel libraries
Prior CUDA or Triton kernel development experience
Familiarity with NeuronCore-v2 architecture and supported numerical formats
Experience benchmarking ML workloads on Trn1 or Trn2 instances

Responsibilities

NKI Kernel Development Review: Evaluate kernels developed using the Neuron Kernel Interface (NKI). Assess whether implementations appropriately target AWS Trainium and Inferentia2 hardware. Review low-level computation patterns for technical correctness. Identify inefficient, incorrect, or hardware-inappropriate implementation choices. Apply practical judgement grounded in hands-on NKI development experience.
CUDA-to-NKI Migration Review: Assess migrations of existing CUDA kernels to NKI. Assess whether computational semantics are preserved across platforms. Identify translation errors, unsupported assumptions, or inefficient migration strategies. Evaluate whether NKI implementations appropriately account for Trainium architecture. Distinguish faithful migrations from implementations that merely reproduce surface-level CUDA structure.
Tile-Based Computation: Assess tile decomposition and computation strategies. Review partitioning decisions against NKI execution constraints. Evaluate whether kernels make effective use of available compute resources. Identify inefficient tiling or data-movement patterns. Assess whether implementation choices align with NKI programming requirements.
Memory Hierarchy Management: Review use of SBUF, PSUM, and HBM. Assess data placement and movement across Trainium memory hierarchies. Evaluate memory-bandwidth utilisation and locality. Identify unnecessary transfers or memory bottlenecks. Review implementation decisions affecting on-chip memory efficiency.
DMA & Data Movement: Evaluate DMA orchestration within NKI kernels. Review sequencing of computation and data-transfer operations. Identify stalls, inefficient transfer patterns, or synchronisation issues. Assess whether data movement appropriately overlaps with computation. Evaluate implementation choices affecting pipeline utilisation.
Trainium Performance Optimisation: Review Trainium-specific optimisation strategies. Assess NeuronCore pipeline utilisation, tensor-engine throughput, and memory-bandwidth behaviour. Identify performance bottlenecks within kernel implementations. Evaluate whether optimisation decisions are supported by profiling evidence. Review trade-offs affecting throughput, latency, and resource utilisation.
Numerical Correctness: Evaluate numerical consistency between GPU and Trainium implementations. Review differences caused by accumulation order, rounding behaviour, and mixed-precision semantics. Assess appropriate tolerances for cross-platform comparisons. Identify numerical discrepancies that indicate implementation defects. Distinguish expected hardware-level variation from substantive correctness problems.
Precision & Data Types: Review kernels using supported formats such as FP32, BF16, FP8, and INT8. Assess precision choices against computational requirements. Evaluate mixed-precision behaviour and numerical stability. Identify inappropriate casting or accumulation strategies. Review whether performance gains are achieved without compromising required correctness.
AWS Neuron Ecosystem: Evaluate implementations using the AWS Neuron SDK. Review interactions between kernel code, compilation, and Trainium execution. Assess compiler-related behaviours where relevant. Apply familiarity with NKI kernel libraries and Neuron tooling. Identify implementation issues arising from platform-specific constraints.
Benchmarking & Validation: Review benchmark results for Trainium workloads. Assess performance comparisons and experimental methodology. Evaluate workloads running on Trn1 or Trn2 instances where applicable. Determine whether claimed performance improvements are supported by evidence. Identify benchmarking methodologies that could produce misleading conclusions.
Rubric-Based Technical Evaluation: Assess assigned kernel-development tasks against structured technical criteria. Provide clear written explanations supporting evaluation decisions. Reference specific implementation, profiling, or numerical evidence. Apply evaluation standards consistently across assignments. Distinguish valid optimisation alternatives from technically flawed approaches.