About The Position

Netlist, Inc. is seeking a highly technical AI Inference / Systems Software Architect to investigate how AI workloads, software stacks, and inference runtimes use hardware, and to architect software that enables new hardware and memory technologies to deliver measurable customer value. The role focuses on AI data center and edge inference workloads, model execution, systems software, memory management, and hardware/software co-design. The architect will work closely with the Chief AI Systems Architect, AI Product Strategist, AI Hardware / Memory Systems Architect, and R&D engineering teams. The work includes analyzing workload behavior and software bottlenecks; designing inference runtimes, systems software, and memory-management approaches; and collaborating on next-generation memory architectures and hardware interfaces. The role also develops prototypes, workload models, emulation environments, and benchmarks to evaluate architectural concepts and quantify expected system-level performance and efficiency benefits.

Requirements

  • Bachelor's or Master's degree in Computer Science, Computer Engineering, Electrical Engineering, or a related technical field.
  • 10+ years of experience in systems software, AI infrastructure, computer architecture, inference systems, compilers, runtimes, or related areas.
  • Deep understanding of modern AI/ML inference workloads.
  • Strong systems programming skills, preferably in C/C++, Python, or comparable languages.
  • Strong understanding of computer architecture, memory systems, operating systems, and data movement.
  • Experience architecting or optimizing large-scale AI inference systems.
  • Demonstrated experience translating application or workload requirements into systems architecture.
  • Strong understanding of performance modeling and benchmarking.
  • Excellent communication skills and ability to work across hardware and software organizations.

Nice To Haves

  • PhD in Computer Science, Computer Engineering, Electrical Engineering, or a related technical field.
  • Experience with LLM inference, Transformer architecture, KV cache, vLLM, PyTorch, CUDA, Triton, TensorRT, ROCm, XLA, MLIR/LLVM, FlashAttention, MoE, quantization, speculative decoding, distributed inference, GPU systems, AI accelerators, HBM, DDR, CXL, NUMA, memory pooling or disaggregation, PIM or near-memory computing, AI networking, AI storage, edge AI, or hardware/software co-design.

Responsibilities

  • Develop a deep understanding of modern AI inference workloads and their system-level behavior, including LLMs, generative AI, agentic AI, long-context inference, MoE, RAG, multimodal AI, and edge inference.
  • Analyze prefill and decode workloads and characterize how batch size, sequence length, concurrency, model size, attention mechanisms, and memory access patterns affect execution.
  • Identify whether workload bottlenecks stem from compute, memory bandwidth or capacity, latency, interconnect, software overhead, or scheduling.
  • Architect software infrastructure to help AI workloads efficiently use heterogeneous memory and compute resources, including runtimes, execution engines, scheduling, resource and memory management, KV-cache management, data movement, prefetching, caching, model partitioning, load balancing, multi-device and distributed inference, accelerator orchestration, telemetry, and performance monitoring.
  • Evaluate and contribute to open-source and commercial AI software stacks, including PyTorch, CUDA, ROCm, Triton, vLLM, and emerging inference runtimes and compiler frameworks.
  • Work with the AI Hardware / Memory Systems Architect on software for next-generation memory architectures and manage resources such as HBM, DDR, CXL memory, memory pools, disaggregated memory, NVMe, persistent storage, model weights, context state, and KV cache.
  • Develop policies and mechanisms for data placement, tiering, migration, caching, prefetching, eviction, compression, decompression, memory sharing, QoS, isolation, and resource allocation.
  • Collaborate with hardware architects to expose and use hardware capabilities such as CXL memory, memory pooling, smart memory controllers, near-memory compute, PIM, inference engines, AI accelerators, compression engines, and specialized cache architectures.
  • Collaborate on runtime architecture, memory and KV-cache management, scheduling, data placement, prefetching, compression, workload orchestration, APIs, software interfaces, and hardware telemetry and monitoring.
  • Ensure hardware innovations deliver measurable application-level benefits, and develop performance-optimization methodologies based on real AI workloads.
  • Measure and optimize tokens per second, TTFT, TPOT, latency, throughput, memory-bandwidth and capacity utilization, accelerator utilization, power efficiency, cost per inference, cost per million tokens, and system-level TCO.
  • Identify performance improvements through kernel optimization, quantization, sparsity, compression, operator fusion, memory placement, scheduling, batching, speculative decoding, KV-cache optimization, prefill/decode optimization, and communication reduction.
  • Help define software architecture for CXL-based memory expansion, pooling, and disaggregation for AI workloads, including placement, allocation, NUMA-aware scheduling, remote-memory performance, bandwidth allocation, QoS, and migration.
  • Develop software mechanisms that let applications use additional memory capacity without unacceptable latency or performance penalties.
  • Develop AI context-memory software architecture, potentially including KV-cache tiering, compression and quantization, prefix sharing, context reuse, cache eviction, context migration, session-aware caching, distributed context management, GPU-to-remote-memory transfers, and context prefetching.
  • Develop software prototypes to validate new hardware architectures; build workload models and emulation environments; create customer-representative benchmark suites; evaluate concepts on CPUs, GPUs, FPGAs, or emulation platforms; quantify software overhead and expected system benefits; and establish performance baselines against existing GPU, CPU, CXL, and inference solutions.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service