Runtime Engineer

Normal Computing Corporation • Palo Alto, CA

2d • $200,000 - $300,000

About The Position

Normal Computing is hiring a senior, hands-on Runtime Engineer to own the design, implementation, and technical direction of the runtime for its AI hardware. The runtime is the layer through which compiled programs are loaded, scheduled, and executed on the company’s hardware, which performs memory-bound large-model inference inside memory. Because the chip is pre-silicon, the runtime will initially be built against simulators and FPGA prototypes behind a shared interface, allowing higher-level software to remain stable when silicon arrives.

The role covers runtime architecture, hardware abstraction, device state, performance, cross-team interfaces, and technical leadership. The engineer will define how compiler and serving layers interact with the hardware through a stable interface, while collaborating with the hardware team responsible for the driver and firmware. Normal Computing works across New York City, Silicon Valley (Mountain View), London, Copenhagen, and Seoul.

Requirements

• Experience building runtimes, drivers, or hardware abstraction layers for accelerators or other non-standard hardware.
• Strong systems programming skills in C++ or Rust, including memory management, concurrency, and low-level performance.
• Track record of owning a complex systems component end to end, from design through production, and driving technical decisions across teams.
• Understanding of LLM inference workloads and their runtime requirements.
• Comfort building software for evolving hardware, from simulation through bring-up.

Nice To Haves

• Experience with open runtime designs such as IREE’s HAL, or CUDA or ROCm runtime internals.
• Experience with in-memory compute or other non-von-Neumann hardware.
• Experience integrating a runtime with an inference server such as vLLM.
• Contributions to open-source runtime or systems infrastructure.

Responsibilities

• Own the runtime architecture, including device management, memory allocation, host-device transfer, executable loading, and command submission.
• Build and maintain a stable interface through which compiler and serving layers program the hardware.
• Abstract differences between simulators, FPGA prototypes, silicon, and future hardware generations.
• Manage hardware-resident state between requests, including resident model weights, so the serving layer can rely on it.
• Keep runtime overhead off the critical path for LLM inference, measuring performance by end-to-end latency and throughput.
• Define the boundary with the compiler above the runtime and the driver and firmware below it.
• Set the runtime’s technical direction, review dependent designs, and collaborate with compiler and serving engineers.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed