Runtime Engineer

Normal Computing Corporation • Palo Alto, CA

1d

About The Position

Normal Computing is hiring a senior, hands-on Runtime Engineer to design and implement the runtime for its pre-silicon hardware. The runtime is the layer through which compiled programs are loaded, scheduled, and executed on the company’s hardware. The engineer will own its design and much of its implementation, and set its technical direction.

The runtime will initially be built and tested against simulators and FPGA prototypes, behind a single interface, so that software above the runtime does not need to change when silicon becomes available. The role covers device management, memory allocation and host-device transfer, executable loading, command submission, hardware abstraction, and management of persistent device state such as resident model weights. The runtime must keep overhead off the critical path for LLM inference, with performance measured by end-to-end latency and throughput.

The engineer will define interfaces with the compiler above the runtime and the driver and firmware below it, which are owned by the hardware team. They will also guide technical direction, review dependent designs, and collaborate with compiler and serving engineers. Normal Computing is an applied AI company building foundational hardware and software for the semiconductor industry, AI infrastructure, and systems, with teams in New York City, Mountain View, London, Copenhagen, and Seoul.

Requirements

• Experience building runtimes, drivers, or hardware abstraction layers for accelerators or other non-standard hardware.
• Strong systems programming skills in C++ or Rust, including memory management, concurrency, and low-level performance.
• Track record of owning a complex systems component end to end, from design through production, and driving technical decisions across teams.
• Understanding of LLM inference workloads and their runtime requirements.
• Comfort building software for evolving hardware, from simulation through bring-up.

Nice To Haves

• Experience with open runtime designs such as IREE’s HAL, or with CUDA or ROCm runtime internals.
• Experience with in-memory compute or other non-von-Neumann hardware.
• Experience integrating a runtime with an inference server such as vLLM.
• Contributions to open-source runtime or systems infrastructure.

Responsibilities

• Own the runtime architecture, including device management, memory allocation and host-device transfer, executable loading, and command submission.
• Abstract differences between simulators, FPGA prototypes, silicon, and future hardware generations behind a stable interface.
• Manage state stored on the hardware between requests, including resident model weights.
• Keep runtime overhead off the critical path for LLM inference and optimize end-to-end latency and throughput.
• Define the boundary between the runtime and the compiler, driver, and firmware.
• Set the runtime’s technical direction and review designs that depend on it.
• Collaborate with compiler and serving engineers to ensure the runtime interface meets their needs.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed