LLM Inference Engineer

Near AI • San Francisco, CA

2d • Remote

About The Position

The NEAR AI team is building decentralized and confidential machine learning infrastructure to enable user-owned AI. Our mission is to build highly scalable and efficient infrastructure for open-source AI at a global scale. We are specifically seeking an expert in high-performance LLM serving systems and inference optimization. In this role, you will push the boundaries of how large language models are served.

Requirements

• Strong hands-on experience in LLM inference, with expertise debugging and optimizing major inference engines such as SGLang, vLLM, or TensorRT.
• Deep knowledge of state-of-the-art GPU architectures, and effectively exploit them using PyTorch, Triton, CuTe, CUDA, etc.
• Proven track record in designing and maintaining end-to-end high-traffic LLM serving systems.
• Strong problem-solving skills and ability to communicate technical ideas clearly.

Nice To Haves

• Experience with Trusted Execution Environments (TEE).
• Active contributor to open-source LLM inference engines.

Responsibilities

• Architect and maintain production high-traffic LLM serving systems.
• Optimize throughput, latency, and cost for leading open-source LLMs.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed