AI Inference Engineer QVAC (100% remote Worldwide)

Tether Operations Limited

22h • Remote

About The Position

Tether is hiring an AI Inference Engineer for QVAC to own the inference backbone behind its local AI stack. The role focuses on the C++ systems layer that enables models to run quickly, reliably, and predictably on user hardware, with attention to startup behavior, memory pressure, throughput and latency, and stability during long sessions. The engineer will define and evolve core runtime abstractions that support new inference features while maintaining performance and maintainability.

The position involves working on the C++ layer that powers local AI, including porting and enhancing inference engines such as llama.cpp or similar engines for efficient operation on edge devices. The focus is on improving model loading speed, resource efficiency, and performance across hardware. The inference layer must be stable, optimized, and ready to integrate with the broader stack. This work supports private, on-device AI experiences and QVAC's peer-to-peer AI products without relying on cloud infrastructure. The role is 100% remote worldwide.

Requirements

• Strong programming skills in C++.
• Strong experience with llama.cpp and ggml inference engines, including deploying models to specific GPU architectures.
• Experience with at least one GPU framework: CUDA, Vulkan, Metal, or OpenCL.
• Good understanding of deep learning concepts and model architectures.
• Experience with transformers, large language models, and diffusion models.
• Demonstrated ability to rapidly learn new technologies and techniques.
• A degree in Computer Science, AI, Machine Learning, or a related field, complemented by a solid track record in AI research and development.

Nice To Haves

• Experience training or fine-tuning a large language model.
• Experience productionizing models.
• Research experience in new model architectures.
• Experience with distributed systems.
• JavaScript experience.

Responsibilities

• Deploy machine learning models to edge devices using frameworks such as llama.cpp and ggml.
• Collaborate with researchers on coding and training, and transition models from research into production environments.
• Integrate AI features into existing products using advancements in machine learning.
• Port and enhance inference engines for efficient operation across edge devices and hardware.
• Improve runtime startup behavior, memory usage, throughput, latency, and long-session stability.

What This Job Offers

• Job Type: Full-time