AI Research Engineer (Kernel & Inference Optimization) - 100% Remote Worldwide

Tether Operations Limited

1d•Remote

About The Position

As a member of Tether’s AI model team, the AI Research Engineer will develop and optimize model-serving architectures and inference strategies for advanced AI systems. The work spans resource-efficient models for limited hardware and complex multimodal architectures integrating text, images, and audio. The role takes a hands-on, research-driven approach to developing, testing, and implementing serving strategies and inference algorithms, with the goal of delivering scalable, high-throughput, low-latency AI performance with a low memory footprint in real-world applications. The engineer will build robust inference pipelines, define performance metrics, and identify and resolve production bottlenecks. The role includes optimizing deployments across diverse environments, including resource-constrained devices, mobile phones, and edge platforms, and collaborating with cross-functional teams to integrate optimized frameworks into production pipelines for edge and on-device applications.

Requirements

A degree in Computer Science or a related field.
Knowledge of Metal Shading Language (MSL) and ability to write custom compute shaders from scratch.
Proven experience in low-level kernel optimization and inference optimization on mobile devices.
Experience delivering measurable improvements in inference latency, throughput, and memory footprint for domain-specific applications, particularly on resource-constrained devices and edge platforms.
Deep understanding of modern model-serving architectures and inference optimization techniques, including methods for low-latency, high-throughput performance and efficient memory management.
Strong expertise writing GPU kernels for mobile devices such as smartphones, and deep understanding of model-serving frameworks and engines.
Practical experience developing and deploying end-to-end inference pipelines, from model optimization through integration on resource-constrained devices.
Ability to apply empirical research to challenges in model serving, including latency optimization, computational bottlenecks, and memory constraints.
Proficiency in designing evaluation frameworks and iterating on optimization strategies.
Understanding of distributed inference techniques, including tensor, pipeline, and expert parallelism.
Deep understanding of the mathematics and structure of diffusion models and vision transformers.
Understanding of pruning, quantization, FlashAttention, KV cache, and speculative decoding (Eagle).

Nice To Haves

A PhD in NLP, Machine Learning, or a related field.
A strong track record in AI research and development, including good publications in A conferences.

Responsibilities

Design and deploy model-serving architectures that provide high throughput and low latency while optimizing memory use.
Ensure inference pipelines operate efficiently across diverse environments, including resource-constrained devices and edge platforms.
Establish performance targets for latency, token response, and memory footprint.
Build, run, and monitor controlled inference tests in simulated and live production environments.
Track latency, throughput, memory consumption, and error rates, with attention to resource-constrained devices.
Document iterative test results and compare them with benchmarks across platforms.
Identify and prepare test datasets and simulation scenarios for real-world deployment challenges on low-resource devices, with measurable evaluation criteria for performance, latency, and memory utilization.
Analyze computational efficiency and diagnose serving-pipeline bottlenecks using processing and memory metrics.
Address issues including suboptimal batch processing, network delays, and high memory use to improve scalability and reliability.
Collaborate with cross-functional teams to integrate optimized serving and inference frameworks into production pipelines for edge and on-device applications.
Define success metrics and ensure continuous monitoring and iterative refinement of serving systems.
Design and optimize distributed inference engines using techniques such as tensor, pipeline, and expert parallelism.