100% Remote Golang Developer with Devops/LLM exp. || W2 Consultant
About The Position
Seeking a Golang Developer with cloud and distributed-services experience and LLM experience as a secondary skill. The role focuses on developing and maintaining a platform for serving large language models, optimized for the GPU platforms on which they run. It involves work across the product development lifecycle, from ideation and experimentation through development, testing, release, and operations. The position is described as 100% remote and as a 6–12 month W2 consulting engagement. The developer will build observability and system-health tooling, auto-tuning capabilities, and benchmarking frameworks to guide model-serving and infrastructure tuning. The work also includes native cross-platform inference support for NVIDIA and AMD GPUs, support for varied model architectures, and contributions to open-source inference engines for use on DigitalOcean cloud.
Requirements
- Proficiency in Golang for building scalable, performant backend services.
- Deep experience building services in modern cloud environments and distributed systems, including containerization, infrastructure as code, CI/CD pipelines, APIs, authentication and authorization, data storage, deployment, logging, monitoring, and alerting.
- Experience working with large language models, particularly hosting them for inference.
- Strong verbal and written communication skills.
- Experience building or using benchmarking tools to evaluate LLM inference across models, engines, and GPU combinations.
- Familiarity with LLM performance metrics such as prefill throughput, decode throughput, TPOT, and TTFT.
- Experience with one or more inference engines, such as vLLM, SGLang, or Modular Max.
- Familiarity with one or more distributed inference-serving frameworks, such as llm-d, NVIDIA Dynamo, or Ray Serve.
- Experience with AMD and NVIDIA GPUs and related software such as CUDA, ROCm, AITER, NCCL, or RCCL.
- Knowledge of distributed inference optimization techniques, including tensor/data parallelism, KV-cache optimizations, and smart routing.
Nice To Haves
- Experience with AMD and NVIDIA GPUs and related software such as CUDA, ROCm, AITER, NCCL, or RCCL.
- Knowledge of distributed inference optimization techniques, including tensor/data parallelism, KV-cache optimizations, and smart routing.
- GPU experience is described as a lower-priority preferred skill.
Responsibilities
- Develop and maintain an inference platform for serving large language models, optimized for the GPU platforms on which they run.
- Work on AI and cloud engineering projects through the entire product development lifecycle, including ideation, product definition, experimentation, prototyping, development, testing, release, and operations.
- Build tooling and observability to monitor system health, and develop auto-tuning capabilities.
- Build benchmarking frameworks to test model-serving performance and guide system and infrastructure tuning.
- Build native cross-platform inference support across NVIDIA and AMD GPUs for a variety of model architectures.
- Contribute to open-source inference engines to improve their performance on DigitalOcean cloud.
- Communicate technical subjects with local and remote colleagues and write detailed documentation.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Education Level
No Education Listed
