Member of Technical Staff, GPU Infrastructure
About The Position
Cohere is hiring a Member of Technical Staff for its Internal Infrastructure team, which builds the infrastructure and tools used to train, evaluate, and serve Cohere’s foundation models. The role works with AI researchers and ML engineers to improve stability, scalability, and observability for the systems behind Cohere’s platform, North. The position operates shared Kubernetes fleets for AI workloads, including multi-cloud GPU and TPU clusters. It covers node health and performance, as well as collaboration with cloud providers and hardware vendors. The role participates in a compensated 24x7 on-call rotation. Cohere hires at multiple levels and determines level based on the scope of work owned and the candidate’s independence in driving it. Cohere is remote-friendly and has offices in Toronto, London, New York City, San Francisco, Montreal, Paris, Berlin, and Seoul. Employees working away from an office may use a co-working benefit; employees receive a home-office stipend.
Requirements
- Experience running GPU Kubernetes clusters with thousands of GPUs for ML or HPC workloads.
- Experience debugging Linux, networking, and performance problems.
- Experience writing software in Python or Go.
- Experience working with researchers, ML engineers, or another team with differing goals.
- Ability to identify bottlenecks, propose fixes, and drive them in a fast-moving environment.
Nice To Haves
- Familiarity with RDMA, NCCL, or accelerator health.
- Preference for extending solid open-source projects rather than rebuilding them.
Responsibilities
- Deploy and operate GPU Kubernetes clusters across clouds, including bringing up new hardware generations and providers.
- Work with cloud providers and hardware partners on reliability, performance, and cost, including RDMA and high-speed interconnects.
- Automate node, GPU, and fabric fault detection and remediation to resolve most faults without human intervention.
- Improve fleet utilization, track costs, and reduce the time required to return faulty nodes to service.
- Troubleshoot production failures and performance problems, and improve system operability.
- Participate in a compensated 24x7 on-call rotation.
Benefits
- Weekly lunch stipend of $75/£75 or local-currency equivalent.
- Full health and dental benefits, including a separate mental-health budget.
- RRSP matching, 401(k), or pension scheme, as applicable.
- 100% parental-leave top-up for up to six months for either parent.
- Annual enrichment benefits for arts and culture, fitness and wellness, quality time, and workspace improvements.
- Education and learning stipend for conferences, courses, and coaching.
- Six weeks of paid vacation (30 working days).
- Travel budget for remote employees to visit other offices, plus an annual company offsite.
- Daily lunch program, snacks, and community and social events for office-based employees.
- Co-working benefit for employees who are not near an office.
- $500 home-office stipend.
- Compensated 24x7 on-call rotation.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Mid Level
Education Level
No Education Listed
