Staff Inference Engineer - AI
About The Position
Thomson Reuters is seeking a Staff Inference Engineer to productionize, optimize, and scale AI and large language model workloads that power its AI-driven products. The role works with platform teams to improve capacity forecasting for AI workloads and with Product, Data Science, Architecture, and Enterprise AI teams to bring research models into production. The engineer will ensure models—from classical machine learning to generative AI—run efficiently across Thomson Reuters’ multicloud footprint, meet enterprise reliability requirements, and integrate with the company’s data backbone. The role involves optimizing inference performance and compute utilization, deploying GPU workloads on cloud and Kubernetes infrastructure, developing production-grade APIs, and implementing routing, failover, monitoring, and governance practices. The engineer will also collaborate with cloud and AI engineering teams to develop infrastructure, guardrails, and automation for enterprise AI services.
Requirements
- 7+ years of relevant experience.
- Strong understanding of ML and LLM fundamentals and inference optimization techniques.
- Hands-on experience with GPU programming; CUDA is preferred.
- Experience with inference runtimes such as TensorRT and ONNX Runtime.
- Experience with deep learning frameworks such as PyTorch or TensorFlow.
- Proficiency in Python and at least one systems language; C++ is strongly preferred for performance-critical inference paths.
- Experience deploying AI workloads to AWS, GCP, or Azure and Kubernetes.
Nice To Haves
- Familiarity with vector search systems such as OpenSearch vectors and retrieval-augmented generation pipelines.
- Knowledge of distributed systems, microservices, CI/CD, and cloud-native architecture.
Responsibilities
- Productionize, optimize, and scale AI and LLM workloads that power Thomson Reuters products.
- Optimize LLMs and ML models for high-performance inference using quantization, pruning, distillation, and hardware-specific tuning.
- Deploy and scale GPU inference workloads on AWS, Azure, GCP, and internal Kubernetes clusters, supporting predictable performance during peak traffic.
- Implement routing and failover strategies for OpenAI, Anthropic, and Vertex AI traffic.
- Integrate models into production-grade APIs for Thomson Reuters products and enterprise workflows.
- Develop optimized environments and eliminate performance bottlenecks to reduce latency.
- Collaborate with Platform Engineering teams to align inference workloads with cloud-native patterns across AWS, Azure, GCP, and OCI.
- Build and optimize containerized inference pipelines using Kubernetes for large-scale distributed workloads.
- Ensure compliance with Thomson Reuters AI standards for deployment, monitoring, governance, and drift detection.
- Profile inference performance, identify GPU and CPU bottlenecks, and optimize compute utilization across heterogeneous hardware.
- Implement observability and health monitoring for inference pipelines to ensure enterprise AI service reliability.
- Collaborate with AI engineers on quantization techniques, numerical precision, and non-standard architectures.
- Support scaling AI infrastructure during critical releases and global product rollouts.
- Partner with cloud engineers to develop guardrails and automation that support inference workloads.
Benefits
- Flexible vacation and paid time off.
- Two company-wide Mental Health Days off and access to the Headspace app.
- Retirement savings plan with company match.
- Tuition reimbursement.
- Employee incentive programs and potential eligibility for an annual bonus.
- Health, dental, vision, disability, and life insurance.
- Paid holidays.
- Parental leave.
- Sabbatical leave.
- Flexible Spending and Health Savings Accounts.
- Fitness reimbursement.
- Employee Assistance Program.
- Commuter benefits.
- Adoption and surrogacy assistance.
- Employee Stock Purchase Plan.
- Two paid volunteer days off annually.
- Flexible hybrid work arrangements, including work from anywhere for up to 8 weeks per year.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
Education Level
No Education Listed
