Intern, AI Application Engineer

Samsung Semiconductor • Folsom, CA

3d • Onsite

About The Position

The SoC Architect team develops candidate SoC-level architectures for emerging memory and storage solutions and demonstrates their viability through hands-on implementation and validation. This internship focuses on deploying and benchmarking LLM inference and evaluating KV cache offloading to network-attached storage, helping shape requirements for future SSD controllers. The role is based at Samsung Semiconductor’s San Jose, California headquarters and requires daily onsite presence, in alignment with the company’s Flexible Work policy. The intern reports to the Senior Director and works closely with a mentor and the storage architecture team.

Requirements

• Pursuing a bachelor’s, master’s, or PhD degree in Computer Science, Computer Engineering, Electrical Engineering, or a related field.
• Must have at least one academic quarter or semester remaining.
• Hands-on experience running LLM inference with Python and PyTorch on NVIDIA GPUs under Linux, using a serving framework such as vLLM, SGLang, or Hugging Face TGI.
• Understanding of LLM serving internals, including KV cache, prefix caching, batching, and tensor or pipeline parallelism, as well as latency metrics such as TTFT, TPOT, and throughput.
• Experience benchmarking and profiling system performance and analyzing results with scripts, such as Python, pandas, and plotting tools, to explain bottlenecks.
• Inclusive approach, adapting communication style to situations and diverse global norms.
• Curiosity and resilience when learning and addressing challenges, using data to build understanding.
• Collaborative approach to building relationships, offering support, and welcoming different approaches.
• Innovative and creative mindset, with a willingness to explore ideas and adapt to change.

Nice To Haves

• Familiarity with KV cache offloading tools such as LMCache or NVIDIA Dynamo/NIXL.
• Familiarity with RDMA/RoCE or NVMe/NVMe-oF storage.
• Familiarity with containers such as Docker or Kubernetes.

Responsibilities

• Deploy and operate an LLM inference service using vLLM to serve Llama-3.3-70B on a server with four NVIDIA H200 GPUs.
• Build an automated benchmark harness that replays long-context, multi-turn, and agent-style request traces and records latency and throughput metrics, including TTFT, TPOT, and goodput.
• Configure KV cache offloading using LMCache and NVIDIA Dynamo/NIXL to network-attached storage over NVMe-oF and RDMA, first on a BlueField-3 storage server and then on the company’s network-attached SSD prototype.
• Run daily experiment sweeps across request rates, context lengths, and cache sizes, maintaining reproducible configurations and results in git.
• Analyze results with Python to identify bottlenecks across the GPU, network, and SSD, and share findings in weekly team meetings.
• Collaborate with FPGA, SoC, and firmware engineers to debug the end-to-end data path and validate KV cache correctness.
• Summarize measured storage requirements, including bandwidth, IOPS, and latency, that inform future SSD controller architecture, and present findings to the team at the end of the internship.
• Complete other responsibilities as assigned.

Benefits

• Medical, dental, and vision benefits.
• Relocation stipend for employees living more than 50 miles from the new office.
• Monthly housing stipend for interns signing a lease for a new place for the summer internship.
• Charitable giving match and opportunities to get involved in the community.
• On-demand emotional wellness apps and paid therapy sessions.
• Onsite café and gym, plus virtual fitness classes.
• Flexible work environment.

What This Job Offers

• Career Level: Intern
• Education Level: No Education Listed