Intern, AI Application Engineer

Samsung Semiconductor • Folsom, CA

4d • $28 - $68 • Onsite

About The Position

The SoC Architect team proposes SoC-level architecture candidates for emerging memory and storage solutions and demonstrates their viability through hands-on RTL implementation. This internship offers the opportunity to explore next-generation SoC and subsystem architectures through practical work in LLM inference, storage systems, benchmarking, and end-to-end validation. The position is based at Samsung Semiconductor’s San Jose, California headquarters and requires daily onsite presence, in alignment with the company’s Flexible Work policy. The intern reports to the Senior Director.

The intern will deploy and operate an LLM inference service using vLLM to serve Llama-3.3-70B on a four-NVIDIA-H200-GPU server. They will build reproducible benchmarking experiments, investigate KV-cache offloading to network-attached storage, analyze system performance, collaborate with FPGA, SoC, and firmware engineers, and report storage requirements that can inform future SSD controller architectures.

Requirements

• Pursuing a bachelor’s, master’s, or PhD degree in Computer Science, Computer Engineering, Electrical Engineering, or a related field.
• Must have at least one academic quarter or semester remaining.
• Hands-on experience running LLM inference with Python and PyTorch on NVIDIA GPUs under Linux, using a serving framework such as vLLM, SGLang, or Hugging Face TGI.
• Understanding of LLM serving internals, including KV cache, prefix caching, batching, tensor and pipeline parallelism, and latency metrics such as TTFT, TPOT, and throughput.
• Experience benchmarking and profiling system performance and analyzing results with scripts such as Python, pandas, and plotting tools to explain bottlenecks.
• Inclusive and able to adapt communication style to situations and diverse global norms.
• Curious and resilient learner who seeks data to build understanding.
• Collaborative approach to building relationships, offering support, and welcoming different approaches.
• Innovative and creative, proactively exploring new ideas and adapting quickly to change.

Nice To Haves

• Familiarity with KV-cache offloading tools such as LMCache or NVIDIA Dynamo/NIXL.
• Familiarity with RDMA/RoCE or NVMe/NVMe-oF storage.
• Familiarity with containers such as Docker or Kubernetes.

Responsibilities

• Deploy and operate a vLLM inference service serving Llama-3.3-70B on a 4 × NVIDIA H200 GPU server, working with a mentor and the storage architecture team.
• Build an automated benchmark harness to replay long-context, multi-turn, and agent-style request traces and record latency and throughput metrics, including TTFT, TPOT, and goodput.
• Configure KV-cache offloading using LMCache and NVIDIA Dynamo/NIXL to network-attached storage over NVMe-oF and RDMA, first on a BlueField-3 storage server and then on a network-attached SSD prototype.
• Run daily experiment sweeps across request rates, context lengths, and cache sizes, tracking configurations and results reproducibly in git.
• Analyze benchmark results with Python to identify bottlenecks across the GPU, network, and SSD, and share findings in weekly team meetings.
• Collaborate with FPGA, SoC, and firmware engineers to debug the end-to-end data path and validate KV-cache correctness.
• Summarize measured storage requirements, including bandwidth, IOPS, and latency, that shape future SSD controller architecture, and present findings to the team at the end of the internship.
• Complete other responsibilities as assigned.

Benefits

• Medical, dental, and vision benefits.
• Relocation stipend for employees moving from more than 50 miles away.
• Monthly housing stipend for interns signing a lease for the summer internship.
• Charitable giving match and opportunities to get involved in community activities.
• On-demand emotional wellness apps and paid therapy sessions.
• Onsite café and gym, plus virtual fitness classes.
• Flexible work environment.

What This Job Offers

• Career Level: Intern
• Education Level: No Education Listed