Data Engineer, AI Platform

Tiktok • San Jose, CA

32d

About The Position

The Intelligent Creation - AI Platform team is a team focusing on building advanced end-to-end AI production pipelines, including deep learning model training, optimization, deployment and applications. We provide AI capabilities to empower content creation and consumption on TikTok and serve billions of users.

Requirements

• 3+ years of experience building data services; strong proficiency in Python.
• Experience with high-concurrency and asynchronous programming is a plus.
• Hands-on with Hive, MySQL, MongoDB, and Elasticsearch; solid understanding of internals; capable of data abstraction and data modeling.
• Practical experience with large-scale data processing frameworks such as Hadoop, Spark, Flink, and Ray.
• Excellent communication and collaboration skills; detail-oriented; strong problem-solving and analytical abilities.

Nice To Haves

• 1-2 years of experience with the Ray framework, including proficient orchestration of GPU/CPU resources; deep understanding of Ray's architecture and usage patterns; strong background in high-concurrency and async processing to boost overall throughput.
• Experience sourcing and curating training datasets for GenAI.

Responsibilities

• Own the management of massive GenAI datasets (PB) scale, including storage for video/image data, data processing, and data validation.
• Build and evolve GenAI data-pipeline infrastructure with a focus on extreme processing speed and end-to-end throughput.
• Partner closely with ML researchers/engineers to accelerate training-data acquisition, improve data quality, support evaluation of model outputs, and enable a closed-loop data lifecycle.
• Lower the barrier to data acquisition and maximize the utility and value of data across use cases.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Industry: Broadcasting and Content Providers
• Education Level: No Education Listed
• Number of Employees: 5,001-10,000 employees