Data Engineer, IS&T Ai & Data Platforms
About The Position
AI & Data Platforms (AiDP) is IS&T's engine for AI-powered innovation. The team brings together data, application development, and machine learning — including generative AI — along with data services and customer success functions, to help IS&T build solutions more efficiently and streamline the adoption and embedding of generative AI across Apple. This position is required for working on the data migration and ongoing support for data platform for Applied Machine Learning enterprise systems at Apple China. The main deliverables will be: Work with SRE on setting up and testing/validating new cluster, pipelines, applications, frameworks, connectivity and privacy and governance. Identify China specific datasets, migrate the datasets to China cluster. Modify and enhance existing applications and frameworks for China cluster.
Requirements
- At least 5 years subsequent data engineering experience
- Writing advanced SQL for large scale relational data warehouses and data lakes.
- Designing data pipelines for structured and semi-structured data.
- Programing data engineering solutions in Python, Java, or Scala.
- Working with distributed frameworks including Apache Kafka and Apache Spark and Flink
- Designing reporting solutions with Tableau, Streamlit or Superset.
- At least a bachelor's degree or equivalent in Computer Science, Information Systems, or a related field.
- At least 5 years of subsequent data engineering experience across the required skills.
Responsibilities
- Work with SRE on setting up and testing/validating new cluster, pipelines, applications, frameworks, connectivity and privacy and governance
- Identify China specific datasets, migrate the datasets to China cluster
- Modify and enhance existing applications and frameworks for China cluster
- Architecting scalable data processing systems for real-time, near-real-time, and batch data pipelines.
- Develop data engineering solutions in Python and advanced SQL.
- Develop self service data engineering applications for business users.
- Design complex Python, PySpark, SQL and data processing for large-scale data platforms including Snowflake and Apache Spark with SparkSQL.
- Developing software system testing and validation procedures and documentation by writing efficient code that results in performant solutions
- Conferring with data processing or project managers to obtain and refine system requirements.
- Monitoring existing software to correct errors, adapt it to new platform requirements, and implement new functionality.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Mid Level
