Lead Software Engineer in Test, Siri AI Quality Engineering

Apple•Cupertino, CA

About The Position

Are you passionate about creating ground-breaking technology for the next generation of Siri products that will usher in the next revolution of human-computer interaction? Join the Siri AI Quality Engineering team at Apple and play a key part in shipping groundbreaking technology for large-scale systems in natural language and artificial intelligence. Our products inspire and delight over a billion people worldwide! We are looking for a Lead Software Development Engineer in Test with a strong background in machine learning, audio & visual technologies, as well as conversational and generative AI. This Engineer will be instrumental in upholding the quality of AI/ML-powered products. Our team crafts and builds evaluation tools, automation frameworks, acoustic evaluation labs, and methodologies that enable product teams across Apple to develop ML-based solutions that power incredibly intelligent user experiences. You will engage with pioneering new-product teams around Apple; bringing your expertise and passion for innovation to solving technical problems for our next-generation products. DESCRIPTION As a technical lead, this role owns the qualification at scale of Siri new experiences; Bringing up automation on-device and virtualization as a critical path to shipping AI customer-facing products and features. The primary responsibilities associated with this position range from data insights, design and implementation of product quality measures, ability to convert research findings into product quality improvements or frameworks, and close collaboration with product teams (both before & after product/feature launches). Our team plays a pivotal role in testing, maintaining, and shipping artifacts/features to make Siri and other AI/ML-related products even more empowering for our customers. You will also: - Leverage innovative AI models to build applications and testing solutions on top of Apple’s most sophisticated technologies - Write clean, maintainable and production code with appropriate documentation and tests - Contribute to architecture decisions, design reviews and peer code reviews - Design and implement testing solutions, automation, and build AI models for quality insights and testing frameworks - Partner closely with other cross-functional teams to align research, messaging, and product roadmaps - all in the service of developing world-class products - Prototype and test new insights to bring your research to life

Requirements

BS or MS or PhD in Computer Science or Electrical Engineering with strong coding & Machine Learning skills and 10+ years of proven experience in industry
4+ years designing and architecting tools and frameworks
Expertise dealing with datasets and performing analysis for product quality insights
Proven depth of knowledge and application of statistics-based evaluation methodologies or user success metrics
Deep Understanding of ML-based concepts; including awareness of how data is used to support ML, experimentation, and analytics
Experience analyzing user behavior, deriving patterns, and designing test strategies & automation frameworks
Deep understanding of Prompt Engineering and Build Prompt fine-tuning
Understanding of generative AI and experience using it for various applications

Nice To Haves

Experience with conversational AI technologies and frameworks
Familiarity with Apple's research and products
Experience with audio signals and analysis
Experience with computer vision
Experience with robotics and automation

Responsibilities

Own the qualification at scale of Siri new experiences
Bring up automation on-device and virtualization as a critical path to shipping AI customer-facing products and features
Data insights, design and implementation of product quality measures
Convert research findings into product quality improvements or frameworks
Close collaboration with product teams (both before & after product/feature launches)
Leverage innovative AI models to build applications and testing solutions on top of Apple’s most sophisticated technologies
Write clean, maintainable and production code with appropriate documentation and tests
Contribute to architecture decisions, design reviews and peer code reviews
Design and implement testing solutions, automation, and build AI models for quality insights and testing frameworks
Partner closely with other cross-functional teams to align research, messaging, and product roadmaps
Prototype and test new insights to bring your research to life