Senior Manager, System Test

Celestica International LP•Austin, TX

2d•Onsite

About The Position

The Senior Manager, System Test, works with cross functional teams of design engineers, customers, supply chain and project managers to rigorously test and validate rack-level, system solutions and ensure new product release readiness. This position will manage engineers with responsibilities for System Integration Testing and Reliability. In this position you will own the test system setup, function, and validation for complex, high-end servers and hyperscale compute & network racks, including applications and operating systems. The ideal candidate will have a rack-level, system integration test background in storage/server, GPU and networking products. This person will develop and provide guidance on Celestica Storage/GPU/network system integration and reliability verification plans, test cases and test coverage, such as: functional test, hardware/software integration test, system and rack integration verification, and product reliability analysis and validation.

Requirements

Proficiency in managing teams of 8+ persons with experience in continuous improvement initiatives and establishing new processes for system test and reliability.
Proficiency in defining test plans, creating & executing tests, and generating test reports for internal and external uses.
Proven experience in test tools such as JIRA, Jenkin (CI/CD), and similar tools and proven success in implementing and maintaining processes for task/defect tracking and reporting.
Experience in troubleshooting with oscilloscopes, logic analyzers, and experienced with specialized test equipment such as PCI protocol analyzers and Traffic generators (Keysight/Ixia or Spirent)
Proficiency in project management of system integration testing. Able to manage simultaneous projects, predict workloads/execution, and deliver per plan.
Must be self-motivated and able to work independently in a fast-paced, multi-tasking environment
Experience with stress testing utilities, stressapp, stress-ng, iperf3, FIO, memtester, etc
Experience in Linux with RHEL, Ubuntu and CentOS. Understanding of x86/Firmware/Linux.
Proficiency in scripting languages: Python, Perl, VBA, REST API, Shell scripting, Ansible, Terraform etc.
Proficiency in collaborating with a cross-functional team of engineers involved in the specification, planning, design, development and test of hardware.
System/software/firmware integration and verification experience
Rack-Level System Integration Expertise: 5+ years is mandatory
Experience should include working with servers, accelerators, switches, CDU (Cooling Distribution Unit) systems, and high-power shelves.
Understanding of AI compute rack configurations and the functionality of all integrated components.
Possess a strong understanding of rack-level management functions and integration best practices
Proven experience in testing and qualifying rack-level solutions to ensure they meet or exceed performance and Reliability benchmarks.
Proficient in the management and use of AI tools and test automation
Exceptional verbal and written communication skills
Exceptional problem-solving skills and attention to detail is critical for success

Responsibilities

Design & Execute the comprehensive validation and integration of all components within advanced AI compute racks, ensuring hardware and software elements function cohesively and optimally, ensuring compatibility, connectivity, performance, security, reliability, cost effectiveness and resilience.
Lead the development and implementation of system integration test solutions and participate in project planning, scheduling, and reporting. Review and interpret product/system requirements/specifications and define system integration and validation plans.
Lead the reliability analysis and validation for new products/systems and components. Participate in project planning, scheduling, and reporting. Review and interpret product/system requirements/specifications and define validation plans.
Provide technical leadership and direction to engineers; be a recognized expert (go to person) in system integration testing and reliability analysis/testing.
System Test Setup & Execution: Oversee the set up and rigorously test product and system solutions to guarantee they meet or exceed stringent performance, reliability, and efficiency standards. Own the test system setup and function for complex, high-end servers and hyperscale compute & network racks, including applications and operating systems. Oversee the qualification and seamless integration of rack components such as servers, accelerators, switches, power shelves, and cooling solutions.
Power Distribution Management: Oversee the integration and configuration of smart PDUs (Power Distribution Units) to efficiently manage power distribution within the rack. Ensure seamless compatibility with rack-level power management functions and overall system optimization.
Rack-Level Management: Maintain a comprehensive understanding of and proficiency in rack-level management functions, including real-time monitoring, control, and automation of critical rack components.
Hands-On Racking and Stacking: Oversee the hands-on lab racking and stacking procedures, including the assembly and setup of intricate, high-touch rack configurations. Ensure meticulous installation and integration of all components.
Documentation and Reporting: Oversee the creation and maintenance of thorough and detailed documentation for setup, testing/analysis and the summarization of results. This includes documenting installation procedures, test plans, analysis/testing results, performance metrics, and any issues identified during the process.
Cross-Functional Collaboration: Own the team collaborations with a diverse range of cross functional teams, including design engineers, program managers, customers, and vendors. The goal is to ensure seamless integration, optimal & reliable operation, and the successful delivery of cutting-edge rack level solutions.
Test Automation & AI: Identify & automate procedures and testing to reduce development time, increase quality and reduce errors. Identify and use AI tools to streamline and improve efficiency of planning, executing, analyzing & reporting.
Troubleshooting & Root-Cause: Oversee troubleshooting, debugging and root-cause analysis on hardware, network, and software related issue