Beamline Controls Engineer

Lawrence Berkeley National Laboratory • Berkeley, CA

6d • $179,000 - $218,000 • Onsite

About The Position

Lawrence Berkeley National Laboratory is hiring a Beamline Controls Engineer within the Engineering division. This position is responsible for the computing, virtualization, network, storage, and EPICS control system infrastructure that ALS beamline operations rely on. The incumbent works independently and decides the methods and approaches for significant and often new problems. This work runs from the server, storage, and network platforms to instrument control and data acquisition, and the incumbent keeps these systems reliable, secure, and maintainable so that experiments can run. The incumbent also sets infrastructure and controls standards, coordinates work across beamline groups, other Engineering functions, other facilities, and vendors, and provides technical direction and mentoring to other staff. The incumbent is a recognized expert whose technical judgment guides infrastructure decisions for the controls environment. We’re here for the same mission, to bring science solutions to the world. Join our team and YOU will play a supporting role in our goal to address global challenges! Have a high level of impact and work for an organization associated with 17 Nobel Prizes!

Requirements

• Bachelor’s Degree, or equivalent, in engineering or related field.
• 12+ years of relevant technical experience with Bachelor’s degree, 8 with a Master’s degree, or equivalent experience
• Extensive experience building and running Linux server, virtualization, storage, and network infrastructure in a scientific or research computing environment that runs continuously.
• Strong hands-on Linux systems engineering, including security hardening and managing servers through their full life, from build to retirement.
• Experience running virtualization and clusters for production work, with shared storage and high availability. Proxmox VE experience applies directly.
• Solid storage and file system work, including disk arrays, networked storage, high-speed I/O, and backup and recovery that has been tested.
• Practical networking and access control in a network that is segmented and used for several purposes, including routing, VLANs, DNS, VPN, firewalls, and account and permission management.
• Good scripting and configuration management skills for automation, for example Python and shell with a configuration management tool such as Puppet or Ansible.
• Knowledge of system monitoring, and a track record of keeping systems running at high availability around the clock.
• Good independent judgment on significant and new high-stakes problems, and the ability to work with separate groups and outside organizations to agree on solutions and standards.
• Ability to explain complex technical choices clearly to scientists, engineers, and management.

Nice To Haves

• Experience with EPICS instrument control and data acquisition; experience supporting a synchrotron, accelerator, or large scientific user facility; experience connecting motion control, detectors and cameras, and PLC I/O to a control system.
• EPICS instrument control, including motion control with commercial controllers and precision actuators, and connecting detectors, cameras, and PLC I/O.
• Data acquisition software and operator screen development in Python and in web front ends.
• Relational and in-memory database administration.
• Cloud and container experience.
• Experience at a synchrotron, accelerator, or other large scientific user facility, or at a DOE lab.

Responsibilities

• Install, configure, and maintain the Linux servers and virtualization platforms that run the beamline control systems, scientific computing, and shared services, including Proxmox VE high-availability clusters.
• Manage the configuration and capacity of these systems, and apply operating system updates, firmware updates, and security fixes across the supported machines.
• Build and maintain the storage systems and the file systems behind them, including disk arrays, networked storage, and the high-speed connections that move data to and from instruments.
• Maintain the backup and disaster recovery systems that protect operational and experiment data.
• Manage the network and access controls in the controls environment. This includes keeping control, research, and management traffic separate, and running DNS, remote access, firewalls, and the accounts and permissions that control who can reach each system.
• Use scripting and configuration management tools to automate builds and configuration, so that systems can be rebuilt and recovered reliably.
• Write, deploy, and support EPICS control software for motors, optics, sensors, detectors, and analog and digital I/O on the beamlines. Connect commercial motion controllers, piezo and other precision actuators, and PLCs to the control system.
• Build and run the data acquisition software and the experiment data systems that sit on top of the control system, along with the operator screens scientists use to run and watch their experiments.
• Add new instruments and devices to the group’s standard control software, solve the hardware and software integration problems that come up, and turn the results into tools the team can reuse.
• Serve as the senior technical person for the largest and least defined infrastructure and controls problems, where there is no set procedure and the best choice depends on the effect on research and on how well the system can be supported over time.
• Set and maintain standards for configuration, documentation, security, and day-to-day operations, and help other staff bring their work in line with them.
• Work with beamline groups, operations, other Engineering functions, other DOE labs, and vendors to agree on shared solutions and standards, and represent the group in meetings that cross divisions and facilities.
• Give technical direction and mentoring to other staff, and when assigned, lead the technical work on a project or direct the work of others.
• Collaborate with engineers and scientists. Direct the work of other coordinators, designers, technicians, and maintenance personnel.
• Support projects by contributing to the creation of cost estimates, budgets, schedules and to the procurement of key components.
• Maintain compliance with applicable LBNL policies and standards (Engineering Process, Electrical Equipment Inspection Program, Safety Software Quality Assurance, safety protocols, respectful workplace, etc.).
• Contribute to the development of the department’s mission and committed to achieving it in a safe and respectful manner, in compliance with applicable standards and regulations.

Benefits

• Exceptional health and retirement benefits, including pension or 401K-style plans
• Opportunities to grow in your career - check out our Tuition Assistance Program
• A culture where you’ll belong - we are invested in our teams!
• In addition to accruing vacation and sick time, we also have a Winter Holiday Shutdown every year.
• Parental bonding leave (for both mothers and fathers)
• Pet insurance

What This Job Offers

• Job Type: Full-time
• Career Level: Senior