Students get a taste of ORNL’s large-scale data centers with online tours and virtual lectures from UT and ORNL staff

By Rachel McDowell, ORNL. Photography by Carlos Jones, ORNL

What does it take to manage a robust data center? And how can engineers get the experience needed to oversee both the facilities—the power, cooling, and electrical components—and the computing aspects, such as hardware and performance?

For 6 years, the US Department of Energy’s (DOE’s) Oak Ridge National Laboratory (ORNL) has partnered with the University of Tennessee, Knoxville (UT), to offer a cutting-edge course in data center design to address these questions. Part of UT’s data center technology and management minor, the interdisciplinary course features instructors from both UT and ORNL who teach students how to navigate the facilities and the computational resources in large-scale data centers. The course aims to give students a close-up look at some of the most complex data centers in the nation—those located in the heart of East Tennessee at ORNL—and the opportunity to design their own data centers using CAD software for drawing 2D and 3D blueprints.

This year, though, the COVID-19 pandemic forced the course into a distance-learning format. Luckily, the organizers already had a plan.

“In the 2019 fall semester, we had a student who was in Erwin, Tennessee, and he worked full-time and couldn’t make it to class,” said course professor David Icove, who is the coordinator of the fire protection engineering graduate concentration at UT. “We sent him the videos, and he stayed up to date and completed all the assignments. His success convinced us that a distance-learning approach would be worthwhile even before the COVID-19 pandemic.”

The student’s project was deemed an “exemplary design,” a first for the course. The student, John Wade, completed the project individually—another unusual feat—while working as a chief electrical engineer at Nuclear Fuel Services. The class is typically divided into groups of four or five to give students a chance to put their heads—and majors—together to design a data center. As a result of his success in the class, Wade was recently hired at ORNL as a senior electrical systems engineer working in the Laboratory Modernization Division after graduating from UT at the end of 2020 with his doctorate in electrical engineering with a concentration in fire protection.

“The most useful part of the class for me personally was the comprehensive look at everything that’s involved in a data center design—getting everything on the table and then integrated,” Wade said. “That’s exemplified in the challenges I had with the project. I went through a number of iterations on that project, where, as more lecture material came along, I’d learn something new and have to go back and revisit material from earlier in the project.”

This year, student majors in the course ranged from computer science to electrical, computer, and industrial engineering. The course featured 10 instructors from ORNL and UT, many of whom have lectured for the course in the past.

“Our UT-ORNL partnership leverages the talent of both organizations to deliver a quality and resilient course in the design of high-performance data centers,” said Ozlem Kilic, the associate dean for academic and student affairs in the Tickle College of Engineering at UT.

Despite the virtual format, the class still covered the full range of topics—data center design concepts, cooling, power, high-performance computing, networking and cabling, construction safety, and more. The instructors prerecorded the lectures, taking advantage of green-screen technology that allowed them to showcase presentations behind them during the lectures.

“Organizing and lecturing a virtual class definitely taught us new ways of presenting information,” said Bart Hammontree, a technical project manager in the Laboratory Modernization Division at ORNL who co-led the course with Icove. “We are actually going to use some of the things we learned in upcoming DOE project reviews as well.”

During a traditional semester, the students visit ORNL to tour data centers at the Oak Ridge Leadership Computing Facility (OLCF), a DOE Office of Science User Facility located at ORNL. These tours include a walkthrough of both the OLCF’s 200-petaflop Summit, the nation’s fastest supercomputer, and an existing room in one of the OLCF’s data centers that serves as the setting for the students’ CAD designs. However, with tours suspended at ORNL in the wake of the COVID-19 pandemic, the course organizers had to adapt.

Katie Bethea, Outreach and Communication group leader in the National Center for Computational Sciences, and ORNL photographer Genevieve Martin worked with staff in ORNL’s Creative Services Division to create virtual, interactive tours of the Summit data center and the “buildout” setting for the students’ data center designs. ORNL has added several interactive tours of its facilities and landmarks.

“We also narrated a video of the data center tours so that the students could watch them narrated or move through them on their own and look at the various callouts that are available,” Hammontree said. “This is a unique opportunity because now, they can go back after the tour if they want to see something new.”

The success of the virtual course highlights the ability of the organizers to reach a wider audience—a move that would open up opportunities for students around the country and possibly around the world to participate in the course.

“We’d really like to make this available to the other DOE lab facilities around the country,” Icove said. “We could see 100 to 200 registrants for a virtual class like this, and since it’s virtual, we wouldn’t have to worry about capping it.”

Icove also mentioned that translating the recorded lectures would make the course accessible to people in other countries.

“These lectures are all closed-captioned, so if someone wanted these in Spanish, Portuguese, Arabic, French, and so on, we could have them translated,” Icove said.

Icove said that the virtual class will run smoothly in the future because the lectures have already been recorded.

“The vision is to run this course every semester, and now that we’ve archived the lectures, we just have to fine-tune some of them and supplement them,” Icove said.

Other course instructors included UT’s Michela Taufer and ORNL’s Paul Abston, Saeed Ghezawi, David Grant, Rick Griffin, Jimmy Landmesser, Jim Rogers, and Jeremy Underwood.

To take a virtual tour of Summit, visit https://my.matterport.com/show/?m=iBfbj7ET4LT&nozoom=1&help=1.

 

UT-Battelle LLC manages Oak Ridge National Laboratory for DOE’s Office of Science, the single largest supporter of basic research in the physical sciences in the United States. DOE’s Office of Science is working to address some of the most pressing challenges of our time. For more information, visit https://energy.gov/science.