More scientists are utilizing high- performance computing (HPC) in their research programs as they test models to validate their theories. This research is becoming more data-intensive, drawing on a growing body of data from many sources. For Perimeter to continue studying the mysteries of the universe, they needed to move beyond their aging HPC environment, which was too small, too slow, and not meeting the needs of their researchers. To attract and keep top researchers, Perimeter wanted to take advantage of recent advances in HPC computing. As a not-for-profit charitable institute, Perimeter was focused on delivering a cost-effective solution that could be easily managed by an IT staff without specialty HPC skills.
Perimeter’s old HPC system required a lot of manual work for management and administration, and they struggled to manage storage resources – something not anticipated in the original design. Ben Davies, Perimeter’s Director of Information Technology, says, “It’s one thing to calculate and parse data at very high rates in a fast set of servers, but if you can’t store that data quickly enough then you slow down the whole computing process.”
Perimeter started their HPC upgrade journey by contacting OEM vendors but quickly ran into difficulties comparing different vendor offerings. “Buying HPC systems is easier today because the hardware has become commoditized,” Davies explains. “But it’s complicated by the trade-offs required to invest a fixed budget in the appropriate balance of compute, network, and storage capacity to meet specific workload targets.”
Davies says that much of the complexity stems from potentially tying hundreds of servers together over two different types of networks to get the required level of control, particularly in the storage architecture. “All these things have to be balanced in terms of how they’re connected, operating, and optimized,” he says.
Scalar has a deep understanding and a great breadth of experience in this area – research and academia. They demonstrated various products and services that might be relevant to us now or in the future which really opened our eyes to some utilities, techniques, and services that we should be looking at downstream.
Faced with this level of complexity, Perimeter decided to seek outside help with advanced knowledge in the current HPC product landscape and the expertise to design and implement a system to meet their power and budget needs.
To fill the expertise gap, Perimeter engaged Scalar to provide guidance on a hardware, software, and networking solution. “When we found Scalar, we quickly realized they had extensive, invaluable, and relevant experience in navigating this very complex landscape,” Davies says. “They could bring a lot of value to us in helping select the technology and – just as importantly – design and implement the system. What looks on the surface to be a simple hardware acquisition is actually a very complex set of interrelated design questions.”
Scalar worked with Perimeter to help the team navigate the complex world of leading-edge research HPC technology, recommending effective systems management tools to streamline HPC administration. Additionally, Scalar designed and implemented the production system.
Scalar helped Perimeter identify the right vendors and right products to meet the Institute’s objectives, and then configured the system around specific use cases. Davies says, “Scalar has a deep understanding and a great breadth of experience in this area – research and academia. They demonstrated various products and services that might be relevant to us now or in the future which really opened our eyes to some utilities, techniques, and services that we should be looking at downstream.”
For hardware, the team selected Dell EMC as the best solution for computing, Dell and Mellanox for networking equipment, and DDN GridScalar for storage.
To manage the hardware, operating system, HPC software, and users, the team selected Bright Cluster Manager for HPC from Bright Computing, which automates much of the administration and management tasks.
Davies says, “Scalar helped us work through these decisions in a way that the OEM vendors were not able. They brought invaluable practical experience from a wide range of similar implementations. They provided essential context from real-world experience to help us make smart choices. They helped us find the right architecture and the right storage technology at the right price, and that was very important. Scalar was very helpful in showing us new and improved software tools that really streamlined management of the equipment. The area where Scalar provided the highest level of value was in helping us come up with a design that was exceptionally useful and met all of our various needs.”