Ohio Supercomputer Center engineers and Dell EMC specialists are testing and preparing to deploy the center’s newest, most efficient supercomputer system, the liquid-cooled, Dell EMC-built Pitzer Cluster.
“Ohio continues to make significant investments in the Ohio Supercomputer Center to benefit higher education institutions and industry throughout the state by making additional high performance computing (HPC) services available,” said John Carey, chancellor of the Ohio Department of Higher Education. “This newest supercomputer system gives researchers yet another powerful tool to accelerate innovation.”
(hi-res photo)
Named for Russell M. Pitzer, a co-founder of the center and emeritus professor of chemistry at The Ohio State University, the Pitzer Cluster is expected to be at full production status and available to clients in November. The new system will power a wide range of research from understanding the human genome to mapping the global spread of viruses.
“The Pitzer Cluster follows the long-running HPC trend of higher performance in a smaller footprint, offering clients nearly as much performance as the center’s most powerful cluster, but in less than half the space and with less power,” said David Hudak, executive director of OSC. “This valuable new addition to our data center allows OSC to continue addressing the growing computational, storage and analysis needs of our client communities in academia, science and industry.”
The theoretical peak performance of the new Dell EMC-built cluster is about 1.3 petaflops, meaning it is capable of performing 1.3 quadrillion calculations per second. In other words, to match the potential of what the Pitzer Cluster could do in just one second, a single person would have to perform one calculation every second for 41,195,394.5 years. The cluster also can achieve seven petaflops of theoretical peak performance for mixed-precision artificial intelligence workloads.
The Pitzer Cluster will feature 260 nodes, including Dell EMC PowerEdge C6420 servers with CoolIT Systems’ Direct Contact Liquid Cooling (DCLC) coupled with PowerEdge R740 servers. In total, the cluster will include 528 Intel® Xeon® Gold 6148 processors, 64 NVIDIA® Tesla® V100 Tensor Core GPUs, all connected with EDR InfiniBand network.
“We worked with Dell EMC to create a highly efficient, dense and flexible petaflop-class system,” said Douglas Johnson, chief systems architect at OSC. “We have designed the Pitzer Cluster with some unique components to complement our existing systems and boost our total center performance to more than 2.8 petaflops.”
The Pitzer Cluster will join existing systems on the OSC data center floor at the State of Ohio Computer Center: The Dell EMC/Intel Owens Cluster (March 2017) and the HP/Intel Ruby Cluster (April 2015). The new system will replace the HP/Intel Oakley Cluster (March 2012).
“Dell EMC is thrilled to continue our great collaboration with OSC with this new dense, efficient and liquid cooled system,” said Thierry Pellegrino, vice president, Dell EMC High Performance Computing. “The Pitzer Cluster brings to bear a multitude of new technologies to help OSC and its researchers more quickly and efficiently tackle immense challenges, using artificial intelligence and deep learning to ultimately drive human progress.”
The Pitzer Cluster will utilize CoolIT Systems’ DCLC, a modular, low-pressure, rack-based cooling solution that enables a dramatic increase in rack density, component performance and power efficiency. To support the high performance requirements of the system, CoolIT’s Passive Coldplate Loop for the PowerEdge C6420 servers delivers dedicated liquid cooling to the Intel processors in each of the 256 CPU nodes, managed by a stand-alone, central pumping CHx650 Coolant Distribution Unit.
To speed up data flow within the Pitzer Cluster, Dell EMC recommended components that improve memory bandwidth on each CPU node and increase network capacity between them. The Intel processors feature 6-channel integrated memory controllers, improving bandwidth by 50 percent compared to cores in the Owens Cluster. Mellanox EDR InfiniBand 100 Gigabit per second provided provides high data throughput, low latency and high message rate of 200 million messages per second. Additionally, the smart In-Network Computing acceleration engine provides higher application performance and overall improved efficiency.
The Pitzer Cluster will provide clients with access to four Large Memory nodes (Dell EMC PowerEdge R940), with up to three terabytes of memory per node, especially helpful for data-intensive operations, such as DNA sequencing. And, the cluster’s GPU nodes (Dell EMC PowerEdge R740) feature NVIDIA® Tesla® V100 Tensor Core GPUs, which are 50 percent more energy efficient than previous generation GPUs. These GPUs offer large increases in speed, especially useful for deep learning algorithms and artificial intelligence projects.
OSC client quotes:
“OSC’s HPC resources—including GPUs—have played an essential role in the research in my lab. We have developed databases and genomic analysis tools using GPU-based alignment methods and are developing new algorithms that benefit from high-performance CPUs and GPUs as well. I believe the resources of the Pitzer Cluster will further facilitate and accelerate the research in our community, especially in analyzing and integrating big data in biomedicine.”
—Yan Zhang, an assistant professor
in the department of Biomedical Informatics
at The Ohio State University
“Deep learning has now become an essential tool to understand the human genome, the gene functions and their regulation to predict the likelihood of diseases. As a research group that has written many best-performing algorithms, we have often faced limitations of computational resources. The Tesla V100 Tensor Core GPUs on the Pitzer Cluster will help to advance our accuracy in the genomics and medical image areas and allow us to complete tasks that cannot otherwise be accomplished by lower tier cards.”
—Yuanfang Guan, Ph.D., an associate
professor of Computational Medicine &
Bioinformatics at the University of Michigan
(who collaborates with Zhang)
“Our group studies viruses that infect microbes in complex communities and the bigger the dataset, the more challenging it is to process. Currently, our major bottleneck is that we cannot even assemble some of these large-scale, fragmented metagenomic datasets, which means starting with pieces instead of ‘genomes.’ The use of existing large memory nodes has helped us map viral diversity globally in the oceans and make discoveries about viral roles in humans and soils. New, higher-memory nodes on the Pitzer Cluster will help us get unprecedented views into viruses in the wild and enable new discoveries.”
—Matthew Sullivan, Ph.D., a professor
in the department of Microbiology
at The Ohio State University
Technology partner quotes:
“CoolIT Systems, in partnership with Dell EMC, is proud to support OSC’s Pitzer Cluster with cutting-edge Direct Contact Liquid Cooling that enables high-density, energy-efficient data center thermal management,” said Patrick McGinn, vice president, business development & product marketing, CoolIT Systems. “Possessing the latest and most efficient liquid-cooling system in OSC’s history, the Pitzer Cluster will support the center’s most demanding research efforts whilst significantly reducing overall energy consumption and total cost of ownership.”
“HPC solutions are a fundamental tool to accelerate academic and scientific discovery. OSC’s Pitzer Cluster will be a leadership-class system based on the Intel® Xeon® Gold 6148 processors, delivering a strong platform for modeling, simulation, data-driven science, and will fuel scientific research and discovery for the next generation of researchers,” said Patricia Damkroger, vice president and general manager, Intel’s Data Center Group.
“By utilizing the smart InfiniBand In-Network Computing technology, the Pitzer Cluster will offer greater performance and new levels of high-performance computing and deep learning applications scalability,” said Gilad Shainer, vice president of marketing at Mellanox Technologies. “We are excited to collaborate with Dell EMC, the Ohio Supercomputer Center and the rest of the partners to bring an innovative research platform to OSC clients.”
“OSC’s Pitzer cluster will enable Ohio’s university and industry researchers to drive a new wave of scientific discovery,” said Ian Buck, vice president and general manager of Accelerated Computing at NVIDIA. “Equipped with NVIDIA® V100 GPUs that provide universal acceleration for scientific computing, AI and machine learning applications, researchers using Pitzer will have direct access to the kind of computing power required to advance their work already underway on genomics analysis, medical imaging and much more to come.”
The Ohio Supercomputer Center (OSC), a member of the Ohio Technology Consortium of the Ohio Department of Higher Education, addresses the expanding computational demands of academic and industrial research communities by providing a robust shared infrastructure and proven expertise in advanced modeling, simulation and analysis. OSC empowers researchers with the vital services essential to make extraordinary discoveries and innovations, partners with businesses and industry to leverage computational science as a competitive force in the global knowledge economy and leads efforts to equip the workforce with the key technology skills required to secure 21st century jobs. For more, visit www.osc.edu.
