New TACC Supercomputer Running 40G InfiniBand

by David Gross

The Texas Advanced Computing Center is expanding its Lonestar Supercomputing Cluster with a new system that will feature 22,656 compute node cores, up from 5,840 in the existing system, connected with a 40G Mellanox (MLNX) InfiniBand network that uses a fat-tree topology.

The new system will feature:

* 302 teraflops peak performance vs. 63 teraflops on the existing Lonestar
* 44.3 terabytes total memory vs. 11.6 TB on the existing system
* 1.2 petabytes raw disk vs. 106 TB on the existing system
* 90 million CPU hours/year to TeraGrid

What's interesting here is that processing capacity and disk space are all rising at a faster rate than maximum I/O speed, which has risen a fairly sluggish 4-fold in the last eight years. This suggests that the I/O bottleneck is likely to become an even larger problem in supercomputing as time goes on.