Is the InfiniBand Bandwagon Actually Growing?

by David Gross

Looks like UK IT magazine The Register is now drinking the InfiniBand Kool-Aid.   They're excited about the InfiniBand Trade Association (IBTA) roadmap to 312 Gbps, and how much faster this will be than the recently ratified 100 Gigabit Ethernet.

One of the points I've been making to people looking at the costs of these technologies is that the defining economic trait
of multi-gigabit networks is clocking, while the defining economic trait of sub-gigabit networks was framing.    What I mean by this is that Ethernet's takeover of networking one gig and down was based on achieving lower costs through extremely high production of framing chips.     This created absurd financial disparities in the past, where Ethernet was often much cheaper than lower rate competitors.   The most glaring case was in the late 90s when 100 Megabit Fast Ethernet adapters starting selling for the unbeatable price of free, and 25 Meg ATM-to-the-Desktop cards were going for $300.

Over a gigabit, high production volume of framing chips gives way to the challenges of clocking, which occur at both the on-chip and network levels when each bit is less than one billionth of a second away from its neighbors.   Many of the underlying technologies that address this issue are the same, and don't allow for one framing protocol to undercut another dramatically in price just because it's shipped in higher volumes.   As a result, a niche technology can hang around, because it shares the 8B/10B encoding, LVDS signaling, and other techniques used to address bits that are bunched so close together.   So over a gigabit, link distance takes over as the major cost factor, and by spending so much effort on 40 and 10 kilometer port types, the IEEE has made Ethernet all things to all people, which has opened up sustainable niches for InfiniBand, particularly in supercomputing and financial trading.

According to cable manufacturer Panduit, virtually all data center links are under 70 meters, and as a practical matter, I'd say a large share are under 20.   This means, a tightly-defined, short-reach interface like InfiniBand should continue to succeed in the data center, as long as the IBTA, as well InfiniBand vendors like Mellanox (MLNX) and Voltaire (VOLT),  do not give in to any temptations to have it compete with the broader networking applications served by Ethernet.