The InfiniBand Trade Association (IBTA) has announced a new roadmap for the low latency bi-directional link. What was Eight Data Rate InfiniBand will now be known as Enhanced Data Rate (EDR) and be upgraded from 20Gbps to 26Gbps per channel. EDR will deliver a 104Gbps data rate in a four-channel InfiniBand (4x) implementation and 312Gbps when used as a 12-channel (12x) interface.
“We can already demonstrate in silicon a 30Gbps transmitter."
Marek Tlalka, Luxtera
“Our June 2008 roadmap originally projected 4x EDR at less than 80Gbps data rate for 2011,” says Skip Jones, director of technology at QLogic and co-chair of the IBTA’s marketing working group. “The IBTA has increased the data speeds for 2011 due to demand for higher throughput.” A 26Gbps channel rate - or 104Gbps for 4x EDR - is to accommodate the overhead associated with 64/66bit encoding.
The IBTA has also added an interim speed, dubbed Fourteen Data Rate (FDR), operating at 14Gbps per channel or 56Gbps for 4x FDR. This, says the IBTA, is to address midrange enterprise applications in the data centre. “Many server OEMs’ backplanes can support speeds up to 56Gbps,” says Jones. “For those OEMs doing a server refresh using existing backplanes, 56Gbps will be the solution they’ll be looking to implement.”
The IBTA dismisses claims by some industry voices that the re-jigged roadmap is to stop InfiniBand falling behind 100 Gigabit Ethernet (GbE) while FDR is to advance InfiniBand while laser vendors grapple with the challenge of developing 26Gbps vertical-cavity surface-emitting lasers (VCSELs) for EDR.
Jones points out that 4x Quad Data Rate (QDR) InfiniBand (4x10Gbps) now accounts for between 60 and 70 percent of newly deployed InfiniBand systems, and that 100Gbps EDR will appear in 2011/ 2012. “The IBTA has a good track record of releasing products on time; as such, 100Gbps InfiniBand will come out much faster than 100 Gigabit Ethernet.” FDR, meanwhile, will benefit from 14Gbps VCSELs for Fibre Channel that will be available next year. Jones admits that developing a 26Gbps VCSEL poses a challenge but that “InfiniBand markets are mostly electrical interconnects”.
“The 4x25G short reach is not going to rise and dominate for quite awhile."
Scott Schube, LightCounting
“VCSELs are going to have a tough time at 26Gbps per lane, though they'll get there,” says Scott Schube, senior analyst and strategist at optical transceiver market research firm, LightCounting. “There's definitely a push to go to 26Gbps per lane to reduce pin counts, and the chip guys look like they will be ready before the VCSELs.”
One company looking to benefit from the emerging market for EDR is Luxtera. The silicon photonics specialist says its modulator has already been demonstrated at 30Gbps. This is fast enough to accommodate EDR, 100 Gigabit Ethernet (a 4-channel design) and the emerging 28Gbps Fibre Channel standard.
“We can already demonstrate in silicon a 30Gbps transmitter using the same laser as in our existing products and modulated in our silicon waveguides,” says Marek Tlalka, vice president of marketing at Luxtera. “That allows us to cover 14Gbps, 26Gbps EDR, parallel Ethernet as well as 28Gbps for serial Fibre Channel.”
Luxtera will need to redesign the transistor circuitry to drive the modulator beyond the current 15Gbps before the design can be brought to market. It will also use an existing silicon modulator design though the company says some optimisation work will be required.
There are two main product offerings from Luxtera: QSFP-based active optical cables and OptoPHY, one and four-channel optical engines. Luxtera’s OptoPHY product is currently being qualified and is not yet in volume production.
For multi-channel designs, Luxtera uses a continuous-wave 1490nm distributed feedback (DFB) laser fed to the modulated channels. Addressing 28Gbps Fibre Channel, an SFP+ form factor will be used. Luxtera may offer a transceiver product or partner with a module maker with Luxtera providing the optical engine. “It’s an open question,” says Tlalka.
“The IBTA has a good track record of releasing products on time; as such, 100Gbps InfiniBand will come out much faster than 100 Gigabit Ethernet.”
Skip Jones, IBTA
The company has said that the single-channel and four-channel 10Gbps OptoPHY engine consumes 450mW and 800mW respectively. Going to 26Gbps will increase the power consumption but only by several tens of percent, it says.
The first product from Luxtera will be a pluggable cable followed by a companion OptoPHY. The pluggable active optical cable from Luxtera will support 100GbE and EDR Infiniband. “I’d still place my bets on InfiniBand deploying first followed by 100GbE,” says Tlalka.
But Schube warns that Luxtera faces a fundamental challenge “Leading-edge designs based on proprietary technology to solve commodity problems - more bandwidth for out-of-the-box connections - are never going to get widely adopted, though Luxtera can fill a niche for awhile," he says.
There is also much work to be done before 100Gbps interfaces will be deployed. “The 4x25G short reach is not going to rise and dominate for quite awhile, no matter what the component availability is,” says Schube. That is because switch ASICs, backplanes, connectors and line cards will all first need to be redesigned.
Meanwhile the IBTA has also announced two future placeholder data rates on its InfiniBand roadmap: High Data Rate (HDR) due in 2014 and the Next Data Rate (NDR) sometime after. “We will refrain from identifying the exact lane speed until we are closer to that timeframe to avoid confusion and the possibility - and probability - of changing future lane speeds,” says Jones.
And Luxtera says its modulator can go faster still. “I think we can easily go 40 and 50Gbps,” says Tlalka. “After 50Gbps we’ll have to look at new magic.”