Choosing paths to future Gigabit Ethernet speeds
The y-axis shows the number of lanes while the x-axis is the speed per lane. Each red dot shows the Ethernet rate at which the signalling (optical or electrical) was introduced. One challenge that John D'Ambrosia highlights is handling overlapping speeds. "What do we do about 100 Gig based on 4x25, 2x50 and 1x100 and ensure interoperability, and do that for every multiple where you have a crossover?" Source: Dell
One catalyst for these discussions has been the progress made in the emerging 400 Gigabit Ethernet (GbE) standard which is now at the first specification draft stage.
“If you look at what is happening at 400 Gig, the decisions that were made there do have potential repercussions for new speeds as well as new signalling rates and technologies,” says John D’Ambrosia, chairman of the Ethernet Alliance.
Before the IEEE P802.3bs 400 Gigabit Ethernet Task Force met in July, two electrical signalling schemes had already been chosen for the emerging standard: 16 channels of 25 gigabit non-return-to-zero (NRZ) and eight lanes of 50 gigabit using PAM-4 signalling.
For the different reaches, three of the four optical interfaces had also been chosen, with the July meeting resolving the fourth - 2km - interface. The final optical interfaces for the four different reaches are shown in the Table.
The adoption of 50 gigabit electrical and optical interfaces at the July meeting has led some industry players to call for a new 50 gigabit Ethernet family to be created, says D’Ambrosia.
Certain players favour the 50 GbE standard to include a four-lane 200 GbE version, just as 100 GbE uses 4 x 25 Gig channels, while others want 50 GbE to be broader, with one, two, four and eight lane variants to deliver 50, 100, 200 and 400 GbE rates.
If you look at what is happening at 400 Gig, the decisions that were made there do have potential repercussions for new speeds as well as new signalling rates and technologies
The 400 GbE standard’s adoption of 100 GbE channels that use PAM-4 signalling has also raised questions as to whether 100 GbE PAM-4 should be added to the existing 100 GbE standard or a new 100 GbE activity be initiated.
“Those decisions have snowballed into a lot of activity and a lot of discussion,” says D’Ambrosia, who is organising an activity to address these issues and to determine where the industry consensus is as to how to proceed.
“These are all industry debates that are going to happen over the next few months,” he says, with the goal being to better meet industry needs by evolving Ethernet more quickly.
Ethernet continues to change, notes D’Ambrosia. The 40 GbE standard exploited the investment made in 10 gigabit signalling, and the same is happening with 25 gigabit signalling and 100 gigabit.
If you buy into the idea of more lanes based around a single signalling speed, then applying that to the next signalling speed at 100 Gigabit Ethernet, does that mean the next speed with be 800 Gigabit Ethernet?
With 50 Gig electrical signalling now starting as part of the 400 GbE work, some industry voices wonder whether, instead of developing one Ethernet family around a rate, it is not better to develop a family of rates around the signalling speed, such as is being proposed with 50 Gig and the use of 1, 2, 4 and 8 lane configurations.
“If you buy into the idea of more lanes based around a single signalling speed, then applying that to the next signalling speed at 100 Gigabit Ethernet, does that mean the next speed with be 800 Gigabit Ethernet?” says D’Ambrosia.
The 400 GbE Task Force is having its latest meeting this week. A key goal is to get the first draft of the standard - Version 1.0 - approved. “To make sure all the baselines have been interpreted correctly,” says D’Ambrosia. What then follows is filling in the detail, turning the draft into a technically-complete document.
Further reading:
LightCounting: 25GbE almost done but more new Ethernet options are coming, click here
An interview with John D'Ambrosia
The chairman of the Ethernet Alliance talks to Gazettabyte about the many ways Ethernet is evolving due to industry requirements.
"We are witnessing the evolution of Ethernet in ways that many of us never planned because there are markets that are demanding different things from it."
John D'Ambrosia describes the industry as feeling like nuts right now. "There is just so much stuff going on in terms of Ethernet," he says.
Besides the development of 400 Gigabit Ethernet (GbE) - the specification work for the emerging Ethernet standard being well underway - new applications are creating requirements that the existing Ethernet specifications cannot meet. These requirements include additional Ethernet speeds; the IEEE 802.3 Ethernet Working Group has created a Study Group to develop single-lane 25GbE for server interconnect.
One busy Ethernet activity involves 100 Gig mid-reach interfaces. Mid-reach covers distances from 500m to 2km. The interfaces are needed in the data centre to connect switches, such as the leaf-spine switch architecture, and to connect switches to the data centre's edge router. The existing IEEE 802.3 Ethernet 100 Gig multi-mode standards - the 100GBASE-SR4 and the 100GBASE-SR10 span 100m only (150m over OM4 fibre), too short for certain data centre applications.
"As we go faster, multimode's reach capabilities are coming down," says D'Ambrosia. "It has got to do with those pesky laws of physics." The next IEEE 802.3 100 Gig interface option, 100GBASE-LR4, has a 10km span, too much for many data centre applications. The 100GBASE-LR4 is also expensive, seven times the cost of the 100GBASE-SR4 interface, according to market research firm, LightCounting.
One of the reasons the IEEE 802.3 Ethernet Working Group created the 802.3bm Task Force was to develop an inexpensive 500m-reach specification. Four proposals resulted: parallel single mode (PSM4), coarse WDM (CWDM), pulse amplitude modulation and discrete multi-tone. None were adopted since each failed to muster sufficient backing. The optical industry then pursued a multi-source agreement (MSA) approach, and since January 2014, four single-mode mid-reach interfaces have emerged: the CLR4 Alliance, the CWDM4, the PSM4 and OpenOptics.
D'Ambrosia says the mid-reach optics debate first arose in 2007 when the IEEE 802.3ba group, developing 40 GbE and 100 GbE standards, discussed whether a 3-4km 100 Gig reach interface was required. "There was still enough people that needed 10km," says D'Ambrosia, and if 3-4km had been chosen then the 10km requirement would have been addressed with an even more complex 40km interface. "In hindsight, I'm not sure that was the right decision but it was the right decision at the time," says D'Ambrosia.
The PSM4 100 GbE mid-reach MSA used four individual fibres for each direction, each fibre operating at 25 Gig. The other three mid-reach interfaces have a 2km reach and use 4x25 Gig wavelengths and duplex fibre, a single fibre in each direction.
The decision to use the ribbon fibre PSM4 or one of the other three WDM-based schemes depends on the existing fibre plant used in a data centre, and the link distance required. The PSM4 module may prove to be less costly that the other three module types but its ribbon fibre is more expensive compared to similar length duplex fibre; the longer the link, the more significant the fibre becomes as part of the overall link cost. "What someone really wants is the lowest cost solution for their application," says D'Ambrosia.
The PSM4 has other, secondary uses that are part of its appeal. "With a breakout solution, even in copper, you can get to lower speeds," says D'Ambrosia. For example, a 40 GbE QSFP optical module using parallel fibre can be viewed as a 40 Gig interface or as a dense 4x10 Gig interface, with each fibre a 10 Gig interface. Such a 'breakout' solution is likely to be attractive earlier on, as applications transition to higher speeds.
Does it serve the industry to have four mid-reach solutions? D'Ambrosia says opinion varies. "My own personal belief is that it would be better for the industry overall if we didn't have so many choices," he says. "But the reality is there are a lot of different applications out there."
25 Gigabit Ethernet
Work has also started on a 25 GbE standard. An IEEE 802.3 Study Group has been created to investigate a copper-based and a multi-mode server interconnect at 25 Gig. In July, the 25G Ethernet Consortium was also announced by firms Google, Microsoft, Arista, Mellanox and Broadcom that is also backing 25 GbE for server interconnect.
"There are a lot of people who are worried that 25 GbE will go everywhere; you just don't introduce a new rate of Ethernet," says D'Ambrosia. And as with 100 Gig mid-reach with its proliferation of MSAs, now there is a concern about a proliferation of Ethernet speeds, he says.
But if there is one thing that D'Ambrosia has learned in his years active in Ethernet standards, it is not to second-guess the market. "If there is a cool application out there that will help save money, the market will figure it out and it [the solution] will become popular."
For now, the IEEE 802.3 25G Study Group has chosen to focus on single lane server interconnects. "That is what the charter is," says D'Ambrosia. "But that doesn't mean 25 Gigabit Ethernet will end there; there is never a single rate project."
400 Gigabit Ethernet
D'Ambrosia, who is also chair of the IEEE 802.3 400G Ethernet Task Force, also highlights the latest developments of the next Ethernet speed increment. There is a multi-mode 400 GbE fibre standard being worked on as well as three single mode fibre objectives.
The multi-mode solution will have a reach of 100m while the single mode options will span 500m, 2km and 10km. "For 500m, that is where everyone thinks parallel fibre can be used," says D'Ambrosia. At 10km, not surprisingly, it will be duplex fibre, while at 2km it is likely to be duplex simply because of the cost of long spans of parallel fibre.
In November, at the next Task Force meeting, proposals will be made as to how best to implement these differing requirements. For the multi-mode, talk is of a 16x25 Gig implementation. "I believe that is what we will see in the proposals in November," says D'Ambrosia. The Task Force is also looking at 50 Gig electrical interfaces for the longer 400 Gig reaches. Such an interface is likely to be ready by the time the 400G Task Force work is completed in 2017.
No one has suggested a 16x25 Gig single mode fibre optical interface, he says: "Do we do it as 50 Gig or 100 Gig?" Non-return-to-zero [NRZ], PAM4 and discrete multi-tone modulation schemes are all being considered. "For NRZ, we might see 8x50 Gig though that is not solidifying yet," he says. "For 500m there is talk of a x4 bundle and also pulse amplitude modulation for a single 100 Gig wavelength."
The November meeting is the last one for new proposals and in January 2015 decisions will be made.
The Ethernet Alliance is sponsoring an industry event this month entitled: "The Rate Debate" at the TEF 2014 event in Santa Clara, CA, on October 16th. The event will look at whether 40 Gig or 50 Gig Ethernet makes more sense, and the likely evolution. And if 50Gig is adopted, will 100 GbE based on 4 channels evolve to 200 Gigabit? There is also interest in extending Category 5 cable from 1 Gig to 2.5 Gig and even 5 Gig to extend the useful life of campus cabling, and that will also be addressed. More recently, there have been two Calls-For-Interest: for a Next Generation Enterprise Access BASE-T PHY and a 25GBASE-T and these will also likely be discussed.
Ethernet speeds used to evolve by a factor of 10, then by a factor of 4 and now 2.5. In future, with 50 Gig, it might also double. "With 40 Gig and 50 Gig, which one will dominate?" says D'Ambrosia. "But they are so close, why can't we come up with a solution that shares technology at both [speeds]?" These are just some of the issues to be discussed at the event.
"We are witnessing the evolution of Ethernet in ways that many of us never planned because there are markets that are demanding different things from it," says D'Ambrosia.