The uphill battle to keep pace with bandwidth demand
Relative traffic increase normalised to 2010 Source: IEEE
Optical component and system vendors will be increasingly challenged to meet the expected growth in bandwidth demand.
According to a recent comprehensive study by the IEEE (The IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment report), bandwidth requirements are set to grow 10x by 2015 compared to demand in 2010, and a further 10x between 2015 and 2020. Meanwhile, the technical challenges are growing for the vendors developing optical transmission equipment and short-reach high-speed optical interfaces.
Fibre bandwidth is becoming a scarce commodity and various techniques will be required to scale capacity in metro and long-haul networks. The IEEE is expected to develop the next-higher speed Ethernet standard to follow 100 Gigabit Ethernet (GbE) in 2017 only. The IEEE is only talking about capacities and not interface speeds. Yet, at this early stage, 400 Gigabit Ethernet looks the most likely interface.
"The various end-user markets need technology that scales with their bandwidth demands and does so economically. The fact that vendors must work harder to keep scaling bandwidth is not what they want to hear"
A 400GbE interface will comprise multiple parallel lanes, requiring the use of optical integration. A 400GbE interface may also embrace modulation techniques, further adding to the size, complexity and cost of such an interface. And to achieve a Terabit, three such interfaces will be needed.
All these factors are conspiring against what the various end-user bandwidth sectors require: line-side and client-side interfaces that scale economically with bandwidth demand. Instead, optical components, optical module and systems suppliers will have to invest heavily to develop more complex solutions in the hope of matching the relentless bandwidth demand.
The IEEE 802.3 Bandwidth Assessment Ad Hoc group, which produced the report that highlights the hundredfold growth in bandwidth demand between 2010 and 2020, studied several sectors besides core networking and data centre equipment such as servers. These include Internet exchanges, high-performance computing, cable operators (MSOs) and the scientific community.
The difference growth rates in bandwidth demand it found for the various sectors are shown in the chart above.
Optical transport
A key challenge for optical transport is that fibre spectrum is becoming a precious commodity. Scaling capacity will require much more efficient use of spectrum.
To this aim, vendors are embracing advanced modulation schemes, signal processing and complex ASIC designs. The use of such technologies also raises new challenges such as moving away from a rigid spectrum grid, requiring the introduction of flexible-grid switching elements within the network.
And it does not stop there.
Already considerable development work is underway to use multi-carriers - super-channels - whose carrier count can be adapted on-the-fly depending on demand, and which can be crammed together to save spectrum. This requires advanced waveform shaping based on either coherent orthogonal frequency division multiplexing (OFDM) or Nyquist WDM, adding further complexity to the ASIC design.
At present, a single light path can be increased from 100 Gigabit-per-second (Gbps) to 200Gbps using the 16-QAM amplitude modulation scheme. Two such light paths give a 400Gbps data rate. But 400Gbps requires more spectrum than the standard 50GHz band used for 100Gbps transmission. And using QAM reduces the overall optical transmission reach achieved.
The shorter resulting reach using 16-QAM or 64-QAM may be sufficient for metro networks (~1000km) but to achieve long-haul and ultra-long-haul spans will require super-channels based on multiple dual-polarisation, quadrature phase-shift keying (DP-QPSK) modulated carriers, each occupying 50GHz. Building up a 400Gbps or 1 Terabit signal this way uses 4 or 10 such carriers, respectively - a lot of spectrum. Some 8Tbps to 8.8Tbps long-haul capacity result using this approach.
The main 100Gbps system vendors have demonstrated 400Gbps using 16-QAM and two carriers. This doubles system capacity to 16-17.6Tbps. A further 30% saving in bandwidth using spectral shaping at the transmitter crams the carriers closer together, raising the capacity to some 23Tbps. The eventual adoption of coherent OFDM or Nyquist WDM will further boost overall fibre capacity across the C-band. But the overall tradeoff of capacity versus reach still remains.
Optical transport thus has a set of techniques to improve the amount of traffic it can carry. But it is not at a pace that matches the relentless exponential growth in bandwidth demand.
After spectral shaping, even more complex solutions will be needed. These include extending transmission beyond the C-band, and developing exotic fibres. But these are developments for the next decade or two and will require considerable investment.
The various end-user markets need technology that scales with their bandwidth demands and does so economically. The fact that vendors must work harder to keep scaling bandwidth is not what they want to hear.
"No-one is talking about a potential bandwidth crunch but if it is to be avoided, greater investment in the key technologies will be needed. This will raise its own industry challenges. But nothing like those to be expected if the gap between bandwidth demand and available solutions grows"
Higher-speed Ethernet
The IEEE's Bandwidth Assessment study lays the groundwork for the development of the next higher-speed Ethernet standard.
Since the standard work has not yet started, the IEEE stresses that it is premature to discuss interface speeds. But based on the state of the industry, 400GbE already looks the most likely solution as the next speed hike after 100GbE. Adopting 400GbE, several approaches could be pursued:
- 16 lanes at 25Gbps: 100GbE is moving to a 4x25Gbps electrical interface and 400GbE could exploit such technology for a 16-lane solution, made up of four, 4x25Gbps interfaces. "If I was a betting man, I'd probably put better odds on that [25Gbps lanes] because it is in the realm of what everyone is developing," John D'Ambrosia, chair of the IEEE 802.3 Industry Connections Higher Speed Ethernet Consensus group and chair of the the IEEE 802.3 Bandwidth Assessment Ad Hoc group, told Gazettabyte.
- 10 lanes at 40Gbps: The Optical Internetworking Forum (OIF) has started work on an electrical interface operating between 39 and 56Gbps (Common Electrical Interface - 56G-Close Proximity Reach). This could lead to 40Gbps lanes and a 10x40Gbps implementation for a 400Gbps Ethernet design.
- Modulation: For the 100Gbps backplane initiative, the IEEE is working on pulse-amplitude modulation (PAM), says D'Ambrosia. Such modulation could be used for 400GbE. Modulation is also being considered by the IEEE to create a single-lane 100Gbps interface. Such a solution could lead to a 4-lane 400GbE solution. But adopting modulation comes at a cost: more sophisticated electronics, greater size and power consumption.
As with any emerging standard, first designs will be large, power-hungry and expensive. The industry will have to work hard to produce more integrated 16-lane or 10-lane designs. Size and cost will also be important given that three 400GbE modules will be needed to implement a Terabit interface.
The challenge for component and module vendors is to develop such multi-lane designs yet do so economically. This will require design ingenuity and optical integration expertise.
Timescales
Super-channels exist now - Infinera is shipping its 5x100Gbps photonic integrated circuit. Ciena and Alcatel-Lucent are introducing their latest generation DSP-ASICs that promise 400Gbps signals and spectral shaping while other vendors have demonstrated such capabilities in the lab.
The next Ethernet standard is set for completion in 2017. If it is indeed based on a 400GbE Ethernet interface, it will likely use 4x25Gbps components for the first design, benefiting from emerging 100GbE CFP2 and CFP4 modules and their more integrated designs. But given the standard will only be completed in five years' time, new developments should also be expected.
No-one is talking about a potential bandwidth crunch but if it is to be avoided, greater investment in the key technologies will be needed. This will raise its own industry challenges. But nothing like those to be expected if the gap between bandwidth demand and available solutions grows.
The next high-speed Ethernet standard starts to take shape
Source: Gazettabyte
The IEEE has begun work to develop the next-speed Ethernet standard beyond 100 Gigabit to address significant predicted growth in bandwidth demand.
The standards body has set up the IEEE 802.3 Industry Connections Higher Speed Ethernet Consensus group, chaired by John D’Ambrosia, who previously chaired the 40 and 100 Gigabit IEEE P802.3ba Ethernet standards ratified in June 2010. "I guess I’m a glutton for punishment,” quips D'Ambrosia.
The Higher Speed Ethernet standard could be completed by early 2017.
The group has been set up after an extensive one-year study by the IEEE 802.3 Bandwidth Assessment Ad Hoc group investigating networking capacity growth trends in various markets. The study looked beyond core networking and data centres - the focus of the 40 and 100 Gigabit Ethernet (GbE) study work - to include high-performance computing, financial markets, Internet exchanges and the scientific community.
One of the resulting report's conclusions (IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment report) is that Terabit capacity will likely be required by 2015, growing a further tenfold by 2020.
“By 2015 core networks on average will need ten times the bandwidth of 2010, and one hundred times [the bandwidth] by 2020,” says D’Ambrosia, who is also the chair of the IEEE 802.3 Ethernet Bandwidth Assessment Ad Hoc group, as well as chief Ethernet evangelist, CTO office at Dell. “If you look at Ethernet in 2010, it was at 100 Gigabit, so ten times 100 Gigabit in 2015 is a Terabit and a hundred times 2010 is 10 Terabit by 2020.”
"We have got to the point where the pesky laws of physics are challenging us"
John D'Ambrosia, chair of the IEEE 802.3 Industry Connections Higher Speed Ethernet Consensus group
D'Ambrosia stresses that the Ad Hoc group's role is to talk about capacity requirements, not interface speeds. The technical details of any interface implementation will only become clear once the standardisation effort is well under way.
A second Ethernet Bandwidth Assessment study finding is that network aggregation nodes are growing faster, and hence require greater capacity earlier, than the network's end points.
"There is also a growing deviation between the big guys and the rest of the market," says D'Ambrosia. He has heard individuals from the largest internet content providers say they need Terabit connections by 2013, while others claim it will be 2020 before a mass market develops for such an interconnect.
D'Ambrosia says the main findings are not necessarily surprising but there were two 'aha' moments during the study.
One was that the core networking growth rates predicted in 2007 by the 40 and 100 Gig High-speed Study Group are still valid five years on.
The other concerned the New York Stock Exchange that had forecast that it would need to install four 100Gbps links in its data centre yet ended up using 13. "If there is any company that has a lot of money on the line and would have the best chance of nailing down their needs, I would put the New York Stock Exchange up there," says D'Ambrosia. "That tells you something about bandwidth growth and that you can still underestimate what is going to happen."
"The reality is that I can't give you any solutions right now that are attractive to do a Terabit"
What next
The IEEE standardisation work for the next speed Ethernet has not started but the completed Ethernet Bandwidth Assessment study will likely form an important input for the Industry Connections Higher Speed Ethernet Consensus group.
The start of the standardisation work is expected in either March or July 2013 with the Study Group phase then taking a further eight months. This compares to 18 months for the IEEE 40GbE and 100GbE Study Group work (see chart above). The Task Force's work - writing the specification - is then expected to take a further two and a half years, completing the standard in early 2017 if all goes to plan.
Technology options
While stressing that the IEEE is talking about capacities and not yet interface speeds, Terabit capacity could be solved using multiple 400 Gigabit Ethernet interfaces, says D'Ambrosia.
At present there is no 400GbE project underway. However, the industry does believe that 400GbE is "doable" economically and technically. "Much of the supply base, when we are talking about Ethernet, is looking at 400 Gigabit," says D'Ambrosia.
Achieving a 1TbE interface looks much more distant. "People pushing for 1 Terabit tend to be the people looking at it from the bandwidth perspective and then looking at upgrading their networks and making multiple investments," he says. "But the reality is that I can't give you any solutions right now that are attractive to do a Terabit."
All agree that the technical challenges facing the industry to meet growing bandwidth demands are starting to mount. "We have got to the point where the pesky laws of physics are challenging us," says D'Ambrosia.
Further reading:
IEEE 802.3 Industry Connections Higher Speed Ethernet Ad Hoc