Infinera goes multi-terabit with its latest photonic IC
In his new book, The Great Acceleration, Robert Colvile discusses how things we do are speeding up.
In 1845 it took U.S. President James Polk six months to send a message to California. Just 15 years later Abraham Lincoln's inaugural address could travel the same distance in under eight days, using the Pony Express. But the use of ponies for transcontinental communications was shortlived once the electrical telegraph took hold. [1]
The relentless progress in information transfer, enabled by chip advances and Moore's law, is taken largely for granted. Less noticed is the progress being made in integrated photonic chips, most notably by Infinera.
In 2000, optical transport sent data over long-haul links at 10 gigabit-per-second (Gbps), with 80 such channels supported in a platform. Fifteen years later, Infinera demonstrated its latest-generation photonic integrated circuit (PIC) and FlexCoherent DSP-ASIC that can transmit data at 600Gbps over 12,000km, and up to 2.4 terabit-per-second (Tbps) - three times the data capacity of a state-of-the-art dense wavelength-division multiplexing (DWDM) platform back in 2000 - over 1,150km.
Infinite Capacity Engine
Infinera dubs its latest optoelectronic subsystem the Infinite Capacity Engine. The subsystem comprises a pair of indium-phosphide PICs - a transmitter and a receiver - and the FlexCoherent DSP-ASIC. The performance capabilities that the Infinite Capacity Engine enables were unveiled by Infinera in January with its Advanced Coherent Toolkit announcement. Now, to coincide with OFC 2016, Infinera has detailed the underlying chips that enable the toolkit. And company product announcements using the new hardware will be made later this year, says Pravin Mahajan, the company's director of product and corporate marketing.
The claimed advantages of the Infinite Capacity Engine include a 82 percent reduction in power consumption compared to a system using discrete optical components and a dozen 100-gigabit coherent DSP-ASICs, and a 53 percent reduction in total-cost-of-ownership compared to competing dense WDM platforms. The FlexCoherent chip also features line rate data encryption.
"The Infinite Capacity Engine is the industry's first multi-terabit it super-channel, says Mahajan. "It also delivers the industry's first multi-terabit layer one encryption."
Multi-terabit PIC
Infinera's first transmitter and receiver PIC pair, launched in 2005, supported 10, 10-gigabit channels and implemented non-coherent optical transmission.
In 2011 Infinera introduced a 500-gigabit super-channel coherent PIC pair used with Infinera's DTN-X platforms and also its Cloud Xpress data centre interconnect platform launched in 2014. The 500 Gigabit design implemented 10, 50 gigabit channels that implemented polarisation-multiplexed, quadrature phase-shift keying (PM-QPSK) modulation. The accompanying FlexCoherent DSP-ASIC was implemented using a 40nm CMOS process node and support a symbol rate of 16 gigabaud.
The PIC design has since been enhanced to also support additional modulation schemes such as as polarisation-multiplexed, binary phase-shift keying (PM-BPSK) and 3 quadrature amplitude modulation (PM-3QAM) that extend the DTN-X's ultra long-haul performance.
In 2015 Infinera also launched the oPIC-100, a 100-gigabit PIC for metro applications that enables Infinera to exploit the concept of sliceable bandwidth by pairing oPIC-100s with a 500 gigabit PIC. Here the full 500 gigabit super-channel capacity can be pre-deployed even if not all of the capacity is used. Using Infinera's time-based instant bandwidth feature, part of that 500 gigabit capacity can be added between nodes in a few hours based on a request for greater bandwidth.
Now, with the Infinite Capacity Engine PIC, the effective number of channels has been expanded to 12, each capable of supporting a range of modulation techniques (see table below) and data rates. In fact, Infinera uses multiple Nyquist sub-carriers spread across each of the 12 channels. By encoding the data across multiple sub-carriers a lower-baud rate can be used, increasing the tolerance to non-linear channel impairments during optical transmission.
Mahajan says the latest PIC has a power consumption similar to its current 500 Gigabit super-channel PIC but because the photonic design supports up to 2.4 terabit, the power consumption in gigabit-per-Watt is reduced by 70 percent.
FlexCoherent encryption
The latest FlexCoherent DSP-ASIC is Infinera's most complex yet. The 1.6 billion transistor 28nm CMOS IC can process two channels, and supports a 33 gigabaud symbol rate. As a result, six DSP-ASICs are used with the 12-channel PIC.
It is the DSP-ASIC that enables the various elements of the advanced coherent toolkit that includes improved soft-decision forward error correction. "The net coding gain is 11.9dB, up 0.9 dB, which improves the capacity-reach," says Mahajan. Infinera says the ultra long-haul performance has also been improved from 9,500km to over 12,000km.
Source: Infinera
The DSP also features layer one encryption implementing the 256-bit Advanced Encryption Standard (AES-256). Infinera says the request for encryption is being led by the Internet content providers but wholesale operators and co-location providers also want to secure transmissions between sites.
Infinera introduced layer two MACsec encryption with its Cloud Xpress platform. This encrypts the Ethernet payload but not the header. With layer one encryption, it is the OTN frames that are encoded. "When we get down to the OTN level, everything is encrypted," says Mahajan. An operator can choose to encrypt the entire super-channel or encrypt at the service level, down to the ODU0 (1.244 Gbps) level.
System benefits
Using the Infinite Capacity Engine, the transmission capacity over a fibre increases from 9.5 terabit to up to 26.4 terabit.
And with the newest PIC, Infinera can expand the sliceable transponder concept for metro-regional applications. The 2.4 terabits of capacity can be pre-deployed and new capacity turned up between nodes. "You can suddenly turn up 200 gigabit for a month or two, rent and then return it," says Mahajan. However, to support the full 2.4 terabits of capacity, the PIC at the other end of the link would also need to support 16-QAM.
Infinera does say there will be other Infinite Capacity Engine variants. "There will be specific engines for specific markets, and we would choose a subset of the modulations," says Mahajan.
One obvious platform that will benefit from the first Infinite Capacity Engine is the DTN-X. Another that will likely use an ICE variant is Infinera's Cloud Xpress. At present Infinera integrates its 500-gigabit PIC in a 2 rack-unit box for data centre interconnect applications. By using the new PIC and implementing PM-16QAM, the line-side capacity per rack unit of a second-generation Cloud Xpress would rise from 250 gigabit to 1.2 terabit. And with layer one encryption, the MACsec IC may no longer be needed.
Mahajan says the Infinite Capacity Engine has already been tested in the Telstra trial detailed in January. "We have already proven its viability but it is not deployed and carrying live traffic," he says.
Infinera speeds up network restoration
- Claimed to be the only hardware implementation of the Shared Mesh Protection protocol
- Provides network-wide protection against multiple network failures
- The chip is already within the DTN-X system; protocol will be activated this year
Pravin Mahajan, Infinera
Infinera has developed a chip to speed up network restoration following faults.
The chip implements the Shared Mesh Protection (SMP) protocol being developed by the International Telecommunication Union (ITU) and the Internet Engineering Task Force (IETF) and Infinera believes it is the only vendor with hardware acceleration of the protocol.
The SMP standard is still being worked on and will be completed this year. Infinera demonstrated its hardware SMP implementation at OFC/NFOEC 2013 and will activate the scheme in operators' networks using a platform software upgrade this year.
The chip, dubbed Fast Shared Mesh Protection (FastSMP), is sprinkled across cards within Infinera's DTN-X platform and will be linked to other FastSMP ICs across the network. The FastSMP chips exchange signalling information and use internal look-up tables with pre-calculated routing data to determine the required protection action when one or more network failures occur.
Network faults
The causes of network faults range from fibre cuts from construction work to natural disasters such as Hurricane Sandy and the Asia Pacific tsunami. Level 3 Communications cited in 2011 that squirrels were the second most common cause of fibre cuts after construction work. The squirrels, chewing through fibre, accounted for 17 percent of all cuts. Meanwhile, one Indian service provider says it experiences 100 fibre cuts nationwide each day, according to Infinera.
Operators are also having to share their network maps with enterprises that want to assess the risk based on geography before choosing a service provider. "End customers no longer necessarily trust the service level agreements they have with operators," says Pravin Mahajan, director, corporate marketing and messaging at Infinera. In riskier regions, for example those prone to earthquakes, enterprises may choose two operators. "A form of 1+1 protection,” says Mahajan.
Operators want resilient networks that adapt to faults quickly, ideally within 50ms, without adding extra cost.
Traditional resiliency schemes include SONET/SDH’s 1+1 protection. This meets the sub-50ms requirement but addresses single faults only and requires dedicated back-up for each circuit. At the IP/MPLS (Internet Protocol/ Multiprotocol Label Switching) layer, the MPLS Fast Re-Route scheme caters for multiple failures and is sub-50ms. But it only addresses local faults, not the full network. And being packet-based - at a higher layer of the network - the scheme is costlier to implement.
"End customers no longer necessarily trust the service level agreements they have with operators"
Infinera's protection scheme uses its digital optical networking approach based on its photonic integrated circuits (PICs) coupled with Optical Transport Networking (OTN). OTN resides between the packet and optical layers, and using a mesh network topology, it can handle multiple failures. By sharing bandwidth at the transport layer, the approach is cheaper than at the packet layer. But being software-based, restoration takes seconds.
Infinera has speeded up the scheme by implementing SMP with its chip such that it meets the 50ms goal.
FastSMP chip
Infinera plans for multiple failures using the Generalized Multiprotocol Label Switching (GMPLS) control plane. “The same intelligence is now implemented in hardware [using the FastSMP processor],” says Mahajan.
The chip is on each 500 Gigabit-per-second (Gbps) line card, within the platform's OTN switch fabric, the client side and as part of the controller. The FastSMP, described as a co-processor to the CPU, hosts look-up tables with rules as to what should happen with each failure. The chips, located in the platform and across the network, then adjust to the back-up plan for each service failure.
Infinera says that the protection is at the service level not at the link level. "It does this at ODU [OTN's optical data unit] granularity," says Mahajan; each circuit can hold different sized services, 2.5 Gigabit-per-second (Gbps) or 10Gbps for example, all carried within a 100Gbps light path. "By defining failure scenarios on a per-service basis, you now need to put all these entries in hardware," says Mahajan.
To program the chip, network failures are simulated using Infinera's network planning tool to determine the required back-up schemes. These can be chosen based on shortest path or lowest latency, for example.
The GMPLS control plane protocol determines the rules as to how the network should be adapted and these are written on-chip. When a failure occurs, the chip detects the failure and performs the required actions.
The FastSMP chip is already on all the DTN-X line cards Infinera has shipped and will be enabled using software upgrade.
The GMPLS control plane recomputes backup paths after a failure has occurred. Typically no action is required but if several failures occur, the new GMPLS backup paths will be distributed to update the FastSMPs' tables. "Only on the third or fourth failure typically will a new backup plan be needed," says Mahajan.
In effect, the more meshed the network topology, the greater the number of failures that can be tolerated. "When you have three or four failures, you need to have new computation at the GMPLS control plane and then it can repopulate the backups for failures 3, 4, and 5," he says.
Instant bandwidth and FastSMP
Infinera is able to turn up bandwidth in real-time using its 500Gbps super-channel PIC. "We slice up the 500 Gig capacity available per line card into 100 Gig chunks," says Mahajan.
This feature, combined with FastSMP, aids operators dealing with failures once traffic is rerouted. The next backup route, if it is close to its full capacity, can have an extra 100 Gigabit of capacity added in case the link is called into use.
A study based on an example 80-node network by ACG Research estimates that the Shared Mesh Protection scheme uses 30 percent less line-side ports compared to an equivalent network implementing the 1+1 protection scheme.