NeoPhotonics’ growing 400G coherent pluggable portfolio
NeoPhotonics has unveiled its first two 400-gigabit coherent pluggable modules that support the OIF’s 400ZR coherent standard and extended ZR+ modes.
The company has delivered samples of its ClearLight CFP2-DCO module for trials. The CFP2-DCO supports 400ZR, metro, and long-haul optional transmissions.
NeoPhotonics has also delivered to a hyperscaler the first samples of a 400-gigabit OSFP pluggable that supports 400ZR and 400ZR+.
Both modules use Inphi’s latest Canopus 7nm CMOS coherent digital signal processor (DSP) chip.
Module types
The OIF has developed the 400ZR standard to enable 400-gigabit signals to be sent between switches or routers in data centres up to 120km apart.
The main three pluggable modules earmarked for 400ZR are the QSFP-DD, OSFP and CFP2-DCO.
These modules differ in size and power envelope, ranging from the QSFP-DD, which is the most compact and has the smallest power envelope, to the CFP2-DCO module which supports the highest power and size.
It is the two client-side module form factors – the QSFP-DD and the OSFP – that will be mainly used for 400ZR.
“The CFP2 has more of a power envelope available so it tends to be used for longer reach applications,” says Ferris Lipscomb, vice president of marketing at NeoPhotonics.
These applications include specialist data-centre-interconnect applications and the metro and long-haul needs of the telecom operators.
400G CFP2-DCO
NeoPhotonics’ ClearLight CFP2-DCO uses an extension of a fibre’s C-band spectrum, what Huawei calls the Super C-band while NeoPhotonics refers to its implementation as C++.
The Super C-band covers 6THz of the spectrum compared to the standard C-band’s 4THz. The extended band can fit 120, 50GHz-wide channels or 80, 75GHz-wide channels.
NeoPhotonics can send 64-gigabaud (GBd), 400-gigabit signals over a 75GHz channel such that using ClearLight CFP2-DCO modules, 32 terabits can be sent overall.
The CFP2-DCO module uses NeoPhotonics’s ultra narrow-band line-width tunable laser that has had its tuning range extended to span the Super C-band. NeoPhotonics also uses its 64GBd intradyne coherent receiver (ICR) and coherent driver modulator.
The ClearLight CFP2-DCO can also send 400-gigabit signals over distances greater than 400ZR’s 120km. In addition, the module supports 200-gigabit transmissions over greater distances.
Sending a 200-gigabit at 64GBd using a 75GHz channel and quadrature phase-shift keying (QPSK) modulation, an optical signal-to-noise ratio (OSNR) of under 14dB is needed. Alternatively, using a 50GHz channel at 32GBd and 16-ary quadrature amplitude modulation (16-QAM), the OSNR is 16dB.
“With these [decibel] numbers, lower is better,” says Lipscomb. “You can go further with 64 gigabaud and QPSK; it’s 2dB better.”
Lipscomb says one use case for the 400-gigabit CFP2-DCO promises significant volumes: “The Super C-band has been used for deployments particularly by the Chinese carriers where they want to get more channels down a fibre.”
OSFP
NeoPhotonics has also unveiled its ClearLight OSFP module that enables the 400ZR standard and 400-gigabit transmissions for metro.
The module incorporates NeoPhotonics’s nano integrated tunable laser assembly (Nano-ITLA) and its silicon photonics-based coherent optical sub-assembly (COSA) that integrates the coherent receiver and modulator driver functions.
The OSFP tunes over 75GHz- or 100GHz-spaced channels, enabling 85 and 64 channels, respectively, as specified by the OIF. The OSFP also supports longer metro reaches at 400 gigabits.
NeoPhotonics also makes arrayed waveguide gratings (AWG) suited for 64GBd and 75GHz channel spacings that both modules support. “You need broader passbands and different channel spacings for 64 gigabaud,” says Lipscomb.
ZR+ interop?
Lipscomb is not a proponent of enforcing standardisation for the ZR+ extended modes, as has been done with 400ZR, despite the resulting lack of interoperability between optical modules from different vendors.
“There will always be the temptation in cases where you need it, to give up interoperability for increased [optical] performance,” he says.
The era of 400G coherent pluggables finally emerges
Part 1: 7nm coherent DSPs, ZR and ZR+
The era of 400-gigabit coherent pluggable modules has moved a step closer with Inphi’s announcement of its Canopus coherent digital signal processor (DSP) and its QSFP-DD ColorZ II optical module.
NeoPhotonics has also entered the fray, delivering first samples of its 400-gigabit ClearLight CFP2-DCO module that uses the Canopus DSP.
The ColorZ II and ClearLight modules support the 400ZR OIF standard used to link data centres up to 120km apart. They also support extended modes, known as ZR+, that is not standardised.
ZR+’s modes include 400 Gigabit-per-second (Gbps) over distances greater than 400ZR’s 120km and lower data rates over metro-regional and long-haul distances.
The announcements of the Canopus DSP and 400-gigabit pluggable coherent modules highlight the approaches being taken for ZR+. Optical module vendors are aligning around particular merchant DSPs such that interoperability exists but only within each camp.
The first camp involves Inphi and three other module vendors, one being NeoPhotonics. The second camp is based on the OpenZR+ specification that offers interoperability between the DSPs of the merchant players, Acacia Communications and NTT Electronics (NEL). Cisco is in the process of acquiring Acacia.
Market analysts, however, warn that such partial interoperability for ZR+ harms the overall market opportunity for coherent pluggables.
“ZR+ should be interoperable like ZR, and come along with the hard decisions the ZR standard required,” says Andrew Schmitt, founder and directing analyst at research form, Cignal AI.
The optical module vendors counter that only with specialist designs – designs that are multi-sourced – can the potential of a coherent DSP be exploited.
Applications
The advent of 400-gigabit coherent optics within compact client-side form factors is a notable development, says Inphi. “The industry has been waiting for this inflextion point of having, for the first time, 400-gigabit coherent pluggables that go on router and switch interfaces,” says Pranay Aiya, vice president of product marketing and applications engineering at Inphi.
“IP over DWDM has never happened; we have all heard about it till the cows come home,” says Aiya.
IP-over-DWDM failed to take off because of the power and space demands of coherent optics, especially when router and switch card slots come at a premium. Using coherent optics on such platforms meant trading off client-side faceplate capacity to fit bulkier coherent optics. This is no longer the case with the advent of QSFP-DD and OSFP coherent modules.
“If you look at the reasons why IP-over-DWDM – coloured optics directly on routers – failed, all of those reasons have changed,” says Schmitt. The industry is moving to open line systems, open network management, and more modular network design.
“All of the traffic is IP and layer-1 switching and grooming isn’t just unnecessary, it is more expensive than low-feature layer-2 switching,” says Schmitt, adding that operators will use pluggables wherever the lower performance is acceptable. Moreover, this performance gap will narrow with time.
The Canopus DSP also supports ZR+ optical performance and, when used within a CFP2-DCO module with its greater power enveloped than OSFP and QSFP-DD, enables metro and long-haul distances, as required by the telecom operators. This is what Neophotonics has announced with its ClearLight CFP2-DCO module.
Canopus
Inphi announced the Canopus DSP last November and revealed a month later that it was sampling its first optical module, the ColorZ II, that uses the Canopus DSP. The ColorZ II is a QSFP-DD pluggable module that supports 400ZR as well as the ZR+ extended modes.
Inphi says that, given the investment required to develop the 7nm CMOS Canopus, it had to address the bulk of the coherent market.
“We were not going after the ultra-long-haul and submarine markets but we wanted pluggables to address 80-90 per cent of the market,” says Aiya.
This meant developing a chip that would support the OIF’s 400ZR, 200-gigabit using quadrature phased-shift keying (QPSK) modulation for long haul, and deliver 400-gigabit over 500-600km.
The Canopus DSP also supports probabilistic constellation shaping (PCS), a technology that until now has been confined to the high-end coherent DSPs developed by the leading optical systems vendors.
With probabilistic shaping, not all the constellation points are used. Instead, those with lower energy are favoured; points closer to the origin on a constellation graph. The only time all the constellation points are used is when sending the maximum data rate for a given modulation scheme.
Choosing the inner, lower-energy constellation points more frequently than the outer points to send data reduces the average energy and improves the signal-to-noise ratio. To understand why, the symbol error rate at the receiver is dominated by the distance between neighbouring points on the constellation. Reducing the average energy keeps the distance between the points the same, but since a constant signal power level is used for DWDM transmission, applying gain increases the distance between the constellation points. The result is improved optical performance.
Probabilistic shaping also allows an exact number of bits-per-symbol to be sent, even non-integer values.
For example, using standard modulation schemes such as 64-QAM with no constellation shaping, 6 bits-per-symbol are sent. Using shaping and being selective as to which constellation points are used, 5.7 bits-per-symbol could be sent, for example. This enables finer control of the sent data, enabling operators to squeeze the maximum data rate to suit the margins on a given fibre link.
“This is the first time a DSP with probabilistic shaping has been available in a size and power that enables pluggables,” says Aiya.
The resulting optical performance using the Canopus is up to 1,500km at 300Gbps signals and up to 2,000km for 200Gbps transmissions (see Table above). As for baud rates, the DSP ranges from 30+ to the mid-60s Gigabaud.
Inphi also claims a 75 per cent reduction in power consumption of the Canopus compared to 16nm CMOS DSPs found in larger, 4×5-inch modules.
Several factors account for the sharp power reduction: the design of the chip’s architecture and physical layout, and the use of 7nm CMOS. The Canopus uses functional blocks that extend the reach, and these can be turned off to reduce the power consumption when lower optical performance is acceptable.
The architectural improvements and the physical layout account for half of the overall power savings, says Aiya, with the rest coming from using a 7nm CMOS.
The result is a DSP a third the size of 16nm DSPs. “It [pluggables] requires the DSP to be very small; it’s not a paperweight anymore,” says Aiya.
400ZR and ZR+
The main challenge for the merchant coherent DSP camps is the several, much larger 400ZR eco-systems from Ciena, Cisco and Huawei.
“Each one of these eco-systems will be larger than the total merchant market of 400ZR,” says Vladimir Kozlov, CEO and founder of LightCounting. The system vendors will make sure that their products offer something extra if plugged into their equipment while maintaining interoperability. “This could be some simple AI-like features monitoring the link performance and warning customers of poor operation of devices on the other side of the link if these are made by another supplier,” says Kozlov.
LightCounting says that ZR+ units will be half to a third of the the number of 400ZR units shipped. However, each ZR+ module will command a higher selling price.
Regarding the ZR+ camps, one standardisation effort is OpenZR+ that adopts the forward-error correction (oFEC) scheme of the OpenROADM MSA, supports multiplexing of 100 Gigabit Ethernet (GbE) and 200GbE client signals, and different line rates – 100-400Gbps – to achieve greater reaches.
The backers of OpenZR+ include the two merchant DSP vendors, Acacia and NEL, as well as Fujitsu Optical Components, Lumentum, and Juniper Networks.
The second ZR+ camp includes four module-makers that are adopting the Canopus: Inphi, Molex Optoelectronics, NeoPhotonics and an unnamed fourth company. According to Schmitt, the unnamed module maker is II-VI. II-VI declined to comment when asked to confirm.
Schmitt argues that ZR+ should be interoperable, just like 400ZR. “I think NEL, Acacia, and Inphi should have an offsite and figure this out,” he says. “These three companies are in a position to nail down the specs and create a large, disruptive coherent pluggable market.”
Simon Stanley, founder and principal consultant at Earlswood Marketing Limited, expects several ZR+ solutions to emerge but that the industry will settle on a common approach. “You will initially see both ZR+ and OpenZR+,” says Stanley. “ZR+ will be specific to each operator but over time I expect OpenZR+ or something similar to become the standard solution.”
But the optical vendors stress the importance of offering differentiated designs to exploit the coherent DSP’s full potential. And maximising a module’s optical performance is something operators want.
“We are all for standards where it makes sense and where customers want it,” says Inphi’s Aiya. “But for customers that require the best performance, we are going to offer them an ecosystem around this DSP.”
“It is always a trade-off,” adds Ferris Lipscomb, vice president of marketing at NeoPhotonics. “More specialised designs that aren’t interoperable can squeeze more performance out; interoperable has to be the lowest common denominator.”
Next-generation merchant DSPs
The next stage in coherent merchant DSP development is to use a 5nm CMOS process, says Inphi. Such a state-of-the-art [CMOS] process will be needed to double capacity again while keeping the power consumption constant.
The optical performance of a 5nm coherent DSP in a pluggable will approach the high-end coherent designs. “It [the optical performance of the two categories] is converging,” says Aiya.
However, demand for such a device supporting 800 gigabits will take time to develop. Several years have passed for demand for 400-gigabit client-side optics to ramp and there will be a delay before telecom operators need 400-gigabit wavelengths in volume, says Inphi.
LightCounting points out that it will take Inphi and its ecosystem of suppliers at least a year to debug their products and demonstrate interoperability.
“And keep in mind that we are talking about the industry that is changing very slowly,” concludes Kozlov.
WaveLogic 5: Packing a suitcase of ideas in 7nm CMOS
-
Ciena’s WaveLogic 5 coherent digital signal processor family comprises the Extreme and Nano chips
-
The WaveLogic 5 Extreme maximises optical capacity and transmission reach while the WaveLogic 5 Nano is targeted at compact, power-conservative applications
Kim Roberts
Advancing coherent optical transmission performance; targeting the emerging coherent pluggable market; selling modules directly, and the importance of being more vertically integrated. All these aspects were outlined by Cisco to explain why it intends to buy the coherent optical transmission specialist, Acacia Communications; a deal that is set to be completed in the spring of 2020.
But such strategic thinking is being pursued by Ciena with its next-generation WaveLogic 5 family of coherent DSPs.
The WaveLogic 5 continues Ciena’s tradition of issuing a coherent digital signal processor (DSP) family approximately every three years: Ciena announced the WaveLogic 3 in 2012 and the WaveLogic Ai in 2016. (Add links).
The company has managed to maintain its three-yearly cadence despite the increasing sophistication of each generation of coherent DSP. For example, the WaveLogic 5 Extreme will support 800 gigabits-per-wavelength, double Ciena’s WaveLogic Ai that has been shipping for nearly two years.
Kim Roberts, vice president of WaveLogic science, says Ciena has managed to deliver its coherent DSPs in a timely manner since much of the algorithmic development work was done 5-6 years ago. The issue has been that certain features developed back then could not be included within the WaveLogic Ai.
WaveLogic 5 is implemented using a 7nm FinFET CMOS process whereas the WaveLogic Ai uses a 28nm specialist CMOS process known as fully-depleted silicon-on-insulator (FD-SOI).
“Seven-nanometer CMOS, due to its density and low heat, allows us to implement things that didn’t make the cut for the WaveLogic Ai,” says Roberts.
The company has a ‘suitcase of ideas’, he says, but not all of the concepts make it into any one generation of chip. “They have to justify performance versus schedule versus heat [generated],” says Roberts. “As we improve the technology, more features make the cut.”
And there are developments that will be included in future designs: “We keep refilling the suitcase,” says Roberts.
NAMING
Ciena first used the Extreme and Nano nomenclature with the WaveLogic 3. In contrast, the WaveLogic Ai, when launched in 2016, was a single-chip targeting the high-end. Ciena chose to change the naming scheme with the Ai since the chip signified a shift with features such as network monitoring.
However, Ciena highlights a key difference between the WaveLogic 3 and WaveLogic 5 families. The WaveLogic 3 Extreme and the WaveLogic 3 Nano could talk to each other on appropriate spans. In contrast, the two WaveLogic 5 chips are distinct. “They are not designed to interwork,” says Roberts.
NETWORKING TRENDS
Telecom service providers are investing in their networks to make them more adaptive. They want their networks to be scalable and programmable, says Ciena.
The operators also want to better understand what is happening in their networks and that requires collecting data, performing analytics and using software to configure their networks in an automated way.
“How do you get there? It is all about coherent technology,” says Helen Xenos, senior director, portfolio marketing at Ciena. “It is a critical element that is helping operators scale their networks.”
By enhancing the traffic-carrying capacity of fibre, coherent technology enables operators to reduce transport costs. “It allows them to be more competitive as they can do more with the hardware they deploy,” says Xenos.
Helen Xenos
Both telcos and cable operators are also applying coherent technology to new applications in their networks such as access.
These transport needs are causing a divergence in requirements.
One is to keep advancing optical performance in terms of the spectral efficiency and the traffic-carrying capacity of links. This is what the WaveLogic 5 Extreme tackles.
The second requirement - producing a compact coherent design for the network edge - is addressed by the WaveLogic 5 Nano.
For access designs, what is important is a compact design where the optics and the DSP can operate over an extended temperature range.
The Nano also addresses the hyperscalers’ need to connect their distributed data centres across a metro. “They need high capacity - 400 gigabits - and short-reach connectivity,” says Xenos. “It really needs to be the smallest footprint to maximise density.”
VERTICAL INTEGRATION
In addition to unveiling the WaveLogic 5 Extreme and Nano ICs, Ciena has outlined how it is more vertically integrated after investing in optics. In 2016, Ciena acquired the high-speed photonics division of Teraxion, gaining expertise in indium phosphide and silicon photonics expertise. {add link}.
Ciena is also now selling coherent optical modules. Gazettabyte revealed last year that Ciena was planning to sell modules using its own optics and WaveLogic technologies. {add link}
The company has no preference regarding indium phosphide and silicon photonics and uses what is best for a particular design.
“Silicon photonics buys you ease-of-manufacturing and cost; indium phosphide is what you need for 800 gigabits,” says Xenos.
Ciena stresses, however, that there is no simple formula as to when each is preferred. In terms of size and heat, silicon photonics has a strong advantage. “In terms of performance, you get better performance in some instances with indium phosphide and then there are overlaps because you bring in cost and other constraints,” says Roberts. “So there is no simple divide.”
“As we move forward, we are going to see an increasing percent of Ciena-custom components in WaveLogic coherent modems,” says Xenos.
Source: Gazettabyte
EXTREME
The WaveLogic 5 Extreme introduces several developments. It operates at specific baud rates ranging from 60 to 95 gigabaud. The baud rates are chosen so that both fixed-grid 100GHz channels and flexible grid ones are supported.
“For the best performance, you have flexible grid when 95 gigabaud is the primary baud rate,” says Roberts.
It is also Ciena’s first coherent DSP that uses probabilistic constellation shaping, a coding scheme used to achieve granular capacity increments. {add link}
“From 200 gigabits to 800 gigabits [in 25-gigabit increments], optimised over any path or the available margin,” says Roberts. “But what is unique about this is that it is optimised for non-linear propagation.”
Initially, the products using the WaveLogic 5 Extreme will use 50-gigabit increments. “This is what is required to service customers’ client requirements today: ten gigabits and multiples of 100-gigabit clients,” says Xenos.
>
“With 25-gigabit steps in client rate, the customer can choose to spend the margin on sending more bits”
The DSP uses four-wave frequency-division multiplexing to mitigate non-linear impairments, particularly beneficial for sub-sea systems.
Ciena says the four-wave frequency-division multiplexing is achieved electrically, reducing the optics to a minimum. “One laser and one modulator are used, so all the [cost-saving] economics of a single optical wavelength,” says Roberts. “But it has the non-linear performance of four tightly-coupled electrical systems.”
Ciena has also added an improved forward-error correction (FEC) scheme - a ‘throughput-optimised FEC’ - that uses variable overhead bits depending on the client rate.
“It will handle 8.6 percent errors compared to what we used in the WaveLogic Ai which handles 3.5 percent errors,” says Roberts. “So it is a decibel better.”
The Extreme chip also has improved link-monitoring capabilities. It monitors the signal-to-noise per channel as well as quantifies the non-linear contributions. “It helps people to understand what is happening in the network and create algorithms to optimise the capacity across the network,” says Xenos.
PROBABILISTIC CONSTELLATION SHAPING
Probabilistic shaping is used to improve the optical performance by lowering the signal energy by not using all the constellation points. Unless, that is, the full data rate is used and then all the constellation points are needed.
The degree of probabilistic shaping used is determined for each link. The parameters used to determine the probabilistic shaping are the amount of dispersion on the link, the span’s reach, and the transmitted client rate.
“The modem will measure what is going on in the link and the customer or some higher-level software will say what the client rate is,” says Roberts. “The modem will then figure out how to do the best non-linear probabilistic shaping to support that rate on the link.”
Roberts says other firms’ probabilistic shaping use one symbol at a time whereas Ciena use blocks, each comprising 128 symbols. “A bigger number would be better but I'm limited by my hardware,” says Roberts.
The 128 symbols equate to 1024 bits: four magnitude bits using 64-ary quadrature amplitude modulation (64-QAM) multiplied by two, one for each polarisation.
This means there are a total of 2^1024 combinations of 1024-bit sequences that could be sent. However, when sending a 400 Gigabit Ethernet (GbE) client signaland, for the benefit of explanation, assuming that 555 bits are needed to carry the data payload and the overhead, the number of possible bit sequences is trimmed to 2^555.
This is still a fantastically huge number but the DSP can work out which are the best 555-bit sequences to send based on them having the most tolerance to linear and non-linear interference.
“The ones that play nicely with their neighbours such that they cause the minimum non-linear degradation on the neighbouring wavelengths and on the other symbols,” explains Roberts.
Ciena is not forthcoming as to how it calculates the best sequences. “Ciena’s algorithms decide which ones are best,” says Xenos. “This is one of our key differentiators.”
The result is that, depending on the fibre type, a 1.5dB performance improvement is achieved for the non-linear characteristics.
“It allows more capacity to be chosen by the customer on that same link,” says Roberts. “With 25-gigabit steps in client rate, the customer can choose to spend the margin on sending more bits.”
Operating the Extreme at 95GBd, a reach of 4,000 km is possible at 400 gigabits and at 600 gigabits, the reach is 1,000 km (see table).
WAVELOGIC 5 NANO
The WaveLogic Nano supports 100-gigabit to 400-gigabit wavelengths and is aimed at applications that need compact designs that generate the least heat.
One application is to enable cable operators to move optics closer to the user and that must operate over an extended temperature range. Here, a packet platform is used that will support line interworking as equipment from different vendors may be at each end of the link.
Another requirement is operating over multiple spans in a metro. Here, compact equipment and low power are more important than spectral efficiency but it is still a challenging environment, says Ciena. Hundreds of nodes may be talking to each other and there may be cascaded reconfigurable optical add-drop multiplexers (ROADMs) with different fibre types making up the network.
A third application is single-span data centre interconnect where achieving the highest density on routers is key. This is the application the 400-gigabit, at least 80km 400ZR specification developed by the Open Internetworking Forum will address.
“The design that we are doing for the WaveLogic 5 Nano for 400ZR is to fit into a QSFP-DD,” says Xenos. “If there is a need for an OSFP [pluggable module], we will offer OSFP.”
Ciena also expects to offer a Nano-based CFP2-DCO module, which will outperform the ZR in terms of reach and features, for more demanding metro applications.
Another new segment requiring coherent optics is 4G and 5G access. “It is to be determined what type of platform is the winning solution in this environment,” says Xenos.
MAKING MODULES
Ciena first made its coherent DSP available to third parties in 2017 when it signed an agreement with Lumentum, NeoPhotonics and at the time Oclaro (since acquired by Lumentum) to use its WaveLogic Ai in their modules.
Now Ciena is selling directly the full coherent modem: the DSP and the optics. This is why Ciena created its Optical Microsystems unit in late 2017.
CMOS PROCESS
Moving to a 7nm FinFET CMOS process delivers several benefits.
It generates much lower heat than the WaveLogic Ai’s 28nm FD-SOI process. It also has a lower quiescent current, the current dissipated independent of whether the chip’s logic is active or not. And 7nm CMOS delivers much greater circuit density: the functionality that can be crammed into a square micrometre of silicon.
“So, a low power [consumption] on features you are not using, and we can include features that if you can't afford the heat, you can turn them off,” says Roberts.
It will offer its Nano in the form of pluggable modules, the WaveLogic Ai as a 5x7-inch module, and the WaveLogic 5 Extreme in another module form factor that will have its own interface. “These would all be viable optics,” says Xenos.
Availability
The first Wave Logic 5 Nano products will appear in the second half of this year while the first Extreme-based products will be available at the end of this year. The 400ZR coherent pluggable module is expected to be available in the first half of 2020.
Lumentum completes sale of certain datacom lines to CIG
Brandon Collings, CTO of Lumentum, talks CIG, 400ZR and 400ZR+, COBO, co-packaged optics and why silicon photonics is not going to change the world.
Lumentum has completed the sale of part of its datacom product lines to design and manufacturing company, Cambridge Industries Group.
The sale will lower the company's quarterly revenues by between $20 million to $25 million. Lumentum also said that it will stop selling datacom transceivers in the next year to 18 months.
The move highlights how fierce competition and diminishing margins from the sale of client-side modules is causing optical component companies to rethink their strategies.
Lumentum’s focus is now to supply its photonic chips to the module makers, including CIG. “From a value-add point of view, there is a lot more value in selling those chips than the modules,” says Brandon Collings, CTO of Lumentum.
400ZR and ZR+
Lumentum will continue to design and sell line-side coherent optical modules, however.
“With coherent, there is a lot of complexity and challenge in the module’s design and manufacture,” says Collings. “We believe we can extract the value we need to continue in that business.”
The emerging 400ZR and 400ZR+ are examples of such challenging coherent interfaces.
The 400ZR specification, developed by the Optical Internetworking Forum (OIF), is a 400-gigabit coherent interface with an 80km reach. The 400 gigabit-per-second (Gbps) line rate will be achieved using a 64-gigabaud symbol rate and a 16-QAM modulation scheme.
>
“[400ZR] is not client-side. Sixty-four gigabaud is very hard to do in such an extremely compact form factor.
”
Module makers will implement the 400ZR interface using client-side pluggable modules such as the QSFP-DD and the OSFP to enable data centre operators to add coherent interfaces directly to their switches.
But implementing 400ZR will be a challenge. “This is not client-side,” says Collings. “Sixty-four gigabaud is very hard to do in such an extremely compact form factor.”
First samples of 400ZR modules are expected by year-end.
The 400ZR+ interface, while not a specification, is a catch-all for a 400-gigabit coherent that exceeds the 400ZR specification. The 400ZR+ will be a multi-rate design that will support additional line rates of 300, 200 and 100Gbps. Such rates coupled with more advanced forward-error correction (FEC) schemes will enable the 400ZR+ to span much greater distances than 80km.
The 400ZR+ interface helps the developers of next-generation coherent DSP chips to recoup their investment by boosting the overall market their devices can address. “It is basically a way of saying I’m going to spend $50 million developing a coherent DSP, and the 400ZR market alone is not big enough for that investment,” says Collings.
Lumentum says there will be some additional functionality that will be possible to fit into a QSFP-DD such that at least one of the ZR+ modes will be supported. But given the QSFP-DD module’s compactness and power constraints, the ZR+ will also be implemented in the CFP2 form factor that has the headroom needed to fully exploit the coherent DSP’s capabilities to also address metro and regional networks.
400ZR+ modules are expected in volume by the end of 2020 or early 2021.
DSP economics
Lumentum will need to source a coherent DSP for its 400ZR/ ZR+ designs as it does not have its own coherent chip. At the recent OFC show held in San Diego, the talk was of new coherent DSP players entering the marketplace to take advantage of the 400ZR/ZR+ opportunity. Collings says he is aware of five DSP players but did not cite names.
NEL and Inphi are the two established suppliers of merchant coherent DSPs. Lumentum (Oclaro) has partnered with Acacia Communications to use its Meru DSP for Lumentum’s CFP2-DCO design, although it is questionable whether Acacia will license its DSP for 400ZR/ ZR+, at least initially.
>
“God forbid if 10 or more players are doing this as no matter how you slice it, people will be losing [money]”
Lumentum and Oclaro also partnered with Ciena to use its WaveLogic Ai for a long-haul module. That leaves room for at least one more provider of a coherent DSP that could be a new entrant or an established system vendor that will license an internal design.
Collings points out that it makes no sense economically to have more than five players. If it takes $50 million to tape out a 7nm CMOS coherent DSP, the five players will invest a total of $250 million. And if the investment cost for the module, photonics and everything else is a comparable amount, that equates to $500 million being spent on the 400-gigabit coherent generation.
As for the opportunity, Collings talks of about a total of up to 500,000 ports a year by 2020. That equates to an investment return in the first year of $1,000 per device sold. “God forbid if 10 or more players are doing this as no matter how you slice it, people will be losing [money].”
Beyond Pluggables
The evolution of optics beyond pluggables was another topic under discussion at OFC.
The Consortium of On-Board Optics (COBO), the developerof an interoperable optical solution that embeds optics on the line card, had a stand at the show and a demonstration of its technology. In turn, co-packaged optics, the stage after COBO in the evolution of optical interfaces that will integrate the optics with the silicon in one package, is also now also on companies' agenda.
Collings explains that COBO came about because the industry thought on-board optics would be needed given the challenge of 400-gigabit pluggables meeting the interface density needed for 12.8-terabit switches . “I shared that opinion four to five years ago,” he says, adding that Lumentum is a member of COBO.
>
“That problem is real. It is a matter of how far the current engineering can go before it becomes too painful.”
But 400-gigabit optics has been engineered to meet the required faceplate density, including ZR for coherent. As a result, COBO is less applicable. “That need to break the paradigm is a lot less,” he says.
That said, Collings says COBO has driven valuable industry discussion given that the data centre is heading in a direction where 32 ports of 800-gigabit interfaces will be needed to get data in and out of next-generation, 25-terabit switches.
“That problem is real,” says Collings. “It is a matter of how far the current engineering can go before it becomes too painful.” Scaling indefinitely what is done today is not an option, he says.
It is possible with the next generation of switch chip to simply use a two-rack-unit box with twice as many 400-gigabit modules. “That has already been done at the 100-gigabit generation that lasted longer because it doubled up the 100-gigabit port count,” he says.
“In the generation after that, you are now asking for stuff that looks very challenging with today’s technology,” he says. “And that is where co-packaging is focused, the 50-terabit switch generation.” Switches using such capacity silicon are expected in the next four years.
But this is where it gets tricky, as co-packaging not only presents significant technical challenges but also will change the supply chain and business models.
Collings points out that hyperscalars do not like making big pioneering investments in new technology, rather they favour buying commodity hardware. “They don’t like risk, they love competition, and they like a healthy ecosystem,” he says.
“There is a lot of talk from the technology direction of how we can solve this problem [using co-packaged optics] but I think on the business side, the riskside, the investment side is putting a lot of pressure on that actually happening,” says Collings. “Where it ends up I don’t honestly know.”
Silicon photonics
One trend evident at OFC was the growing adoption of silicon photonics by optical component companies.
Indeed, the market research firm, LightCounting, in a research note summarising OFC 2019, sees silicon photonics as a must-have technology given co-packaged optics is now clearly on the industry’s roadmap.
However, Collings stresses that Lumentum’s perspective remains unchanged regarding the technology.
“It’s a fabless exercise so we can participate in silicon photonics and, quite frankly, that is why a lot of other companies are participating because the barrier to entry is quite low,” says Collings. “Nevertheless, we look at silicon photonics as another tool in the toolbox: it has advantages in some areas, some significant disadvantages in others, and in some places, it is simply comparable.”
When looking at a design from a system perspective such as a module, other considerations come into play besides the cost of the silicon photonics chip itself. Collings cites the CFP2 coherent module. While the performance of its receiver is good using silicon photonics, the modulator is questionable. You also need a laser and a semiconductor optical amplifier to compensate for silicon photonics higher loss, he says,
The alternative is to use an indium phosphide-based design and that has its own design issues. “What we are finding when you look at the right level is that the two are the same or indium phosphide has the advantage,” says Collings. “And as we go faster, we are finding silicon is not really keeping up in bandwidth and performance.”
As a result, Lumentum is backing indium phosphide for coherent operating at 64 gigabaud.
“A lot of people are talking about silicon photonics because they can talk about it,” says Collings. “It’s not worthless, don’t get me wrong, but its success outside of Acacia has been niche, and Acacia is top notch at doing this stuff.”
Acacia bets on silicon as coherent enters its next phase
Gazettabyte interviewed Acacia Communications’ president and CEO, Murugesan ‘Raj’ Shanmugaraj, as the coherent technology company celebrates its 10th anniversary.
Raj Shanmugaraj
Acacia Communications has come a long way since Raj Shanmugaraj (pictured) first joined the company as CEO in early 2010. “It was just a few conference rooms and we didn't have enough chairs,” he says.
The company has since become a major optical coherent player with revenues of $340 million in 2018; revenues that would have been higher but for the four-month trade ban imposed by the US on Chinese equipment maker ZTE, an Acacia customer.
And as the market for coherent technology continues to grow, Acacia and other players are preparing for new opportunities.
“We are still in the early stages of the disruption," says Shanmugaraj. “You will see higher performance [coherent systems] in some parts of the network but there is going to be growth as coherent moves closer to the network edge.”
Here, lower power, flexibility and more integrated coherent solutions will be needed as the technology moves inside the data centre and closer to the network edge with the advent of 5G, higher-speed access and the Internet of Things (IoT).
Competitive landscape
Shanmugaraj prefers to focus on Acacia’s own strengths and products when asked about the growing competition in the coherent marketplace. However, recent developments present challenges for the company.
Systems vendors such as Huawei and Ciena are becoming more vertically integrated, developing not only their own coherent digital signal processor (DSP) ASICs but also optics. Ciena has also made its WaveLogic Ai DSP available to optical module makers Lumentum and NeoPhotonics and will sell its own optical modules using its latest WaveLogic 5 coherent silicon.
>
“You will see higher performance [coherent systems] in some parts of the network but there is going to be growth as coherent moves closer to the network edge ”
New coherent digital signal processor (DSP) players are also expected to enter the marketplace alongside established competitors, NEL and Inphi. The entrance of new players developing coherent DSPs is motivated by the unit volumes promised by 400ZR, the emerging 80km data centre interconnect interface standard.
“We are proponents of the fact that the merchant market will continue to grow, driven by interoperability and standardisation,” says Shanmugaraj. Such growth will lead to multiple markets where coherent technology will play. “There are going to be a few winners, not just one or two,” he says.
Acacia’s revenues were hit in 2018 following the US Department of Commerce’s enforced trade ban imposed on ZTE. However, the company recorded a strong fourth quarter posting revenues of $107 million, up almost a quarter on the revenues a year earlier. This followed strong ZTE orders after the ban was revoked.
Shanmugaraj says diversification has always been a priority for the company, independent of the trade issues between the US and China. The company has also been working to diversify its Chinese customer base. “So we are well positioned as these trade issues get resolved,” he says.
Origins
Acacia was established in mid-2009 by a core team from Mintera, a sub-system supplier that provided 40-gigabit DPSK line cards to network equipment suppliers. But Mintera folded and was eventually sold to Oclaro in July 2010.
Before joining Acacia, Shanmugaraj was at systems vendor Alcatel-Lucent where he learned two lessons.
One is that the long-term success of a company is based on technology leadership. “You want to be driven by technology or you fall behind your competitors,” he says. The second lesson was that the largest systems companies build products internally before an ecosystem becomes established, after which they buy from merchant suppliers.
This matched the vision of Acacia’s founders that sought to exploit their optical expertise gained at Mintera to become a leading merchant supplier of coherent transmission technology.
Stealth years
Acacia remained in secrecy for nearly half its existence, only revealing its technology and products in 2014 with the launch of the AC-100 CFP coherent pluggable module. The AC-100 is aimed at metro networks delivering a transmission reach of 80km to 1,200km. However, Acacia had already been selling 5x7-inch modules for 100-gigabit long-haul and ultra-long-haul applications as well as a 40-gigabit ultra-long-haul module.
“In the early years, there were just a few companies working on coherent,” says Shanmugaraj. “We had to be careful in terms of what products we were developing and what customers we were going after.”
Shanmugaraj says Acacia secured multi-million dollar commitments from customers even before it had a product. “It was the expertise of the founding team as well as the product concepts they were proposing that got them the commitments,” he says.
The backing enabled the company to manage with only $53 million of venture funding prior to its successful initial public offering in 2016.
“This was a pretty significant feat,” says Shanmugaraj. “Hardware start-ups, whether semiconductor or systems companies, use significantly more cash; these are expensive technologies to get off the ground.”
Shanmugaraj describes the early years as intense, with staff working between 60 and 70 hours a week.The then start-up had to be prudent with funding, not growing too quickly yet having sufficient resources to meet orders from systems customers that had their own orders to fulfil.
Coherent technologies
Acacia’s founders chose silicon for its coherent solutions, to replace ‘exotic materials’ such as indium phosphide and lithium niobate used in traditional optical transmission systems.
The company backed silicon photonics for the coherent optics, an industry trailblazing decision. To this aim, Acacia recruited Chris Doerr, the renowned optical integration specialist and Bell Labs Fellow.
The company also decided to develop its own coherent DSPs. By developing the optics and the DSP, Acacia could use a co-design approach when designing the hardware, trading off the performance of the optics and the signal processing to achieve an optimal design.
Shanmugaraj explains that the company chose a silicon-based approach to exploit the huge investment made by the semiconductor industry in chips and their packaging. Basing the components on silicon would not only simplify high-speed networks, he says, but it would also lower their power consumption and enable products to be made more quickly and cheaply.
“The beauty of silicon photonics is that it can be placed right next to a heat source, in this case, the high-power coherent DSP ASIC that generates a lot of heat,” says Shanmugaraj. “This allows for smaller form-factor designs.” In contrast, indium phosphide-based optics need to be temperature controlled when placed next to a hot chip, he says.
“Five or six years ago, people were challenging whether silicon photonics was even going to work at 100 and 200 gigabits,” says Shanmugaraj. Acacia has now used silicon photonics in all its products, including its latest high-end 1.2 terabits AC1200 coherent module.
Shanmugaraj sees Acacia's portfolio of coherent products as the company's biggest achievement: "You see start-ups that come out with one product that is a bestseller but we have continued to innovate and today we have a broad portfolio."
AC1200
The AC1200 module supports two optical wavelengths, each capable of supporting 100 to 600-gigabit transmissions in increments of 50 gigabits.
The AC1200 can be used for data centre interconnect links through to long distance submarine links. Acacia recently demonstrated the AC1200 transmitting a 400-gigabit signal over a 6,600km submarine cable.
“We are seeing strong interest in our AC1200 from network operators and expect our equipment customers to begin deployments this quarter,” says Shanmugaraj.
There are several reasons why network operators are choosing to deploy the AC1200, he says: “High capacity is important in data centre interconnect edge applications where we expect hyperscale operators may use the AC1200 in its full 1.2-terabit mode, but these applications are also sensitive to cost, power and density.”
The AC1200 also provides higher capacity in a smaller footprint than the 5x7-inch form factors currently available, he says, while for longer-reach applications, the AC1200 offers a combination of performance and flexibility that is setting the pace for the competition.
The data centre interconnect market represents a good opportunity for coherent interconnect suppliers because the operators drive and deploy technology at pace, says Shanmugaraj. Hyperscalers are continually looking to add more capacity in the same size and power constraints that exist today. Accordingly, this has been a priority development area for Acacia.
To increase capacity, companies have boosted the symbol rate from 32 gigabaud to 64 gigabaud while systems vendors Ciena and Infinera have recently detailed upcoming systems that support 800-gigabit wavelengths that use a symbol rate approaching 100 gigabaud.
The AC1200, which is due in systems in the coming quarter, demonstrates silicon photonics based modulation operating at up to 70 gigabaud while first indium-phosphide 800-gigabit per wavelength systems are due by the year-end.
“We don’t really see silicon photonics lagging behind indium phosphide,” says Shanmugaraj. “We think there is a path to even higher baud rates with silicon photonics, and 128 gigabaud is the next logical step up because it would double the data rate without needing to increase the modulation order.”
Higher modulation orders are also possible but the benefits must be weighed against increased complexity, he says.
400-gigabit coherent pluggables
Shanmugaraj says that the 400ZR pluggable module standard continues the trend to reduce the size and power consumption of optical transport systems in the data centre.
>
“You want to be driven by technology or you fall behind your competitors”
The current generation of data centre interconnect platforms, ranging from a 1 rack unit pizza box to a several rack-unit-sized chassis, were developed to be more compact than conventional optical transport platforms.
Now, with the advent of 400ZR that fits into a client-side QSFP-DD or OSFP module, data centre operators will be able to do away with such platforms for distances up to 80km by plugging the modules into the switch or router platforms and connecting them to open line systems.
“Costs come down because it [coherent] is getting down to the client-side form factors and that gives the hyperscalers more faceplate density,” says Shanmugaraj. “The hyperscalers also gain multi-vendor interoperability [with 400ZR] which is important as they want standardisation.”
Shanmugaraj admits that with the advent of 400ZR will bring greater competition. But he points out that the 400ZR is a complicated product to built that will challenge companies. Those players that have both the optics and a low-power DSP will have an advantage. “As long as it opens up the market wider, it is good for Acacia as it is in our control how we can win in the market,” says Shanmugaraj.
The industry expectation is that the 400ZR will start to be deployed in the second half of 2020.
There is also industry talk about 400ZR+, an interface that will be able to go beyond 80km that will require more advanced dispersion compensation and forward error correction schemes.
Shanmugaraj says it will be the same DSP ASIC that will support both the 400ZR and 400ZR+. However, a 400ZR+ interface will consume more power and so will likely require a larger module form factor than the ZR.
Meanwhile, the 400-gigabit CFP2-DCO pluggable for metro networks is built along the same lines as the 400ZR, says Shanmugaraj.
“Here you have applications like the Open ROADM MSA where network operators are trying to drive the same interoperability and not be stuck with one vendor,” he says. “This is driving the 400-gigabit evolution in the metro network for some of the largest telcos.”
There is also the open networking packet-optical opportunity, white-box platforms such as the Voyager and Cassini being developed by the Telecom Infra Project (TIP). Shanmugaraj says such white boxes rely on software solutions that are a work-in-progress and that much work is still to be done.
“The first generation showed that there is more work required to standardise the software and how that can be used by the hyperscalers,” he says. “It is an opportunity but we view it as more of a longer-term one.”
Emerging opportunities
The markets that are growing today are the metro, long haul, sub-sea and data centre interconnect, says Shanmugaraj.
The coherent applications that are emerging will result in products within the data centre as well as for 5G, access, the Internet of Things (IoT) and even autonomous vehicles.
Ultimately, what will lead to coherent being adopted within the data centre is the speed of the interfaces. “As you go to higher speeds, direct detection technology gets constrained [due to dispersion and other impairments],” says Shanmugaraj.
But for this to happen certain conditions will need to be met: the speed of interfaces on switches will need to increase, not just to 400 gigabits but 800 gigabits and greater.
“Looking to higher data rates beyond 400 gigabits, it gets more challenging for direct detect to achieve the necessary link budgets cost-effectively,” says Shanmugaraj. “It may be necessary to move from four-lane solutions to eight lanes in order to support the desired reaches. At the same time, we are working to make coherent more cost-effective for these applications.”
The other two conditions are the challenge of what form factors the coherent technology be squeezed into, andcost. Coherent optics is more expensive but its cost is driven by such factors as volumes, the level of automation that can be used to make the module, and the yield.
“There could be inflextion points where coherent becomes cost-competitive for some applications in the data centre,” says Shanmugaraj.
Companies will continue to innovate in both direct detect and coherent technologies and the market will determine the transition points. “But we do believe that coherent can be adopted inside data centres in the future,” he says.
In turn, metro and long-haul networks are already being upgraded in anticipation of 5G and the access requirements. “4G networks have a lot of 1-gigabit and 10-gigabit links but 5G has an order of magnitude higher throughput requirement,” says Shanmugaraj.
That means more capacity is needed for backhaul and that will lead to a proliferation of low-cost 100-gigabit coherent. A similar story is unfolding in access with the likes of the cable operators moving fibre closer to the network edge. This too will need low-cost 100-gigabit coherent interfaces.
IoT is a longer term opportunity and will be dependent on dense deployments of devices before the traffic will require sufficient aggregation to justify coherent.
“I don’t know if your refrigerator will have a coherent interface,” concludes Shanmugaraj. “But as you aggregated these [devices] into aggregation points, that becomes a driver for coherent at the edge.”
Acacia eyes pluggables as it demos its AC1200 module
The emerging market opportunity for pluggable coherent modules is causing companies to change their strategies.
Ciena is developing and plans to sell its own coherent modules. And now Acacia Communications, the coherent technology specialist, says it is considering changing its near-term coherent digital signal processor (DSP) roadmap to focus on coherent pluggables for data centre interconnect and metro applications.
Source: Gazettabyte
DSP roadmap
Acacia’s coherent DSP roadmap in recent years has alternated between an ASIC for low-power, shorter-reach applications followed by a DSP to address more demanding, long-haul applications.
In 2014, Acacia announced its Sky 100-gigabit DSP for metro applications that was followed in 2015 by its Denali dual-core DSP that powers its 400-gigabit AC-400 5x7-inch module. Then, in 2016, Acacia unveiled its low-power Meru, used within its pluggable CFP2-DCO modules. The high-end 1.2-terabit dual-core Pico DSP used for Acacia’s board-mounted AC1200 coherent module was unveiled in 2017.
“The 400ZR is our next focus,” says Tom Williams, senior director of marketing at Acacia.
The 400ZR standard, promoted by the large internet content providers, is being developed to link switches in separate data centres up to 80km apart. Acacia’s subsequent coherent DSP that follows the 400ZR may also target pluggable applications such as 400-gigabit CFP2-DCO modules that will span metro and metro-regional distances.
“There is a trend to pluggable, not just the 400ZR but the CFP2-DCO [400-gigabit] for metro,” says Williams. “We are still evaluating whether that causes a shift in our overall cadence and DSP development.”
AC1200 trials
Meanwhile, Acacia has announced the results of two transatlantic trials involving its AC1200 module whose production is now ramping.
>
“There is a trend to pluggable, not just the 400ZR but the CFP2-DCO [400-gigabit] for metro”
In the first trial, Acacia, working with ADVA, transmitted a 300-gigabit signal over a 6,800km submarine cable. The 300-gigabit wavelength occupied a 70GHz channel and used ADVA’s Teraflex technology, part of ADVA’s FSP 3000 CloudConnect platform. Teraflex is a one-rack-unit (1RU) stackable chassis that supports three hot-pluggable 1.2-terabit sleds, each sled incorporating an Acacia AC1200 module.
In a separate trial, the AC1200 was used to send a 400-gigabit signal over 6,600km using the Marea submarine cable. Marea is a joint project between Microsoft, Facebook and Telxius that links the US and Spain. The cable is designed for performance and uses an open line system, says Williams: “It is not tailored to a particular company’s [transport] solution”.
The AC1200 module - 40 percent smaller than the 5x7-inch AC400 module - uses Acacia’s patented Fractional QAM (quadrature amplitude modulation) technology. The technology uses probabilistic constellation shaping that allows for non-integer constellations. “Instead of 3 or 4 bits-per-symbol, you can have 3.56 bits-per-symbol,” says Williams.
Acacia’s Fractional QAM also uses an adaptive baud rate. For the trial, the 400-gigabit wavelength was sent using the maximum baud rate of just under 70 gigabaud. Using the baud rate to the full allows a lower constellation to be used for the 400-gigabit wavelength thereby achieving the best optical signal-to-noise ratio (OSNR) and hence reach.
In a second demonstration using the Marea cable, Acacia demonstrated a smaller-width channel in order to maximise the overall capacity sent down the fibre. Here, a lower baud rate/ higher constellation combination was used to achieve a spectral efficiency of 6.41 bits-per-second-per-Hertz (b/s/Hz). “If you built out all the channels [on the fibre], you achieve of the order of 27 terabits,” says Williams.
Pluggable coherent
The 400ZR will be implemented using the same OSFP and QSFP-DD pluggable modules used for 400-gigabit client-side interfaces. This is why an advanced 7nm CMOS process is needed to implement the 400ZR DSP so that its power consumption will be sufficiently low to meet the modules’ power envelopes when integrated with Acacia’s silicon-photonics optics.
There is also industry talk of a ZR+, a pluggable module with a reach exceeding80km. “At ECOC, there was more talk about the ZR+,” says Williams. “We will see if it becomes standardised or just additional proprietary performance.”
Another development is the 400-gigabit CFP2-DCO. At present, the CFP2-DCO delivers up to 200-gigabitwavelengths but the standard, as defined by the Optical Internetworking Forum (OIF), also supports 400 gigabits.
Williams says that there a greater urgency to develop the 400ZR than the 400-gigabit CFP2-DCO. “People would like to ramp the ZR pretty close to the timing of the 400-gigabit client-side interfaces,” says Williams. And that is likely to be from mid-2019.
In contrast, the 400-gigabit CFP2-DCO pluggable while wanted by carriers for metro applications, is not locked to any other infrastructure build-out, says Williams.
400ZR will signal coherent’s entry into the datacom world
- 400ZR will have a reach of 80km and a target power consumption of 15W
- The coherent interface will be available as a pluggable module that will link data centre switches across sites
- Huawei expects first modules to be available in the first half of 2020
- At OFC, Huawei announced its own 250km 400-gigabit single-wavelength coherent solution that is already being shipped to customers
Coherent optics will finally cross over into datacom with the advent of the 400ZR interface. So claims Maxim Kuschnerov, senior R&D manager at Huawei.
Maxim Kuschnerov400ZR is an interoperable 400-gigabit single-wavelength coherent interface being developed by the Optical Internetworking Forum (OIF).
The 400ZR will be available as a pluggable module and as on-board optics using the COBO specification. The IEEE is also considering a proposal to adopt the 400ZR specification, initially for the data-centre interconnect market. “Once coherent moves from the OIF to the IEEE, its impact in the marketplace will be multiplied,” says Kuschnerov.
But developing a 400ZR pluggable represents a significant challenge for the industry. “Such interoperable coherent 16-QAM modules won’t happen easily,” says Kuschnerov. “Just look at the efforts of the industry to have PAM-4 interoperability, it is a tremendous step up from on-off keying.”
Despite the challenges, 400ZR products are expected by the first half of 2020.
400ZR use cases
The web-scale players want to use the 400ZR coherent interface to link multiple smaller buildings, up to 80km apart, across a metropolitan area to create one large virtual data centre. This is a more practical solution than trying to find a large enough location that is affordable and can be fed sufficient power.
Once coherent moves from the OIF to the IEEE, its impact in the marketplace will be multiplied
Given how servers, switches and pluggables in the data centre are interoperable, the attraction of the 400ZR is obvious, says Kuschnerov: “It would be a major bottleneck if you didn't have [coherent interface] interoperability at this scale.”
Moreover, the advent of the 400ZR interface will signal the start of coherent in datacom. Higher-capacity interfaces are doubling every two years or so due to the webscale players, says Kuschnerov, and with the advent of 800-gigabit and 1.6-terabit interfaces, coherent will be used for ever-shorter distances, from 80km to 40km and even 10km.
At 10km, volumes will be an order of magnitude greater than similar-reach dense wavelength-division multiplexing (DWDM) interfaces for telecom. “Datacom is a totally different experience, and it won’t work if you don’t have a stable supply base,” he says. “We see the ZR as the first step combining coherent technology and the datacom mindset.”
Data centre players will plug 400ZR modules into their switch-router platforms, avoiding the need to interface the switch-router to a modular, scalable DWDM platform used to link data centres.
The 400ZR will also find use in telecom. One use case is backhauling residential traffic over a cable operator’s single spans that tend to be lossy. Here, ZR can be used at 200 gigabits - using 64 gigabaud signalling and QPSK modulation - to extend the reach over the high-loss spans. Similarly, the 400ZR can also be used for 5G mobile backhaul, aggregating multiple 25-gigabit streams.
Another application is for enterprise connectivity over distances greater than 10km. Here, the 400ZR will compete with direct-detect 40km ER4 interfaces.
Having several use cases, not just data-centre interconnect, is vital for the success of the 400ZR. “Extending ZR to access and metro-regional provides the required diversity needed to have more confidence in the business case,” says Kuschnerov.
The 400ZR will support 400 gigabits over a single wavelength with a reach of 80km, while the target power consumption is 15W.
The industry is still undecided as to which pluggable form factor to use for 400ZR. The two candidates are the QSFP-DD and the OSFP. The QSFP-DD provides backward compatibility with the QSFP+ and QSFP28, while the OSFP is a fresh design that is also larger. This simplifies the power management at the expense of module density; 32 OSFPs can fit on a 1-rack-unit faceplate compared to 36 QSFP-DD modules.
The choice of form factor reflects a broader industry debate concerning 400-gigabit interfaces. But 400ZR is a more challenging design than 400-gigabit client-side interfaces in terms of trying to cram optics and the coherent DSP within the two modules while meeting their power envelopes.
The OSFP is specified to support 15W while simulation results published at OFC 2018 suggest that the QSFP-DD will meet the 15W target. Meanwhile, the 15W power consumption will not be an issue for COBO on-board optics, given that the module sits on the line card and differs from pluggables in not being confined within a cage.
Kuschnerov says that even if it proves that only the OSFP of the two pluggables supports 400ZR, the interface will still be a success given that a pluggable module will exist that delivers the required face-plate density.
400G coherent
Huawei announced at OFC 2018 its own single-wavelength 400-gigabit coherent technology for use with its OptiX OSN 9800 optical and packet OTN platform, and it is already being supplied to customers.
The 400-gigabit design supports a variety of baud rates and modulation schemes. For a fixed-grid network, 34 gigabaud signalling enables 100 gigabits using QPSK, and 200 gigabits using 16-QAM, while at 45 gigabaud 200 gigabits using 8-QAM is possible. For flexible-grid networks, 64 gigabaud is used for 200-gigabit transmission using QPSK and 400 gigabits using 16-QAM.
Huawei uses an algorithm called channel-matched shaping to improve optical performance in terms of data transmission and reach. This algorithm includes such techniques as pre-emphasis, faster-than-Nyquist, and Nyquist shaping. According to Kuschnerov, the goal is to squeeze as much capacity out of a network’s physical channel so that advanced coding techniques such as probabilistic constellation shaping can be used to the full. For Huawei’s first 400-gigabit wavelength solution, constellation shaping is not used but this will be added in its upcoming coherent designs.
Huawei has already demonstrated the transmission of 400 gigabits over 250km of fibre. “Current generation 400G-per-lambdas does not enable long-haul or regional transmission so the focus is on shorter reach metro or data-centre-interconnect environments,” says Kuschnerov.
When longer reaches are needed, Huawei can offer two line cards, each supporting 200 gigabits, or a single line card hosting two 200-gigabit modules. The 200-gigabits-per-wavelength is achieved using 64 gigabaud and QPSK modulation, resulting in a 2,500km reach.
Up till now, such long-haul distances have been served using 100-gigabitwavelengths. Now, says Kuschnerov, 200 gigabit at 64 gigabaud is becoming the new norm in many newly built networks while the 34 gigabaud 200 gigabit is being favoured in existing networks based on a 50GHz grid.
COBO issues industry’s first on-board optics specification
- COBO modules supports 400-gigabit and 800-gigabit data rates
- Two electrical interfaces have been specified: 8 and 16 lanes of 50-gigabit PAM-4 signals.
- There are three module classes to support designs ranging from client-slide multi-mode to line-side coherent optics.
- COBO on-board optics will be able to support 800 gigabits and 1.6 terabits once 100-gigabit PAM-4 electrical signals are specified.
Source: COBO
Interoperable on-board optics has moved a step closer with the publication of the industry’s first specification by the Consortium for On-Board Optics (COBO).
COBO has specified modules capable of 400-gigabits and 800-gigabits rates. The designs will also support 800-gigabit and 1.6-terabit rates with the advent of 100-gigabit single-lane electrical signals.
“Four hundred gigabits can be solved using pluggable optics,” says Brad Booth, chair of COBO and principal network architect for Microsoft’s Azure Infrastructure. “But if I have to solve 1.6 terabits in a module, there is nothing out there but COBO, and we are ready.”
Origins
COBO was established three years ago to create a common specification for optics that reside on the motherboard. On-board optics is not a new technology but until now designs have been proprietary.
I have to solve 1.6 terabits in a module, there is nothing out there but COBO, and we are ready
Brad BoothSuch optics are needed to help address platform design challenges caused by continual traffic growth.
Getting data on and off switch chips that are doubling in capacity every two to three years is one such challenge. The input-output (I/O) circuitry of such chips consumes significant power and takes up valuable chip area.
There are also systems challenges such as routing the high-speed signals from the chip to the pluggable optics on the platform’s faceplate. The pluggable modules also occupy much of the faceplate area and that impedes the air flow needed to cool the platform.
Using optics on the motherboard next to the chip instead of pluggables reduces the power consumed by shortening the electrical traces linking the two. Fibre rather than electrical signals then carries the data to the faceplate, benefiting signal integrity and freeing faceplate area for the cooling.
Specification 1.0
COBO has specified two high-speed electrical interfaces. One is 8-lanes wide, each lane being a 50-gigabit 4-level pulse-amplitude modulation (PAM-4) signal. The interface is based on the IEEE’s 400GAUI-8, the eight-lane electrical specification developed for 400 Gigabit Ethernet.
The second electrical interface is a 16-lane version for an 800-gigabit module. Using a 16-lane design reduces packaging costs by creating an 800-gigabit module instead using two separate 400-gigabit ones. Heat management is also simpler with one module.
There are also systems benefits using an 800-gigabit module.“As we go to higher and higher switch silicon bandwidths, I don’t have to populate as many modules on the motherboard,” says Booth.
The latest switch chips announced by several companies have 12.8 terabits of capacity that will require 32, 400-gigabit on-board modules but only 16, 800-gigabit ones. Fewer modules simplify the board’s wiring and the fibre cabling to the faceplate.
Designers have a choice of optical formats using the wider-lane module, such as 8x100 gigabits, 2x400 gigabits, and even 800 gigabits.
COBO has tested its design and shown it can support a 100-gigabit electrical interface. The design uses the same connector as the OSFP pluggable module.
“In essence, with an 8-lane width, we could support an 800-gigabit module if that is what the IEEE decides to do next,” says Booth. “We could also support 1.6 terabits if that is the next speed hop.”
It is very hard to move people from their standard operating model to something else until there is an extreme pain point
Form factor and module classes
The approach chosen by COBO differs from proprietary on-board optics designs in that the optics is not mounted directly onto the board. Instead, the COBO module resembles a pluggable in that once placed onto the board, it slides horizontally to connect to the electrical interface (see diagram, top).
A second connector in the middle of the COBO module houses the power, ground and control signals. Separating these signals from the high-speed interface reduces the noise on the data signals. In turn, the two connectors act as pillars supporting the module.
The robust design allows the modules to be mounted at the factory such that the platform is ready for operation once delivered at a site, says Booth.
COBO has defined three module classes that differ in length. The shortest Class A modules are used for 400-gigabit multi-mode interfaces while Class B suits higher-power IEEE interfaces such as 400GBASE-DR4 and the 100G Lambda MSA’s 400G-FR4.
The largest Class C module is for the most demanding and power-hungry designs such as the coherent 400ZR standard. “Class C will be able to handle all the necessary components - the optics and the DSP - associated with that [coherent design],” says Booth.
The advantage of the on-board optics is that it is not confined to a cage as pluggables are. “With an on-board optical module, you can control the heat dissipation by the height of the heat sink,” says Booth. “The modules sit flatter to the board and we can put larger heat sinks onto these devices.”
We realised we needed something as a stepping stone [between pluggables and co-packaged optics] and that is where COBO sits
Next steps
COBO will develop compliance-testing boards so that companies developing COBO modules can verify their designs. Booth hopes that by the ECOC 2018 show to be held in September, companies will be able to demonstrate COBO-based switches and even modules.
COBO will also embrace 100-gigabit electrical work being undertaken by the OIF and the IEEE to determine what needs to be done to support 8-lane and 16-lane designs. For example, whether the forward-error correction needs to be modified or whether existing codes are sufficient.
Booth admits that the industry remains rooted to using pluggables, while the move to co-packaged optics, where the optics and the chip are combined in the same module - remains a significant hurdle, both in terms of packaging technology and the need for vendors to change their business models to build such designs.
“It is very hard to move people from their standard operating model to something else until there is an extreme pain point,” says Booth.
Setting up COBO followed the realisation that a point would be reached when faceplate pluggables would no longer meet demands while in-packaged technology would not be ready.
“We realised we needed something as a stepping stone and that is where COBO sits,” says Booth.
Further information
For information on the COBO specification, click here.
COBO targets year-end to complete specification
Part 3: 400-gigabit on-board optics
- COBO will support 400-gigabit and 800-gigabit interfaces
- Three classes of module have been defined, the largest supporting at least 17.5W
The Consortium for On-board Optics (COBO) is scheduled to complete its module specification this year.
A draft specification defining the mechanical aspects of the embedded optics - the dimensions, connector and electrical interface - is already being reviewed by the consortium’s members.
Brad Booth“The draft specification encompasses what we will do inside the data centre and what will work for the coherent market,” says Brad Booth, chair of COBO and principal network architect for Microsoft’s Azure Infrastructure.
COBO was established in 2015 to create an embedded optics multi-source agreement (MSA). On-board optics have long been available but until now these have been proprietary solutions.
“Our goal [with COBO] was to get past that proprietary aspect,” says Booth. “That is its true value - it can be used for optical backplane or for optical interconnect and now designers will have a standard to build to.”
The draft specification encompasses what we will do inside the data centre and what will work for the coherent market
Specification
The COBO modules are designed to be interchangeable. Unlike front-panel optical modules, the COBO modules are not ‘hot-pluggable’ - they cannot be replaced while the card is powered. But the design allows for COBO modules to be interchanged.
The COBO design supports 400-gigabit multi-mode and single-mode optical interfaces. The electrical interface chosen is the IEEE-defined CDAUI-8, eight lanes each at 50 gigabits implemented using a 25-gigabit symbol rate and 4-level pulse-amplitude modulation (PAM-4). COBO also supports an 800-gigabit interface using two tightly-coupled COBO modules.
The consortium has defined three module categories that vary in length. The module classes reflect the power envelope requirements; the shortest module supports multi-mode and the lower-power module designs while the longest format supports coherent designs. “The beauty of COBO is that the connectors and the connector spacings are the same no matter what length [of module] you use,” says Booth.
The COBO module is described as table-like, a very small printed circuit board that sits on two connectors. One connector is for the high-speed signals and the other for the power and control signals. “You don't have to have the cage [of a pluggable module] to hold it because of the two-structure support,” says Booth.
To be able to interchange classes of module, a ‘keep-out’ area is used. This area refers to board space that is deliberately left empty to ensure the largest COBO module form factor will fit. A module is inserted onto the board by first pushing it downwards and then sliding it along the board to fit the connection.
Booth points out that module failures are typically due to the optical and electrical connections rather than the optics itself. This is why the repeated accuracy of pick-and-place machines are favoured for the module’s insertion. “The thing you want to avoid is having touch points in the field,” he says.
Coherent
A working group was set up after the Consortium first started to investigate using the MSA for coherent interfaces. This work has now been included in the draft specification. “We realised that leaving it [the coherent work] out was going to be a mistake,” says Booth.
The main coherent application envisaged is the 400ZR specification being developed by the Optical Internetworking Forum (OIF).
The OIF 400ZR interface is the result of Microsoft’s own Madison project specification work. Microsoft went to the industry with several module requirements for metro and data centre interconnect applications.
Madison 1.0 was a two-wavelength 100-gigabit module using PAM-4 that resulted in Inphi’s 80km ColorZ module that supports up to 4 terabits over a fibre. Madison 1.5 defines a single-wavelength 100-gigabit module to support 6.4 to 7.2 terabits on a fibre. “Madison 1.5 is probably not going to happen,” says Booth. “We have left it to the industry to see if they want to build it and we have not had anyone come forward yet.”
Madison 2.0 specified a 400-gigabit coherent-based design to support a total capacity of 38.4 terabits - 96 wavelengths of 400 gigabits.
Microsoft initially envisioned a 43 gigabaud 64-QAM module. However, the OIF's 400ZR project has since adopted a 60-gigabaud 16-QAM module which will achieve either 48 wavelengths at 100GHz spacing or 64 wavelengths at 75GHz spacing, capacities of 19.2Tbps and 25.6Tbps, respectively.
In 2017, the number of coherent metro links Microsoft will use will be 10x greater than the number of metro and long-haul coherent links it used in 2016.
Once Microsoft starting talking about Madison 2.0, other large internet content providers came forward saying they had similar requirements which led to the initiative being driven into the OIF. The result is the 400ZR MSA that the large-scale data centre players want to be built by as many module companies as possible.
Booth highlights the difference in Microsoft’s coherent interface volume requirements just in the last year. In 2017, the number of coherent metro links Microsoft will use will be 10x greater than the number of metro and long-haul coherent links it used in 2016.
“Because it is an order of magnitude more, we need to have some level of specification, some level of interop because now we're getting to the point where if I have an issue with any single supplier, I do not want my business impeded by it,” he says.
Regarding the COBO module, Booth stresses that it will be the optical designers that will determine the different coherent specifications possible. Thermal simulation work already shows that the module will support 17.5W and maybe more.
“There is a lot more capability in this module that there is in a standard pluggable only because we don't have the constraint of a cage,” says Booth. “We can always go up in height and we can always add more heat sink.”
Booth says the COBO specification will likely need a couple more members’ reviews before its completion. “Our target is still to have this done by the end of the year,” he says.
Amended on Sept 4th, added comment about the 400ZR wavelength plans and capacity options
The OIF’s 400ZR coherent interface starts to take shape
Part 2: Coherent developments
The Optical Internetworking Forum’s (OIF) group tasked with developing two styles of 400-gigabit coherent interface is now concentrating its efforts on one of the two.
When first announced last November, the 400ZR project planned to define a dense wavelength-division multiplexing (DWDM) 400-gigabit interface and a single wavelength one. Now the work is concentrating on the DWDM interface, with the single-channel interface deemed secondary.
Karl Gass"It [the single channel] appears to be a very small percentage of what the fielded units would be,” says Karl Gass of Qorvo and the OIF Physical and Link Layer working group vice chair, optical, the group responsible for the 400ZR work.
The likelihood is that the resulting optical module will serve both applications. “Realistically, probably both [interfaces] will use a tunable laser because the goal is to have the same hardware,” says Gass.
The resulting module may also only have a reach of 80km, shorter than the original goal of up to 120km, due to the challenging optical link budget.
Origins and status
The 400ZR project began after Microsoft and other large-scale data centre players such as Google and Facebook approached the OIF to develop an interoperable 400-gigabit coherent interface they could then buy from multiple optical module makers.
The internet content providers’ interest in an 80km-plus link is to connect premises across the metro. “Eighty kilometres is the magic number from a latency standpoint so that multiple buildings can look like a single mega data centre,” says Nathan Tracy of TE Connectivity and the OIF’s vice president of marketing.
Since then, traditional service providers have shown an interest in 400ZR for their metro needs. The telcos’ requirements are different to the data centre players, causing the group to tweak the channel requirements. This is the current focus of the work, with the OIF collaborating with the ITU.
The catch is how much can we strip everything down and still meet a large percentage of the use cases
“The ITU does a lot of work on channels and they have a channel measurement methodology,” says Gass. “They are working with us as we try to do some division of labour.”
The group will choose a forward error correction (FEC) scheme once there is common agreement on the channel. “Imagine all those [coherent] DSP makers in the same room, each one recommending a different FEC,” says Gass. “We are all trying to figure out how to compare the FEC schemes on a level playing field.”
Meeting the link budget is challenging, says Gass, which is why the link might end up being 80km only. “The catch is how much can we strip everything down and still meet a large percentage of the use cases.”
The cloud is the biggest voice in the universe
400ZR form factors
Once the FEC is chosen, the power envelope will be fine-tuned and then the discussion will move to form factors. The OIF says it is still too early to discuss whether the project will select a particular form factor. Potential candidates include the OSFP MSA and the CFP8.
Nathan TracyThe industry assumption is that the 80km-plus 400ZR digital coherent optics module will consume around 15W, requiring a very low-power coherent DSP that will be made using 7nm CMOS.
“There is strong support across the industry for this project, evidenced by the fact that project calls are happening more frequently to make the progress happen,” says Tracy.
Why the urgency? “The cloud is the biggest voice in the universe,” says Tracy. To support the move of data and applications to the cloud, the infrastructure has to evolve, leading to the data centre players linking smaller locations spread across the metro.
“At the same time, the next-gen speed that is going to be used in these data centres - and therefore outside the data centres - is 400 gigabit,” says Tracy.