How DSP smarts continue to improve optical transport

- Kim Roberts explains the signal processing techniques Ciena is using for its WaveLogic 6 coherent DSP.
- Roberts explains how the techniques squeeze, on average, a 15 per cent improvement in spectral efficiency.
- The WaveLogic 6 Extreme chip can execute 1,600 trillion (1.6 x 1015) operations per second and uses the equivalent of 4km of on-chip copper interconnect.
Part 2: WaveLogic 6’s digital signal processing toolkit
Bumping into Kim Roberts on the way to the conference centre at OFC, held in San Diego in March, I told him how, on the Ciena briefing about its latest WaveLogic 6 coherent digital signal processor (DSP), there had been insufficient time to dive deeply into the signal processing techniques used.
“What are you doing now?” said Roberts.
“I’m off to the plenary session to catch the keynotes.”
Chatting some more, I realised I was turning down a golden opportunity to sit down with a leading DSP and coherent modem architect.
“Is that offer still open?” I asked.
He nodded.
We grabbed a table at a nearby cafe and started what would prove to be an hour-long conversation.
High-end coherent DSPs
Many leading coherent modem vendors unveiled their latest designs in the run-up to the OFC show. It is rare for so many announcements to be aligned, providing a valuable glimpse of the state of high-performance optical transport.
Nokia announced its PSE-6s, which has a symbol rate of up to 130 gigabaud (GBd) and supports 1.2 terabit wavelengths. Infinera announced its 1.2-terabit ICE-7, which has a baud rate of up to 148GBd, while Fujitsu detailed it is using its 135GBd 1.2-terabit wavelength coherent DSP for its 1FINITY Ultra optical platform.
Meanwhile, Acacia, a Cisco company, revealed its 140GBd Jannu 1.2-terabit DSP has been shipping since late 2022. Acacia announced the Jannu DSP in March 2022.
All these coherent DSPs are implemented using 5nm CMOS and are shipping or about to.
And Ciena became the first company to detail a coherent DSP fabricated using a 3nm CMOS process. The WaveLogic 6 Extreme supports 1.6-terabit wavelengths and has a symbol rate of up to 200GBd.
Ciena’s WaveLogic 6 Extreme improves spectral efficiency by, on average, 15 per cent. WaveLogic 6 Extreme-based coherent modems will be available from the first half of 2024.
Customer considerations
Kim Roberts begins by discussing what customers want.
“With terrestrial systems, it is cost-per-bit [that matters], and if you’re not going very far, it is cost-per-modem,” says Roberts.
For the shortest reaches (tens of km), 100 gigabit may be enough while 200 gigabit or more is overkill. Here, a coherent pluggable module does the job.
“What matters is the cost per modem to get the flexibility of coherent connectivity so that you can plug it in and it works,” says Roberts.
With medium and long-haul terrestrial routes, cost-per-bit and heat-per-bit are the vital issues. With heat, area and volume of the coherent design are important. “I need volume to get the heat out of the chip on the card and into the air,” says Roberts.
Another use case is where spectral efficiency is key, for networks where fibre is scarce. An operator could be leasing dark fibre, or it could be a submarine network.
Ciena’s WaveLogic 6 Extreme’s 15 per cent improvement in spectral efficiency improves capacity over the same link. “Equivalently, you can go a dB (decibel) further or have a dB more signal margin,” says Roberts.
A common refrain heard is that spectral efficiency is no longer improving due to the Shannon limit being approached. Shannon’s limit is being approached because of the considerable progress already made by the industry in coherent optics.
“There is no 6dB to be had like in the old days,” says Roberts. “WaveLogic 3 was 2.5dB better than WaveLogic 2, but those multiple dBs are no longer there.”
The returns are diminishing, but striving for improvements remains worthwhile. “If you’re an operator that cares about spectral efficiency, that’s important,” he says.
Nonlinearity mitigation
Roberts returns to the issue of Shannon’s limit, based on the work of famed mathematician and information theorist, Claude Shannon.
“Shannon defines a theoretical limit for the capacity of a channel having linear propagation with additive Gaussian noise,” says Roberts.
This defines a strict mathematical limit, and it is pointless to go beyond that; he says: “In terms of linear performance, modems are getting close to the limit, within a couple of dB.”
Shannon’s limit doesn’t wholly define fibre since the channel is nonlinear.
Roberts says there is a whole research area addressing the bounds given such nonlinearities.
“We’re a long way from those theoretical nonlinear limits, but what matters is what’s the practical limit, and it’s getting hard,” he says
Increasing transmit power improves the optical signal-to-noise ratio (OSNR) and strengthens nonlinearities. Indeed, the nonlinearities grow faster with increased transmit power until, eventually, they dominate.
Because tackling nonlinearities is so complicated, Ciena’s approach is to approximate the problem as a linear Gaussian noise channel and do everything possible to mitigate the effects of nonlinearity rather than embrace it.
This is done by compensating at the transmitter the nonlinearities expected to happen along the fibre. The receiver performs measurements on a second-by-second basis and sends the results back. These are used as estimates of the anticipated nonlinearity about to be encountered and subtracted from the symbols to be sent.

Even though the exact nonlinearity is unknown, this is still a valid approximation. “It gives a quarter to one dB of performance improvement,” says Roberts
Edgeless clock recovery
Robert explains other clever signal processing techniques that buy a 6 per cent spectral efficiency improvement.
With wavelength division multiplexing (WDM), the laser-generated signals are placed next to each other across the fibre’s spectrum.
For WaveLogic 6, when running at its maximum symbol rate of 200 gigabaud, the spectrum occupies a 200GHz-wide channel.
Usually, the signal in the frequency domain is not perfectly square-shaped; the signal rolls off in the frequency domain so that in the time domain there is no inter-symbol interference. “But [as a result] you’re wasting spectrum; you are not fully using that spectrum,” says Roberts.

With WaveLogic 6, Ciena has created an idealised flat-topped, vertically edged signal spectra allowing the signals to be crammed side by side thereby making best use of the fibre’s spectrum (see diagram).
The challenge is that the clocking information used for data recovery at the receiver resided in this roll-off region. Now, that is no longer there so Cienahas developed another method to recover clock information.
A second challenge with signal recovery is that the transmit laser and the receive laser are not rigidly fixed in frequency. Being so close together, care is needed to recover only the wavelength – signal – of interest.
Yet another complication is how a rectangle in the frequency domain causes the signal in the time domain to ‘ring’ and go on forever.
“There are several signal processing methods that we had to develop to make this possible,” says Roberts.
Frequency-division multiplexing
Ciena also uses frequency division multiplexing (FDM), a technique it first introduced with the WaveLogic 5 Extreme.
The difference between WDM and FDM, explains Roberts, is that WDM uses different lasers to generate the wavelengths while FDM is generated by applying digital techniques to the same laser. “You are digitally combining different streams,” he says.
This is useful because it turns out that each fibre route has an optimum baud rate because of nonlinearities.
“If I’m using the full symbol rate of 200GBd, I can divide that into parallel streams, which behave as if they were independent circuits as far as nonlinearity is concerned,” says Roberts. “The optimum number of FDM in your spectrum is proportional to the square root of the total amount of dispersion, so high dispersion, more FDMs, low dispersion, just one.”
Ciena first added the option of four FDM with the WaveLogic 5. Now, WaveLogic 6 implements 1,2,4, and 8 FDM channels.

“For short distances, you want to go one signal at 200 gigabaud, or smaller if you’re reducing baud rate, but if you’re going very long distances, lots of dispersion, you go at eight parallel streams being sent at 25 gigabaud each,” he says.
But introducing FDM causes notches in the near-idealised rectangular spectrum mentioned earlier. Ciena has had to tackle that too.
“If you measure the spectrum, it’s completely flat, there are no notches between the FDMs, there is no wasted spectrum,” says Roberts.
Multi-dimensional coding
Multi-dimensional coding is a further technique used by Ciena to improve optical transmission, especially in troublesome cables where there are much nonlinearity and noise. It is challenging to get information through.
To understand multi-dimensional constellations, Roberts uses the example of a 16-QAM constellation, which he describes as a two-dimensional (2D) representation in one polarisation.
But if both polarisations of light are considered one signal, it becomes a 4D, 256-point (16×16) symbol. This can be further extended by including the symbols in adjacent time slots to form an 8D representation.
Ciena introduced this technique with its WaveLogic 3 Extreme coherent DSP, which supported the multi-dimension coding scheme 8D-2QAM to improve the reach or capacity of long-reach spans.
Now Ciena has introduced a family of such multi-dimensional schemes with WaveLogic 6 Extreme, executing in regions of very high nonlinearity and noise. These include 4, 8, and 16-dimensional constellations.

An example where the technique is used includes cases where there is twice as much noise as there is signal. “So the signal-to-noise ratio is -3dB,” says Roberts. Yet even here, 100 gigabits can still get through.
WaveLogic 6 Nano
Ciena also announced its 3nm CMOS WaveLogic 6 Nano DSP aimed at pluggable coherent modules. Is the Nano’s role to implement a subset of the signal processing capabilities of the Extreme?
Here, the customer’s requirements are different: heat, space and footprint are the dominant concerns. The Nano has to fit the heat envelope of the different sizes of pluggables, says Roberts. The optical performance is chosen based on fitting that heat requirement.
One of the merits of 3nm FinFET transistor technology is that if you don’t clock a circuit, only 1 per cent of the heat is generated compared to when it’s clocked, notes Roberts: “So, for different features, I can turn off the clock.”
A suitcase still full of tools?
At the time of the WaveLogic 5 launch, Roberts mentioned that there were still many tools left in the suitcase of ideas. Is this still true with the WaveLogic 6?
For Roberts, the question is: will it be economically viable to put in new capabilities based on the heat and performance and in terms of the size, schedule, and the amount of work involved?
Then, with a broad smile, he says: “There is room to occupy us as to how to get the next 10 to 20 per cent of spectral efficiency.”
And with that, we each set off for a day of meetings.
Roberts headed off to his hotel before his 10am meeting. I set off for the OFC exhibition hall and a meeting with the OIF.
As I walked to the convention centre, I kept thinking about the impromptu briefing and how I so nearly passed up on Roberts’ expertise and generosity.
Has coherent optical transmission run its course?
Feature: Coherent's future
Three optical systems vendors share their thoughts about coherent technology and the scope for further improvement as they look two generations ahead to symbol rates approaching 100 gigabaud
Optical transmission using coherent detection has made huge strides in the last decade. The latest coherent technology with transmitter-based digital signal processing delivers 25x the capacity-reach of 10-gigabit wavelengths using direct-detection, according to Infinera.
Since early 2016, the optical systems vendors Infinera, Ciena and Nokia have all announced new coherent digital signal processor (DSP) designs. Each new generation of coherent DSP improves the capacity that can be transmitted over an optical link. But given the effectiveness of the latest coherent systems, has most of the benefits already been achieved?
Source: Infinera
“It is getting harder and harder,” admits Kim Roberts, vice president, WaveLogic science at Ciena. “Unlike 10 years ago, there are no factors of 10 available for improvement.”
Non-linear Shannon limit
It is the non-linear Shannon limit that defines how much information can be sent across a fibre, a function of the optical signal-to-noise ratio.
Kim Roberts of CienaThe limit is based on the work of famed mathematician and information theorist, Claude Shannon. Shannon's work was based on a linear communication channel with added Gaussian noise. Optical transport over a fibre is a more complex channel but the same Shannon bound applies, although assumptions for the non-linearities in the fibre must be made.
Roberts stresses that despite much work, the industry still hasn't figured out just what the upper limit is over a fibre for a given optical signal-to-noise ratio.
It is getting harder and harder. Unlike 10 years ago, there are no factors of 10 available for improvement.
"There are papers that show that with this method and this method, you can do this much," says Roberts. "And there are other papers that show that as the power goes up, there is no theoretical limit until you melt the fibre."
These are theoretical things, he says, but the key is that the headroom available remains unknown. What is known is that the theoretical limit remains well ahead of practical systems. Accordingly, systems performance can be improved using a combination of techniques and protocols coupled with advances in electro-optics.
Design goals
A key goal when designing a new optical transmission system is to increase the data sent for a given cost i.e. decrease the cost-per-bit. This is an ongoing requirement as the service providers contend with ever growing network traffic.
Another challenge facing engineers is meeting the demanding power, density and thermal constraints of their next-generation optical transport system designs.
One way to reduce the cost-per-bit is to up the symbol rate to increase the data sent over a wavelength. Traditional 100-gigabit and 200-gigabit dense wavelength-division multiplexing (DWDM) systems use 32-35 gigabaud (GBaud). The latest coherent DSPs already support more than one baud rate: Nokia’s PSE-2s coherent DSP supports 33Gbaud or 45Gbaud while Ciena’s WaveLogic Ai chipset supports 35Gbaud or 56Gbaud.
Having a choice of baud rates coupled with the various modulation scheme options means the same number of bits can be sent over a range of optical reaches. The more complex the modulation scheme, the closer the points are in a constellation and the harder it is to correctly detect the data at the receiver in the presence of noise. Accordingly, using the combination of a simpler modulation scheme and a higher baud rate allows the same data to be sent further.
Capacity-reach is what matters: how much capacity you can extract for a given reach
Nokia's 1.4-billion transistor PSE-2s supports two 200 gigabit-per-second (Gbps) formats: polarisation-multiplexing, 16-ary quadrature amplitude modulation (PM-16QAM) at 33Gbaud, or using PM-8QAM at 45Gbaud. The 200-gigabit wavelength has an optical reach of some 800km using 16-QAM at 33Gbaud but this rises to 1,600km when PM-8QAM at 45Gbaud is used. Alternatively, using 45Gbaud and PM-16QAM, more data can be sent: 250 gigabits-per-wavelength over 800km.
Nokia's Randy EisenachCoherent systems designers are not stopping there. “The next higher baud rate the industry is targeting is 61-68 Gbaud,” says Randy Eisenach, senior product marketing manager, optical networks at Nokia.
Operating at the higher gigabaud range - Infinera talks of 65-70Gbaud - a single transmitter-receiver pair sends twice the amount of data of traditional 32-35Gbaud systems using the same modulation format. But the higher-baud rates require the electro-optics to operate twice as fast. The analogue-to-digital and digital-to-analogue converters of the coherent DSP must sample at twice the baud rate - at least 130 billion samples-per-second. A 65-70Gbaud rate also requires silicon implemented using a more advanced and expensive CMOS process mode - 16nm instead of 28nm. In turn, the optical modulator and drivers need to work well at these higher rates.
“The optical networking industry is well on its way to solving these engineering and component issues in the next year or so,” says Eisenach.
The capacity-per-wavelength also goes up with baud rate. For shorter reach links, 400-600 gigabits-per-wavelength are possible at 65-70Gbaud and, according to Pravin Mahajan, Infinera’s director of product and corporate marketing, power consumption in terms of watts-per-gigabit will improve by some 2.5x.
Pravin Mahajan of InfineraAnd the system vendors are not stopping there: the next baud rate hike after 65-70Gbaud will be in the region of 80-100 Gbaud. The coherent DSPs that will support such data rates will need to be implemented using 7nm CMOS process (see table).
“Capacity-reach is what matters: how much capacity you can extract for a given reach,” says Mahajan. “These successive generations [of faster baud rates] all keep moving that curve upwards.”
DSP features
In addition to the particular baud rates chosen by the vendors for their DSP designs, each includes unique features.
Instead of modulating the data onto a single carrier, Infinera’s FlexCoherent DSP uses multiple Nyquist sub-carriers spread across a channel. The number of subs-carriers varies depending on the link. The benefit of the approach, says Infinera, is that it allows a lowering of the baud rate used which increases the tolerance to non-linear channel impairments experienced during optical transmission.
The FlexCoherent DSP also supports enhanced soft-decision forward-error correction (SD-FEC) including the processing of two channels that need not be contiguous. This is possible as the FlexCoherent DSP is dual-channel which particularly benefits long-haul and subsea applications, claims Infinera. By pairing two channels, the FEC codes can be shared. Pairing a strong channel with a weak one and sharing the codes allows some of the strength of the strong signal to be used to bolster the weaker one, extending its reach or even allowing a more advanced modulation scheme to be used.
Infinera has just announced that by using Nyquist sub-carriers and the FEC gain sharing technologies, its customer, Seaborn Networks, is able delivering 11.8 terabits of capacity over a 10,600km submarine link.
Nokia’s PSE-2s DSP has sufficient processing performance to support two coherent channels. Each channel can implement a different modulation format if desired, or the two can be tightly coupled to form a super-channel. Using 45Gbaud and PM-16QAM, two 250-gigabit channels can be implemented to enable a 500-gigabit muxponder card. The PSE-2s can also implement 400-gigabit wavelength but that is the only format where only one channel can be supported by the PSE-2s.
Ciena’s WaveLogic Ai, meanwhile, uses advanced coding schemes such that it no longer mentions particular modulation schemes but rather a range of line rates in 50-gigabit increments.
Coding schemes with names such as set-partition QPSK, matrix-enhanced PM-BPSK, and 8D-2QAM, have already started to appear in the vendors’ coherent DSPs.
“Vendors use a lot of different terms essentially for the same thing: applying some type of coding to symbols to improve performance,” says Eisenach.
There are two main coding approaches: constellation shaping, also known as probabilistic shaping, and multi-dimensional coding. Combining the two - probabilistic shaping and multi-dimensional coding - promises enhanced performance in the presence of linear and non-linear transmission impairments. These are now detailed.
Probabilistic shaping
The four constellation points of QPSK modulation are equidistant from the origin. With more advanced modulation schemes such as 16-QAM, the constellation points differ in their distance from the origin and hence have different energies. Points in the corners of the constellation, furthest from the origin, have the most energy since a point’s power is the square of the distance from the origin.
Here the origin is at the centre of the square 64-QAM constellation. With probabilistic shaping, more of the points closer to the origin are chosen with the resulting data rate going down. Source: Nokia
Probabilistic shaping uses the inner constellation points more than the outer points, thereby reducing the overall average energy and this improves the signal-to-noise ratio. To understand why, Ciena points out that the symbol error rate at the receiver is dominated by the distance between neighbouring points of the constellation. Reduced the average energy still keeps the distance between the points the same, but when gain is applied to restore the signal’s power levels, the effect is to increase the distance between points. “It means we have better separation between the points, we’ve expanded everything,” says Roberts.
Using probabilistic shaping delivers a maximum 1.53dB of improvement in a linear transmission channel. “That is the theoretical limit,” says Roberts. “In a non-linear world, we get a greater benefit from shaping beyond just shaping the noise.”
Probabilistic shaping also has another benefit: it allows the number of bits sent per symbol to be defined.
Using standard modulation schemes such as 64-QAM with no constellation shaping, 6 bits-per-symbol are sent. Using shaping and being selective in what points are used, fewer bits are sent and they don’t need to be integer values. “I can send 5.7, 5.6, 5.3, even 5.14 bits-per symbol,” says Roberts. “Until I get to 5 bits, and then I have a choice: do I use more shaping or do I start with 32-QAM, which is 5 bits-per-symbol.”
Technology A shows today's coherent DSPs: operating at 30-35Gbaud and delivering 100, 150 and 200Gbps capacities per wavelength. Technology B is Ciena's WaveLogic A. Operating at 56Gbaud, it delivers up to 400Gbps per wavelength in 50Gbps. Technology C will continue this trend. Operating around 70Gbaud, up to 600Gbps per wavelength will be possible in even finer speed increments of 25Gbps. Is this Ciena's next WaveLogic? Source: Ciena
This is very useful as it allows fine control of the data sent such that operators can squeeze just enough data to suit the margins available on a particular fibre link. “You don't have to choose between 100-gigabit and 200-gigabit wavelengths,” says Roberts. "You can use smaller jumps and that sometimes means sending more capacity.”
Three things are needed to fine-tune a link in this way. One is a coherent DSP that can deliver such variable increments on a wavelength using probabilistic shaping. Also needed is a flexible client signalling scheme such as the OIF’s Flexible Ethernet (FlexE) protocol, a protocol mechanism to vary the Ethernet payload for transmission. Lastly, intelligent networking software is required to determine what is happening in the network and the margins available to assess how much data can be squeezed down a link.
Ciena says it has not implemented probabilistic shaping in its latest WaveLogic Ai coherent DSP. But given the Ai will be a family of devices, the technique will feature in upcoming coherent DSPs.
Nokia published a paper at the OFC event held earlier this year showing the use of probabilistic shaping over a transatlantic link. Using probabilistic-shaped 64-QAM (PS-64QAM), a spectral efficiency of 7.46b/s/Hz was achieved over the 5,523km link. This equates to 32 terabits of capacity over the fibre, more than 2.5x the 12 terabits of the existing DWDM system that uses 100Gbps PM-QPSK.
Advanced coding
Multi-dimensional coding is another technique used to improve optical transmission. A 16-QAM constellation is a two-dimensional (2D) representation in one polarisation, says Roberts. But if both polarisations of light are considered as one signal then it becomes a 4D, 256-point (16x16) symbol. This can be further extended by including the symbols in adjacent time slots. This forms an 8D representation.
Non-linear compensation has been an interesting research topic. Nokia continues to investigate the topic and implementation methods but the benefits appear small for most real-world applications
The main two benefits of multi-dimensional coding are better noise performance and significantly better performance in the presence of non-linear impairments.
Nokia’s PSE-2s uses coding for its set-partition QPSK (SP-QPSK). Standard PM-QPSK uses amplitude and phase modulation, resulting in a 4-point constellation. With SP-QPSK, only three of the four constellation points are used for each symbol. A third fewer constellation points means less data is transported but the benefit of SP-QPSK is extended reach due to the greater Euclidean distance between the symbol points created by carefully mapping the sequence of symbols. This results in 2.5dB of extra gain compared to PM-QPSK, for a reach beyond 5,000km.
Using the PSE-2’s 45Gbaud symbol rate, the fewer constellation points of SP-QPSK can be compensated for to achieve the same overall 100Gbps capacity as PM-QPSK at 33Gbaud.
Infinera’s FlexCoherent uses what it calls matrix-enhanced PM-BPSK, a form of averaging that adds 1dB of gain. “Any innovation that adds gain to a link, the margin that you give to operators, is always welcome,” says Mahajan.
Ciena’s WaveLogic 3 Extreme coherent DSP supports the multi-dimension coding scheme 8D-2QAM to improve reach or capacity of long-reach spans.
Such techniques mean vendors have a wealth of available choices available. It is also why Ciena has stopped referring to modulation schemes and talks about its WaveLogic Ai at 35Gbaud supporting 100-250Gbps data rates in 50-gigabit increments while at 56Gbaud, the WaveLogic Ai delivers 100-400Gbps optical channels in 50-gigabit steps.
Probabilistic shaping and multi-dimensional coding are distinct techniques but combining the two means the shaping can be done across dimensions.
Design engineers thus have various techniques to keep improving performance and there are other directions too.
Forward-error correction is about 2dB from the theoretical limit and with improved design Ciena’s Roberts expects 1dB can be reclaimed.
In turn, signal processing techniques could be applied at the transmitter to compensate for expected non-linear effects. “Non-linear compensation has been an interesting research topic,” says Eisenach. “Nokia continues to investigate the topic and implementation methods but the benefits appear small for most real-world applications.”
So is there much scope for further overall improvement?
“There is still a lot more juice left," says Mahajan.
“It [coherent transmission improvement] is getting harder and harder,” adds Roberts. “It is taking more mathematics and more and more CMOS gates, but Moore’s law is providing lots of CMOS gates.”
This is an updated and extended version of an article that first appeared in Optical Connections magazine earlier this year.
