Ciena becomes a computer weaver
- Ciena is to buy optical interconnect start-up Nubis Communications for $270 million.
- The deal covers optical and copper interconnect technology for data centres
Ciena has announced its intention to buy optical engine specialist Nubis Communications for $270 million. If the network is the computer, Nubis’ optical engine and copper integrated circuit (IC) expertise will help Ciena better stitch together AI’s massive compute fabric.
Ciena signalled its intention to target the data centre earlier this year at the OFC show when it showcased its high-speed 448-gigabit serialiser-deserialiser IC technology and coherent lite modem. Now, Ciena has made a move for start-up Nubis, which plays at the core of AI data centres.
“Ciena’s expertise in high-speed components is relevant to 400G per lane Ethernet transceivers, but they never sold any products to this market,” says Vladimir Kozlov, CEO of LightCounting. “Nubis offers them an entry point with several designs and customer engagements.”
With the deal, Ciena is extending its traditional markets of wide area networks (WAN), metro, and short-reach dense wavelength division multiplexing (DWDM) to include AI networking opportunities. These opportunities include scale-across networks, where AI workloads are shared across multiple data centres, something Ciena can address, to now scale-out and scale-up networks for AI clusters in the data centre.
Puma optical engine
Nubis has developed two generations of compact optical engines for near-package optics (NPO) and co-package optics (CPO) applications. Its first-generation engine operates at 100 gigabits per second (Gbps), while its second, dubbed Puma, operates at 200 Gbps.
Nubis’s optical engine philosophy is based on escaping the optical channels from the surface of the optical engine, not its edge. The start-up also matches the number of optical channels to the electrical ones. The optical engine can be viewed as a sieve: data from the input channels flow through the chip and emerge in the same number of channels at the output. The engine acts as a two-way gateway, with one side handling electrical signals and the other, optical ones.
The Puma optical engine uses 16 channels in each direction, 16 by 200Gbps electrical signals for a total of 3.2 terabits per second (Tbps), and 16 fibres, each fibre carrying 200Gbps of data in the form of a wavelength. Puma’s total capacity is thus 6.4 terabits per second (Tbps). The engine also needs four external lasers to drive the optics, each laser feeding four channels or four fibres. The total fibre bundle of the device consists of 36 fibres: 32 for data (16 for receive and 16 for transmit), and four for the laser light sources.
Nubis is also a proponent of linear drive technology. Here, the advanced serdes on the adjacent semiconductor chip drives the optical engine, thereby avoiding the need for an on-engine digital signal processor (DSP) that requires power. The start-up has also developed a system-based simulator software tool that it uses to model the channel, from the transmitter to the receiver. The tool models not only the electrical and optical components within the channel but also the endpoints, such as the serdes.
Nitro
Nubis has an analogue IC team that designs its trans-impedance amplifiers (TIAs) and drivers used for the optical engine. The hardware compensates for channel impairments with low noise, high linearity, and at high speed. It is this channel simulator tool that Nubis used to optimise its optical engine, and to develop its second key technology, which Nubis calls Nitro —a chip that extends the reach of copper cabling.
“We use our linear optics learning and apply it to copper straight out of the gate, “said Peter Winzer, founder & CTO at Nubis, earlier this year. By using its end-to-end simulator tool, Nubis developed the Nitro IC, which extends the 1m reach of direct-attached copper to 4m using an active copper cable design. “We don’t optimise the driver chip, we optimise the end-to-end system,” says Winzer.
Nubis was also part of a novel design based on a vertical line card to shorten the trace length between an ASIC and pluggable modules.
Ciena’s gain
The acquisition of Nubis places Ciena at the heart of the electrical-optical transition inside the data centre. Ciena will cover both options: copper and optical interconnect. Ciena will gain direct-drive technology expertise for electrical and optical interfaces, enabling scale-up, as well as optical engine technology for scale-out, adding to its coherent technology expertise.
Ciena’s technologies will span coherent ultra-long-haul links all the way to AI accelerators, the heart of AI clusters. By combining Ciena’s 448-gigabit serdes with Nubis’s optical engine expertise, Ciena has a roadmap to develop 12.8Tbps and faster optical engines.
The acquisition places Ciena among new competitors that have chip and optical expertise and deliver co-packaged optics solutions alongside complex ICs such as Broadcom and Marvell.
The deal adds differentiation from Ciena’s traditional system vendor competitors, such as Cisco/ Acacia and Nokia. Huawei is active in long-haul optical and makes AI clusters. Ciena will also compete with existing high-speed optical players, including co-packaged optics specialists Ayar Labs and Ranovus, microLED player Avicena, and optical/IC fabric companies such as LightMatter and Celestial AI.
“Ciena will be a unique supplier in the co-packaged optics/near-packaged optics/active copper cabling data centre interconnect market,” says Daryl Inniss, Omdia’s thought lead of optical components and advanced fibre. “The other suppliers either have multiple products in the intra data centre market, like Broadcom and Nvidia, or they are interconnect-focused start-ups. These suppliers should all wonder what Ciena will do next inside the data centre.”
Ciena will enhance its overall expertise in chips, optics, and signal processing with the Nubis acquisition. It will also put Ciena in front of key processor players and different hyperscaler engineering teams, which drive next-generation AI systems.
Ciena will also have all the necessary parts for the various technologies, regardless of the evolving timescales associated with the copper-to-optical transition within AI systems. Ciena will add direct-detect technology and copper interconnect. On the optical side, it has coherent optical expertise, now coupled with near-package optics and co-packaged optics.
Nubis’ gain
Nubis’ 50-plus staff get a successful exit. The start-up was founded in 2020. Nubis will become a subsidiary of Ciena.
Nubis will be joining a much bigger corporate entity with deep expertise and pockets. Ciena has a good track record with its mergers. Think Nortel at the system level and Blue Planet, a software acquisition. Now the Nubis deal will bring Ciena firmly inside the data centre.
“This is a great deal for Nubis,” says Kozlov. “Congratulations to their team.”
What next?
The deal is expected to close in the fourth quarter of this year. Ciena expects the deal to start adding to its revenues from 2028, requiring Ciena and Nubis to develop products and deliver design wins in the data centre.
“Given the breadth of Ciena’s capabilities, its deep pockets, and products like its data centre out-of-band (DCOM) measurement product, router, and coherent transceivers, one can imagine that Ciena would offer more than co-packaged optics/ near-packaged optics/ active copper cabling inside the data centre,” says Inniss.
Nubis' bandwidth-packed tiny optical engine
- Nubis Communications has revealed its ambitions to be an optical input-output (I/O) solutions provider
- Its tiny 1.6-terabit optical engine measures 5mm x 7.5mm
- The optical engine has a power consumption of below 4 picojoule/bit (pJ/b) and a bandwidth density of 0.5 terabits per millimetre.
- “Future systems will be I/O with an ASIC dangling off it.”
Nubis Communications has ended its period of secrecy to unveil an optical engine targeted at systems with demanding data input-output requirements.
The start-up claims its optical engine delivers unmatched bandwidth density measured in terabits per millimetre (T/mm) and power consumption performance metrics.
“In the timeframe of founding the company [in 2020], it became obvious that the solution space [for our product] was machine learning-artificial intelligence,” says Dan Harding, the CEO of Nubis.
Company Background
Nubis has raised over $40 million, with the lead investor being Matrix Partners. Venture capital company Matrix Partners backed Acacia Communications, acquired by Cisco in 2021.
Other Nubis backers are Weili Dai, a co-founder of Marvell Technologies, and Belgium-based imec.xpand.
“We have raised enough money to get to production with our product,” says Harding, who joined Nubis in 2021 from Broadcom.
Peter Winzer is the CTO and founder of the company. Formerly at Nokia Bell Labs, Winzer was the 2018 winner of the Optica (then OSA) and IEEE Photonics Society’s John Tyndall Award for his work on coherent optical communications.
Nubis has 40 staff, mostly engineers.
“As a team, we are multidisciplinary,” says Winzer. The company’s expertise includes silicon photonics, analogue IC design including serialisers/ deserialisers (serdes), packaging – electrical and optical, and software including advanced simulation tools.
“It is all geared towards a systems solution,” says Winzer. “We are not just looking at the PIC [photonic integrated circuit] or the electronics; we have the system and the architecture in mind.”
The input-output challenge
Machine learning workloads continue to grow at a staggering pace, doubling more than twice each year. Not surprisingly, computing systems running such workloads are struggling to keep up.
Scaling such systems not only requires more processing – more graphics processing units (GPUs) – but also networking to connect clusters of GPUs.
What the compute vendors want is any-to-any connectivity between processors and between clusters. This is creating a tremendous input-output challenge in terms of bandwidth density while keeping the power consumption under control.
“Over half the power of that cluster can be taken up by traditional optics,” says Harding. “So it is clear that the industry wants new solutions.”
“Whatever cents-per-gigabit [figure] you use, if you multiply it by the I/O capacity, the number you’ll get is many times that of [the cost of] an ASIC,” adds Winzer. “We say that future systems will be I/O with an ASIC dangling off it.”
Design details
Nubis’ optical engine is a 16 x 112-gigabit design with a footprint of 5mm x 7.5mm.
“Because we have our electronics flip-chipped on top, that’s the entire footprint,” says Winzer. “We maintain that it is the highest density by far of any optical engine.”
Nubis says many parallel fibres can be interfaced to the optical engine despite its tiny size.
Supporting parallel fibres is essential for machine learning systems as the fibres are fanned out to enable any-to-any connectivity.
Nubis’ engine uses a 4 by DR4 fan-out architecture with 36 fibres arranged in a 3×12 array.
Surface coupling in a 2D array interfaces the 36 fibres to the PIC: 32 fibres are for data and four for the external laser light source.
There is only a physical limit to the number of fibres that can be connected if edge coupling is used, says Winzer. But surface coupling in a 2D array means the optical engine delivers 5-10x more density than its competitors.
The start-up also has designed the engine’s electronics: the optical modulator driver and the trans-impedance amplifier (TIA). The electronics use advanced equalisation to boost the electrical channel, given direct drive has demanding requirements, says Harding.
The XT1600 optical module
Nubis’ first product is the XT1600 optical module. Here, a substrate houses the company’s PIC and electronics onto which is packaged a lid containing the optical fibres.
Nubis has developed in-house the packaging and the fibre attach solution.
The substrate is 15x15mm, somewhat larger than the engine. Harding says this is deliberate to support products under development.
The 1.6 terabits – in fact, 16x112Gbps full duplex – module has a 2km reach. Its power consumption is below 4 pJ/b.
The fibres exit the module vertically and bend to the side. “[Going] vertical is good but the 2D is the much more important aspect here,” says Winzer.
A 2D approach is logical, says Nubis. An electrical ball grid array (BGA) all the bottom surface. It makes sense that the optics is similarly massively 2D, especially for designs where its a 100-gigabit electrical signal in and a 100-gigabit optical signal out.
Multiple rings of optical I/O engines can surround the ASIC because the fibres exit vertically. “Nobody else can do that because they are escaping from the [PIC] edge,” says Winzer.
Winzer highlights another benefit of the design.
The Universal Chiplet Interconnect Express (UCIe) specification calls for 2T/mm bandwidth escape density. An optical chiplet can only achieve this if wavelength-division multiplexing (WDM) is used due to the large fibre size. Nubis can achieve this density optically without having to use WDM because of 2D surface coupling.
Doing all-to-all at scale remains a big system challenge. “We’re just a part of that challenge,” says Harding. But for optical I/O to become pervasive in the data centre over the next five years, the optics must be significantly lower power, smaller, and efficient.
“If you crack that 2D nut, you can do many, many great things down the road,” says Winzer. “We’ve solved a huge technology problem that allows us to scale much better than anybody else.”
Status
Nubis has not named its foundry and contract manufacturing partners but says they are large, high-volume manufacturers.
Harding says there are now up to five credible silicon photonic foundries available.
“There was some early product definition which some foundries were better suited to support,“ says Harding. “And there was a robustness of the initial PDKs [process design kits] to get us an early product that was important to us.”
Choosing a contract manufacturer proved easier, given the maturity of the players.
Nubis’ first product has 16 optical channels each at 112 gigabit, but future designs will offer N by 224-gigabit channels.
Meanwhile, the XT1600 optical engine is available for sampling.
ON2020 rallies industry to address networking concerns
Source: ON2020
The slide shows how router-blade client interfaces are scaling at 40% annually compared to the 20% growth rate of general single-wavelength interfaces (see chart).
Extrapolating the trend to 2024, router blades will support 20 terabits while client interfaces will only be at one terabit. Each blade will thus require 20 one-terabit Ethernet interfaces. “That is science fiction if you go off today’s technology,” says Winzer, director of optical transmission subsystems research at Nokia Bell Labs and a member of the ON2020 steering committee.
This is where ON2020 comes in, he says, to flag up such disparities and focus industry efforts so they are addressed.
ON2020
Established in 2016, the companies driving ON2020 are Fujitsu, Huawei, Nokia, Finisar, and Lumentum.
The reference to 2020 signifies how the group looks ahead four to five years, while the name is also a play on 20/20 vision, says Brandon Collings, CTO of Lumentum and also a member of the steering committee.
Brandon CollingsON2020 addresses a void in the industry, says Collings. The Optical Internetworking Forum (OIF) organisation may have a similar conceptual mission but it is more hands-on, focussing on components and near-term implementations. ON2020 looks further out.
“Maybe you could argue it is a two-step process,” says Collings. “First, ON2020 is longer term followed by the OIF’s definition in the near term.”
To build a longer-term view, ON2020 surveyed network operators worldwide including the largest internet content providers players and leading communications service providers.
ON2020 reported its findings at the recent ECOC show under three broad headings: traffic growth and the impact on fibre capacity and interfaces, interconnect requirements, and network management and operations.
Things will have to get cheaper; that is the way things are.
Network management
One key survey finding is the importance network operators attach to software-defined networking (SDN) although the operators are frustrated with the lack of SDN solutions available, forcing them to work with vendors to address their needs.
Peter WinzerThe network operators also see value in white boxes and disaggregation, to lower hardware costs and avoid vendor lock-in. But as with SDN, there are challenges with white boxes and disaggregation.
“Let’s not forget that SDN comes from the big webscales,” says Winzer, companies with abundant software and control experience. Telecom companies don’t have such sophisticated resources.
“This produces a big conundrum for the telecom operators: they want to get the benefits without spending what the webscales are spending,” says Winzer. The telcos also need higher network reliability such that their job is even harder.
Responding to ON2020’s anonymous survey, the telecom players stress how SDN, disaggregation and the adoption of white boxes will require a change in practices and internal organisation and even the employment of system integrators.
“They are really honest. They say, nice, but we are just overwhelmed,” says Winzer. “It highlights the very important organisational challenges operators are facing.”
Operators are frustrated with the lack of SDN solutions available.
Capacity and connectivity
The webscales and telecom operators were also surveyed about capacity and connectivity issues.
Both classes of operator use 10-terabit links or more and this will soon rise to 40 terabits. The consensus is that the C-band alone is insufficient given their capacity needs.
Those operators with limited fibre want to grow capacity by also using the L-band with the C-band, while operators with plenty of fibre want to combine fibre pairs - a form of spatial division multiplexing - and using the C and L bands. The implication here is that there is an opportunity for hardware integration, says ON2020.
Network operators use backbone wavelengths at 100, 200 and 400 gigabits. As for service feeds - what ON2020 refers to as granularity - webscale players favour 25 gigabit-per-second (Gbps) whereas telecom operators continue to deal with much slower feeds - 10Mbps, 100Mbps, and 1Gbps.
What can ON2020 do to address the demanding client-interface requirements of IP router blades, referred to in the chart?
Xiang Liu, distinguished scientist, transmission product line at Huawei and a key instigator in the creation of ON2020, says photonic integration and a tighter coupling between photonics and CMOS will be essential to reduce the cost-per-bit and power-per-bit of future client interfaces.
Xiang Liu
“As the investment for developing routers with such throughputs could be unprecedentedly high, it makes sense for our industry to collectively define the specifications and interfaces,” says Liu. “ON2020 can facilitate such an industry-wide effort.”
Another survey finding is that network operators favour super-channels once client interfaces reach 400 gigabits and higher rates. Super-channels are more efficient in their use of the fibre’s spectrum while also delivering operations, administration, and management (OAM) benefits.
The network operators were also asked about their node connectivity needs. While they welcome the features of advanced reconfigurable optical add-drop multiplexers (ROADMs), they don’t necessarily need them all. A typical response being they will adopt such features if they are practically for free.
This, says Winzer, is typical of carriers. “Things will have to get cheaper; that is the way things are.”
Photonic integration and a tighter coupling between photonics and CMOS will be essential to reduce the cost-per-bit and power-per-bit of future client interfaces
Future plans
ON2020 is still seeking feedback from additional network operators, the survey questionnaire being availability for download on its website. “The more anonymous input we get, the better the results will be,” says Winzer.
Huawei’s Liu says the published findings are just the start of the group’s activities.
ON2020 will conduct in-depth studies on such topics as next-generation ROADM and optical cross-connects; transport SDN for resource optimisation and multi-vendor interoperability; 5G-oriented optical networking that delivers low latency, accurate synchronisation and network slicing; new wavelength-division multiplexing line rates beyond 200 gigabit; and optical link technologies beyond just the C-band and new fibre types.
ON2020 will publish a series of white papers to stimulate and guide the industry, says Liu.
The group also plans to provide input to standardisation organisations to enhance existing standards and start new ones, create proof-of-concept technology demonstrators, and enable multi-vendor interoperable tests and field trials.
Discussions have started for ON2020 to become an IEEE Industry Connections programme. “We don’t want this to be an exclusive club of five [companies],” says Winzer. “We want broad participation.”
SDM and MIMO: An interview with Bell Labs
Part 2: The capacity crunch and the role of SDM
The argument for spatial-division multiplexing (SDM) - the sending of optical signals down parallel fibre paths, whether multiple modes, cores or fibres - is the coming ‘capacity crunch’. The information-carrying capacity limit of fibre, for so long described as limitless, is being approached due to the continual yearly high growth in IP traffic. But if there is a looming capacity crunch, why are we not hearing about it from the world’s leading telcos?
“It depends on who you talk to,” says Peter Winzer, head of the optical transmission systems and networks research department at Bell Labs. The incumbent telcos have relatively low traffic growth - 20 to 30 percent annually. “I believe fully that it is not a problem for them - they have plenty of fibre and very low growth rates,” he says.
Twenty to 30 percent growth rates can only be described as ‘very low’ when you consider that cable operators are experiencing 60 percent year-on-year traffic growth while it is 80 to 100 percent for the web-scale players. “The whole industry is going through a tremendous shift right now,” says Winzer.
In a recent paper, Winzer and colleague Roland Ryf extrapolate wavelength-division multiplexing (WDM) trends, starting with 100-gigabit interfaces that were adopted in 2010. Assuming an annual traffic growth rate of 40 to 60 percent, 400-gigabit interfaces become required in 2013 to 2014, and the authors point out that 400-gigabit transponder deployments started in 2013. Terabit transponders are forecast in 2016 to 2017 while 10 terabit commercial interfaces are expected from 2020 to 2024.
In turn, while WDM system capacities have scaled a hundredfold since the late 1990s, this will not continue. That is because systems are approaching the Non-linear Shannon Limit which estimates the upper limit capacity of fibre at 75 terabit-per-second.
Starting with 10-terabit-capacity systems in 2010 and a 30 to 40 percent core network traffic annual growth rate, the authors forecast that 40 terabit systems will be required shortly. By 2021, 200 terabit systems will be needed - already exceeding one fibre’s capacity - while petabit-capacity systems will be required by 2028.
Even if I’m off by an order or magnitude, and it is 1000, 100-gigabit lines leaving the data centre; there is no way you can do that with a single WDM system
Parallel spatial paths are the only physical multiplexing dimension remaining to expand capacity, argue the authors, explaining Bell Labs’ interest in spatial-division multiplexing for optical networks.
If the telcos do not require SDM-based systems anytime soon, that is not the case for the web-scale data centre operators. They could deploy SDM as soon as 2018 to 2020, says Winzer.
The web-scale players are talking about 400,000-server data centres in the coming three to five years. “Each server will have a 25-gigabit network interface card and if you assume 10 percent of the traffic leaves the data centre, that is 10,000, 100-gigabit lines,” says Winzer. “Even if I’m off by an order or magnitude, and it is 1000, 100-gigabit lines leaving the data centre; there is no way you can do that with a single WDM system.”
SDM and MIMO
SDM can be implemented in several ways. The simplest way to create parallel transmission paths is to bundle several single-mode fibres in a cable. But speciality fibre can also be used, either multi-core or multi-mode.
For the demo, Bell Labs used such a fibre, a coupled 3-core one, but Sebastian Randel, a member of technical staff, said its SDM receiver could also be used with a fibre supporting a few spatial modes. By increasing slightly the diameter of a single-mode fibre, not only is a single mode supported but two second-order modes. “Our signal processing would cope with that fibre as well,” says Winzer.
The signal processing referred to, that restores the multiple transmissions at the receiver, implements multiple input, multiple output or MIMO. MIMO is a well-known signal processing technique used for wireless and digital subscriber line (DSL).
They are garbled up, that is what the rotation is; undoing the rotation is called MIMO
Multi-mode fibre can support as many as 100 spatial modes. “But then you have a really big challenge to excite all 100 spatial modes individually and detect them individually,” says Randel. In turn, the digital signal processing computation required for the 100 modes is tremendous. “We can’t imagine we can get there anytime soon,” says Randel.
Instead, Bell Labs used 60 km of the 3-core coupled fibre for its real-time SDM demo. The transmission distance could have been much longer except the fibre sample was 60 km long. Bell Labs chose the coupled-core fibre for the real-time MIMO demonstration as it is the most demanding case, says Winzer.
The demonstration can be viewed as an extension of coherent detection used for long-distance 100 gigabit optical transmission. In a polarisation-multiplexed, quadrature phase-shift keying (PM-QPSK) system, coupling occurs between the two light polarisations. This is a 2x2 MIMO system, says Winzer, comprising two inputs and two outputs.
For PM-QPSK, one signal is sent on the x-polarisation and the other on the y-polarisation. The signals travel at different speeds while hugely coupling along the fibre, says Winzer: “The coherent receiver with the 2x2 MIMO processing is able to undo that coupling and undo the different speeds because you selectively excite them with unique signals.” This allows both polarisations to be recovered.
With the 3-core coupled fibre, strong coupling arises between the three signals and their individual two polarisations, resulting in a 6x6 MIMO system (six inputs and six outputs). The transmission rotates the six signals arbitrarily while the receiver, using 6x6 MIMO, rotates them back. “They are garbled up, that is what the rotation is; undoing the rotation is called MIMO.”
Demo details
For the demo, Bell Labs generated 12, 2.5-gigabit signals. These signals are modulated onto an optical carrier at 1550nm using three nested lithium niobate modulators. A ‘photonic lantern’ - an SDM multiplexer - couples the three signals orthogonally into the fibre’s three cores.
The photonic lantern comprises three single-mode fibre inputs fed by the three single-mode PM-QPSK transmitters while its output places the fibres closer and closer until the signals overlap. “The lantern combines the fibres to create three tiny spots that couple into a single fibre, either single mode or multi-mode,” says Winzer.
At the receiver, another photonic lantern demultiplexes the three signals which are detected using three integrated coherent receivers.
Don’t do MIMO for MIMO’s sake, do MIMO when it helps to bring the overall integrated system cost down
To implement the MIMO, Bell Labs built a 28-layer printed circuit board which connects the three integrated coherent receiver outputs to 12, 5-gigabit-per-second 10-bit analogue-to-digital converters. The result is an 600 gigabit-per-second aggregate output digital data stream. This huge data stream is fed to a Xilinx Virtex-7 XC7V2000T FPGA using 480 parallel lanes, each at 1.25 gigabit-per-second. It is the FPGA that implements the 6x6 MIMO algorithm in real time.
“Computational complexity is certainly one big limitation and that is why we have chosen a relatively low symbol rate - 2.5 Gbaud, ten times less than commercial systems,” says Randel. “But this helps us fit the [MIMO] equaliser into a single FPGA.”
Future work
With the growth in IP traffic, optical engineers are going to have to use space and wavelengths. “But how are you going to slice the pie?” says Winzer.
With the example of 10,000, 100-gigabit wavelengths, will 100 WDM channels be sent over 100 spatial paths or 10 WDM channels over 1,000 spatial paths? “That is a techno-economic design optimisation,” says Winzer. “In those systems, to get the cost-per-bit down, you need integration.”
That is what the Bell Lab’s engineers are working on: optical integration to reduce the overall spatial-division multiplexing system cost. “Integration will happen first across the transponders and amplifiers; fibre will come last,” says Winzer.
Winzer stresses that MIMO-SDM is not primarily about fibre, a point frequently misunderstood. The point is to enable systems with crosstalk, he says.
“So if some modulator manufacturer can build arrays with crosstalk and sell the modulator at half the price they were able to before, then we have done our job,” says Winzer. “Don’t do MIMO for MIMO’s sake, do MIMO when it helps to bring the overall integrated system cost down.”
Further Information:
Space-division Multiplexing: The Future of Fibre-Optics Communications, click here
For Part 1, click here
Heading off the capacity crunch
Improving optical transmission capacity to keep pace with the growth in IP traffic is getting trickier.
Engineers are being taxed in the design decisions they must make to support a growing list of speeds and data modulation schemes. There is also a fissure emerging in the equipment and components needed to address the diverging needs of long-haul and metro networks. As a result, far greater flexibility is needed, with designers looking to elastic or flexible optical networking where data rates and reach can be adapted as required.
Figure 1: The green line is the non-linear Shannon limit, above which transmission is not possible. The chart shows how more bits can be sent in a 50 GHz channel as the optical signal to noise ratio (OSNR) is increased. The blue dots closest to the green line represent the performance of the WaveLogic 3, Ciena's latest DSP-ASIC family. Source: Ciena.
But perhaps the biggest challenge is only just looming. Because optical networking engineers have been so successful in squeezing information down a fibre, their scope to send additional data in future is diminishing. Simply put, it is becoming harder to put more information on the fibre as the Shannon limit, as defined by information theory, is approached.
"Our [lab] experiments are within a factor of two of the non-linear Shannon limit, while our products are within a factor of three to six of the Shannon limit," says Peter Winzer, head of the optical transmission systems and networks research department at Bell Laboratories, Alcatel-Lucent. The non-linear Shannon limit dictates how much information can be sent across a wavelength-division multiplexing (WDM) channel as a function of the optical signal-to-noise ratio.
A factor of two may sound a lot, says Winzer, but it is not. "To exhaust that last factor of two, a lot of imperfections need to be compensated and the ASIC needs to become a lot more complex," he says. The ASIC is the digital signal processor (DSP), used for pulse shaping at the transmitter and coherent detection at the receiver.
Our [lab] experiments are within a factor of two of the non-linear Shannon limit, while our products are within a factor of three to six of the Shannon limit - Peter Winzer
At the recent OFC 2015 conference and exhibition, there was plenty of announcements pointing to industry progress. Several companies announced 100 Gigabit coherent optics in the pluggable, compact CFP2 form factor, while Acacia detailed a flexible-rate 5x7 inch MSA capable of 200, 300 and 400 Gigabit rates. And research results were reported on the topics of elastic optical networking and spatial division multiplexing, work designed to ensure that networking capacity continues to scale.
Trade-offs
There are several performance issues that engineers must consider when designing optical transmission systems. Clearly, for submarine systems, maximising reach and the traffic carried by a fibre are key. For metro, more data can be carried on a single carrier to improving overall capacity but at the expense of reach.
Such varied requirements are met using several design levers:
- Baud or symbol rate
- The modulation scheme which determines the number of bits carried by each symbol
- Multiple carriers, if needed, to carry the overall service as a super-channel
The baud rate used is dictated by the performance limits of the electronics. Today that is 32 Gbaud: 25 Gbaud for the data payload and up to 7 Gbaud for forward error correction and other overhead bits.
Doubling the symbol rate from 32 Gbaud used for 100 Gigabit coherent to 64 Gbaud is a significant challenge for the component makers. The speed hike requires a performance overhaul of the electronics and the optics: the analogue-to-digital and digital-to-analogue converters and the drivers through to the modulators and photo-detectors.
"Increasing the baud rate gives more interface speed for the transponder," says Winzer. But the overall fibre capacity stays the same, as the signal spectrum doubles with a doubling in symbol rate.
However, increasing the symbol rate brings cost and size benefits. "You get more bits through, and so you are sharing the cost of the electronics across more bits," says Kim Roberts, senior manager, optical signal processing at Ciena. It also implies a denser platform by doubling the speed per line card slot.
As you try to encode more bits in a constellation, so your noise tolerance goes down - Kim Roberts
Modulation schemes
The modulation used determines the number of bits encoded on each symbol. Optical networking equipment already use binary phase-shift keying (BPSK or 2-quadrature amplitude modulation, 2-QAM) for the most demanding, longest-reach submarine spans; the workhorse quadrature phase-shift keying (QPSK or 4-QAM) for 100 Gigabit-per-second (Gbps) transmission, and the 200 Gbps 16-QAM for distances up to 1,000 km.
Moving to a higher QAM scheme increases WDM capacity but at the expense of reach. That is because as more bits are encoded on a symbol, the separation between them is smaller. "As you try to encode more bits in a constellation, so your noise tolerance goes down," says Roberts.
One recent development among system vendors has been to add more modulation schemes to enrich the transmission options available.
From QPSK to 16-QAM, you get a factor of two increase in capacity but your reach decreases of the order of 80 percent - Steve Grubb
Besides BPSK, QPSK and 16-QAM, vendors are adding 8-QAM, an intermediate scheme between QPSK and 16-QAM. These include Acacia with its AC-400 MSA, Coriant, and Infinera. Infinera has tested 8-QAM as well as 3-QAM, a scheme between BPSK and QPSK, as part of submarine trials with Telstra.
"From QPSK to 16-QAM, you get a factor of two increase in capacity but your reach decreases of the order of 80 percent," says Steve Grubb, an Infinera Fellow. Using 8-QAM boosts capacity by half compared to QPSK, while delivering more signal margin than 16-QAM. Having the option to use the intermediate formats of 3-QAM and 8-QAM enriches the capacity tradeoff options available between two fixed end-points, says Grubb.
Ciena has added two chips to its WaveLogic 3 DSP-ASIC family of devices: the WaveLogic 3 Extreme and the WaveLogic 3 Nano for metro.
WaveLogic3 Extreme uses a proprietary modulation format that Ciena calls 8D-2QAM, a tweak on BPSK that uses longer duration signalling that enhances span distances by up to 20 percent. The 8D-2QAM is aimed at legacy dispersion-compensated fibre that carry 10 Gbps wavelengths and offers up to 40 percent additional upgrade capacity compared to BPSK.
Ciena has also added 4-amplitude-shift-keying (4-ASK) modulation alongside QPSK to its WaveLogic3 Nano chip. The 4-ASK scheme is also designed for use alongside 10 Gbps wavelengths that introduce phase noise, to which 4-ASK has greater tolerance than QPSK. Ciena's 4-ASK design also generates less heat and is less costly than BPSK.
According to Roberts, a designer’s goal is to use the fastest symbol rate possible, and then add the richest constellation as possible "to carry as many bits as you can, given the noise and distance you can go".
After that, the remaining issue is whether a carrier’s service can be fitted on one carrier or whether several carriers are needed, forming a super-channel. Packing a super-channel's carriers tightly benefits overall fibre spectrum usage and reduces the spectrum wasted for guard bands needed when a signal is optically switched.
Can symbol rate be doubled to 64 Gbaud? "It looks impossibly hard but people are going to solve that," says Roberts. It is also possible to use a hybrid approach where symbol rate and modulation schemes are used. The table shows how different baud rate/ modulation schemes can be used to achieve a 400 Gigabit single-carrier signal.
Note how using polarisation for coherent transmission doubles the overall data rate. Source: Gazettabyte
But industry views differ as to how much scope there is to improve overall capacity of a fibre and the optical performance.
Roberts stresses that his job is to develop commercial systems rather than conduct lab 'hero' experiments. Such systems need to be work in networks for 15 years and must be cost competitive. "It is not over yet," says Roberts.
He says we are still some way off from when all that remains are minor design tweaks only. "I don't have fun changing the colour of the paint or reducing the cost of the washers by 10 cents,” he says. “And I am having a lot of fun with the next-generation design [being developed by Ciena].”
"We are nearing the point of diminishing returns in terms of spectrum efficiency, and the same is true with DSP-ASIC development," says Winzer. Work will continue to develop higher speeds per wavelength, to increase capacity per fibre, and to achieve higher densities and lower costs. In parallel, work continues in software and networking architectures. For example, flexible multi-rate transponders used for elastic optical networking, and software-defined networking that will be able to adapt the optical layer.
After that, designers are looking at using more amplification bands, such as the L-band and S-band alongside the current C-band to increase fibre capacity. But it will be a challenge to match the optical performance of the C-band across all bands used.
"I would believe in a doubling or maybe a tripling of bandwidth but absolutely not more than that," says Winzer. "This is a stop-gap solution that allows me to get to the next level without running into desperation."
The designers' 'next level' is spatial division multiplexing. Here, signals are launched down multiple channels, such as multiple fibres, multi-mode fibre and multi-core fibre. "That is what people will have to do on a five-year to 10-year horizon," concludes Winzer.
For Part 2, click here
See also:
- Scaling Optical Fiber Networks: Challenges and Solutions by Peter Winzer
- High Capacity Transport - 100G and Beyond, Journal of Lightwave Technology, Vol 33, No. 3, February 2015.
A version of this article first appeared in an OFC 2015 show preview
Space-division multiplexing: the final frontier
System vendors continue to trumpet their achievements in long-haul optical transmission speeds and overall data carried over fibre.
Alcatel-Lucent announced earlier this month that France Telecom-Orange is using the industry's first 400 Gigabit link, connecting Paris and Lyon, while Infinera has detailed a trial demonstrating 8 Terabit-per-second (Tbps) of capacity over 1,175km and using 500 Gigabit-per-second (Gbps) super-channels.
"Integration always comes at the cost of crosstalk"
Peter Winzer, Bell Labs
Yet vendors already recognise that capacity in the frequency domain will only scale so far and that other approaches are required. One is space-division multiplexing such as using multiple channels separated in space and implemented using multi-core fibre with each core supporting several modes.
"We want a technology that scales by a factor of 10 to 100," says Peter Winzer, director of optical transmission systems and networks research at Bell Labs. "As an example, a fibre with 10 cores with each core supporting 10 modes, then you have the factor of 100."
Space-division multiplexing
Alcatel-Lucent's research arm, Bell Labs, has demonstrated the transmission of 3.8Tbps using several data channels and an advanced signal processing technique known as multiple-input, multiple-output (MIMO).
In particular, 40 Gigabit quadrature phase-shift keying (QPSK) signals were sent over a six-spatial mode fibre using two polarisation modes and eight wavelengths to achieve 3.8Tbps. The overall transmission uses 400GHz of spectrum only.
Alcatel-Lucent stresses that the commercial deployment of space-division multiplexing remains years off. Moreover operators will likely first use already-deployed parallel strands of single-mode fibre, needing the advanced signal processing techniques only later.
"You might say that is trivial [using parallel strands of fibre], but bringing down the cost of that solution is not," says Winzer.
First, cost-effective integrated amplifiers will be needed. "We need to work on a single amplifier that can amplify, say, ten existing strands of single-mode fibre at the cost of two single-mode amplifiers," says Winzer. An integrated transponder will also be needed: one transponder that couples to 10 individual fibres at a much lower cost than 10 individual transponders.
With a super-channel transponder, several wavelengths are used, each with its own laser, modulator and detector. "In a spatial super-channel you have the same thing, but not, say, three different frequencies but three different spatial paths," says Winzer. Here photonic integration is the challenge to achieve a cost-effective transponder.
Once such integrated transponders and amplifiers become available, it will make sense to couple them to multi-core fibre. But operators will only likely start deploying new fibre once they exhaust their parallel strands of single-mode fibre.
Such integrated amplifiers and integrated transponders will present challenges. "The more and more you integrate, the more and more crosstalk you will have," says Winzer. "That is fundamental: integration always comes at the cost of crosstalk."
Winzer says there are several areas where crosstalk may arise. An integrated amplifier serving ten single-mode fibres will share a multi-core erbium-doped fibre instead of ten individual strands. Crosstalk between those closely-spaced cores is likely.
The transponder will be based on a large integrated circuit giving rise to electrical crosstalk. One way to tackle crosstalk is to develop components to a higher specification but that is more costly. Alternatively, signal processing on the received signal can be used to undo the crosstalk. Using electronics to counter crosstalk is attractive especially when it is the optics that dominate the design cost. This is where MIMO signal processing plays a role. "MIMO is the most advanced version of spatial multiplexing," says Winzer.
To address crosstalk caused by spatial multiplexing in the Bell Labs' demo, 12x12 MIMO was used. Bell Labs says that using MIMO does not add significantly to the overall computation. Existing 100 Gigabit coherent ASICs effectively use a 2x2 MIMO scheme, says Winzer: “We are extending the 2x2 MIMO to 2Nx2N MIMO.”
Only one portion of the current signal processing chain is impacted, he adds; a portion that consumes 10 percent of the power will need to increase by a certain factor. The resulting design will be more complex and expensive but not dramatically so, he says.
Winzer says such mitigation techniques need to be investigated now since crosstalk in future systems is inevitable. Even if the technology's deployment is at least a decade away, developing techniques to tackle crosstalk now means vendors have a clear path forward.
Parallelism
Winzer points out that optical transmission continues to embrace parallelism. "With super-channels we go parallel with multiple carriers because a single carrier can’t handle the traffic anymore," he says. This is similar to parallelism in microprocessors where multi-core designs are now used due to the diminishing return in continually increasing a single core's clock speed.
For 400Gbps or 1 Terabit over a single-mode fibre, the super-channel approach is the near term evolution.
Over the next decade, the benefit of frequency parallelism will diminish since it will no longer increase spectral efficiency. "Then you need to resort to another physical dimension for parallelism and that would be space," says Winzer.
MIMO will be needed when crosstalk arises and that will occur with multiple mode fibre.
"For multiple strands of single mode fibre it will depend on how much crosstalk the integrated optical amplifiers and transponders introduce," says Winzer.
Part 1: Terabit optical transmission