Ciena becomes a computer weaver
- Ciena is to buy optical interconnect start-up Nubis Communications for $270 million.
- The deal covers optical and copper interconnect technology for data centres
Ciena has announced its intention to buy optical engine specialist Nubis Communications for $270 million. If the network is the computer, Nubis’ optical engine and copper integrated circuit (IC) expertise will help Ciena better stitch together AI’s massive compute fabric.
Ciena signalled its intention to target the data centre earlier this year at the OFC show when it showcased its high-speed 448-gigabit serialiser-deserialiser IC technology and coherent lite modem. Now, Ciena has made a move for start-up Nubis, which plays at the core of AI data centres.
“Ciena’s expertise in high-speed components is relevant to 400G per lane Ethernet transceivers, but they never sold any products to this market,” says Vladimir Kozlov, CEO of LightCounting. “Nubis offers them an entry point with several designs and customer engagements.”
With the deal, Ciena is extending its traditional markets of wide area networks (WAN), metro, and short-reach dense wavelength division multiplexing (DWDM) to include AI networking opportunities. These opportunities include scale-across networks, where AI workloads are shared across multiple data centres, something Ciena can address, to now scale-out and scale-up networks for AI clusters in the data centre.
Puma optical engine
Nubis has developed two generations of compact optical engines for near-package optics (NPO) and co-package optics (CPO) applications. Its first-generation engine operates at 100 gigabits per second (Gbps), while its second, dubbed Puma, operates at 200 Gbps.
Nubis’s optical engine philosophy is based on escaping the optical channels from the surface of the optical engine, not its edge. The start-up also matches the number of optical channels to the electrical ones. The optical engine can be viewed as a sieve: data from the input channels flow through the chip and emerge in the same number of channels at the output. The engine acts as a two-way gateway, with one side handling electrical signals and the other, optical ones.
The Puma optical engine uses 16 channels in each direction, 16 by 200Gbps electrical signals for a total of 3.2 terabits per second (Tbps), and 16 fibres, each fibre carrying 200Gbps of data in the form of a wavelength. Puma’s total capacity is thus 6.4 terabits per second (Tbps). The engine also needs four external lasers to drive the optics, each laser feeding four channels or four fibres. The total fibre bundle of the device consists of 36 fibres: 32 for data (16 for receive and 16 for transmit), and four for the laser light sources.
Nubis is also a proponent of linear drive technology. Here, the advanced serdes on the adjacent semiconductor chip drives the optical engine, thereby avoiding the need for an on-engine digital signal processor (DSP) that requires power. The start-up has also developed a system-based simulator software tool that it uses to model the channel, from the transmitter to the receiver. The tool models not only the electrical and optical components within the channel but also the endpoints, such as the serdes.
Nitro
Nubis has an analogue IC team that designs its trans-impedance amplifiers (TIAs) and drivers used for the optical engine. The hardware compensates for channel impairments with low noise, high linearity, and at high speed. It is this channel simulator tool that Nubis used to optimise its optical engine, and to develop its second key technology, which Nubis calls Nitro —a chip that extends the reach of copper cabling.
“We use our linear optics learning and apply it to copper straight out of the gate, “said Peter Winzer, founder & CTO at Nubis, earlier this year. By using its end-to-end simulator tool, Nubis developed the Nitro IC, which extends the 1m reach of direct-attached copper to 4m using an active copper cable design. “We don’t optimise the driver chip, we optimise the end-to-end system,” says Winzer.
Nubis was also part of a novel design based on a vertical line card to shorten the trace length between an ASIC and pluggable modules.
Ciena’s gain
The acquisition of Nubis places Ciena at the heart of the electrical-optical transition inside the data centre. Ciena will cover both options: copper and optical interconnect. Ciena will gain direct-drive technology expertise for electrical and optical interfaces, enabling scale-up, as well as optical engine technology for scale-out, adding to its coherent technology expertise.
Ciena’s technologies will span coherent ultra-long-haul links all the way to AI accelerators, the heart of AI clusters. By combining Ciena’s 448-gigabit serdes with Nubis’s optical engine expertise, Ciena has a roadmap to develop 12.8Tbps and faster optical engines.
The acquisition places Ciena among new competitors that have chip and optical expertise and deliver co-packaged optics solutions alongside complex ICs such as Broadcom and Marvell.
The deal adds differentiation from Ciena’s traditional system vendor competitors, such as Cisco/ Acacia and Nokia. Huawei is active in long-haul optical and makes AI clusters. Ciena will also compete with existing high-speed optical players, including co-packaged optics specialists Ayar Labs and Ranovus, microLED player Avicena, and optical/IC fabric companies such as LightMatter and Celestial AI.
“Ciena will be a unique supplier in the co-packaged optics/near-packaged optics/active copper cabling data centre interconnect market,” says Daryl Inniss, Omdia’s thought lead of optical components and advanced fibre. “The other suppliers either have multiple products in the intra data centre market, like Broadcom and Nvidia, or they are interconnect-focused start-ups. These suppliers should all wonder what Ciena will do next inside the data centre.”
Ciena will enhance its overall expertise in chips, optics, and signal processing with the Nubis acquisition. It will also put Ciena in front of key processor players and different hyperscaler engineering teams, which drive next-generation AI systems.
Ciena will also have all the necessary parts for the various technologies, regardless of the evolving timescales associated with the copper-to-optical transition within AI systems. Ciena will add direct-detect technology and copper interconnect. On the optical side, it has coherent optical expertise, now coupled with near-package optics and co-packaged optics.
Nubis’ gain
Nubis’ 50-plus staff get a successful exit. The start-up was founded in 2020. Nubis will become a subsidiary of Ciena.
Nubis will be joining a much bigger corporate entity with deep expertise and pockets. Ciena has a good track record with its mergers. Think Nortel at the system level and Blue Planet, a software acquisition. Now the Nubis deal will bring Ciena firmly inside the data centre.
“This is a great deal for Nubis,” says Kozlov. “Congratulations to their team.”
What next?
The deal is expected to close in the fourth quarter of this year. Ciena expects the deal to start adding to its revenues from 2028, requiring Ciena and Nubis to develop products and deliver design wins in the data centre.
“Given the breadth of Ciena’s capabilities, its deep pockets, and products like its data centre out-of-band (DCOM) measurement product, router, and coherent transceivers, one can imagine that Ciena would offer more than co-packaged optics/ near-packaged optics/ active copper cabling inside the data centre,” says Inniss.
A coherent roadmap for co-packaged optics
Is coherent optics how co-packaged will continue to scale? Pilot Photonics certainly thinks so.
Part 1: Co-packaged optics
Frank Smyth, CTO and founder of Pilot Photonics, believes the firm is at an important inflection point.
Known for its comb laser technology, Pilot Photonics has just been awarded a €2.5 million European Innovation Council grant to develop its light-source technology for co-packaged optics.
The Irish start-up is also moving to much larger premises and is on a recruitment drive. “Many of our projects and technologies are maturing,” says Smyth.
Company
Founded in 2011, the start-up spent its early years coupled to Dublin City University. It raised its first notable investment in 2017.
The company began by making lab instrumentation based on its optical comb laser technology which emits multiple sources of light that are frequency- and phased-locked. But a limited market caused the company to pivot, adding photonic integration to its laser know-how.
Now, the start-up has a fast-switching, narrow-linewidth tunable laser, early samples of which are being evaluated by several “tier-one” companies.
Pilot Photonics also has a narrowband indium-phosphide comb laser for optical transport applications. This will be the next product it samples.
More recently, the start-up has been developing a silicon nitride-based comb laser for a European Space Agency project. “The silicon nitride micro-resonator in the comb is a non-linear element that enables a very broad comb for highly parallel communication systems and for scientific applications,” says Smyth. It is this laser type that is earmarked for the data centre and for co-packaged optics applications.
Smyth stresses that while still being a small company, the staff has broad expertise. “We cover the full stack,” he says.
Skills range from epitaxial wafer design, photonic integrated circuit (PIC)s and lasers, radio frequency (RF) and thermal expertise, and digital electronics and control design capabilities.
“We learned early on that it’s all well making a PIC, but if no one can interface to it, you are wasting your time,” says Smyth.
Co-packaged optics
Co-packaged optics refers to adding optics next to an ASIC that has significant input-output (I/O) data requirements. Examples of applications for co-packaged optics include high-capacity Ethernet switch chips and artificial intelligence (AI) accelerators. The goal is to give the chip optical rather than electrical interfaces, providing system-scaling benefits; as electrical signals get faster, their reach shrink.
The industry has been discussing co-packaged optics for over a decade. Switch-chip players and systems vendors have shown prototype designs and even products. And more than half a dozen companies are developing the optical engines that surround, and are packaged with, the chip.
However, the solutions remain proprietary, and while the OIF is working to standardise co-packaged optics, end users have yet to embrace the technology. In part, this is because pluggable optical modules continue to advance in data speeds and power consumption, with developments like linear-drive optics.
The ecosystem supporting co-packaged optics is also developing. Hyperscalers will only deploy co-packaged optics in volume when reliability and a broad manufacturing base are proven.
Yet industry consensus remains that optical I/O is a critical technology and that deployments will ramp up in the next two years. Ethernet switch capacity doubles every two years while AI accelerator chips are progressing rapidly. Moreover, the number of accelerator chips used in AI supercomputers is growing fast, from thousands to tens of thousands.
Pilot Photonics believes its multi-wavelength laser technology, coupled with the intellectual property it is developing, will enable co-packaged optics based on coherent optics to address such scaling issues.
Implementations
Co-packaged optics uses optical chiplets or ‘engines’ that surround the ASIC on a shared substrate. The optical engines typically use an external laser source although certain co-packaged optics solutions such as from Intel and Ranovus can integrate the laser as part of the silicon-photonics based optical engine.
Designers can scale the optical engine’s I/O capacity in several ways. They can increase the number of fibres connected to the optical engine, send more wavelengths down each fibre, and increase the wavelength’s data rate measured in gigabits per second (Gbps).
In co-packaged optics designs, 16 engines typically surround the chip. For a 25.6-terabit Ethernet chip, 16 x 1.6-terabit engines are used, each 1.6-terabit engine sending a 100Gbps DR1 signal per fibre. The total fibres per engine equals 32: 16 for the transmit and 16 for the receive (see table).
| Switch capacity/Tbps | Optical engine/Tbps | Optical engines | Data rate/fibre | No. fibres/ engine* |
| 25.6 | 1.6 | 16 | 100G DR, 500m | 32 |
| 25.6 | 3.2 | 8 | 100G DR, 500m | 64 |
| 51.2 | 6.4 | 8 | 400G FR4, 2km | 32 |
| 102.4 (speculative) | 6.4 | 16 | 400G FR4, 2km | 16 |
| 102.4 (speculation) | 12.8 | 8 | 400G FR4, 2km | 32 |
*Not counting the external laser source fibre.
Broadcom’s co-packaged optical approach uses eight optical engines around its 25.6-terabit Tomahawk 4 switch chip, each with 3.2Tbps capacity. For the Tomahawk 5, 51.2-terabit Bailly co-packaged optics design, Broadcom uses eight, 6.4Tbps optical engines, sending 400-gigabit FR4, or 4-wavelength coarse WDM wavelengths, across each fibre. Using FR4 instead of DR1 halves the number of optical engines while doubling overall capacity.
The co-packaging solutions used in the next-generation 102.4-terabit switch chip are still to be determined. Capacity could be doubled using twice as many fibres, or by using 200-gigabit optical wavelengths based on 112G PAM-4 electrical inputs, twice the speed currently used.
But scaling routes for the generation after that – 204.8-terabit switch chips and beyond – and the co-packaged optics design become unclear due to issues of dispersion and power constraints, says Smyth.
Scaling challenges
Assuming eight engines were used alongside the 200-terabit ASIC , each would need to be 25.6Tbps. The fibre count per engine could be doubled again or more wavelengths per fibre would be needed. One player, Nubis Communications, scales its engines and fibres in a 2D array over the top of the package, an approach suited to fibre-count growth.
Doubling the wavelength count is another option but adopting an 8-wavelength CWDM design with 20nm spacing means the wavelengths would cover 160nm of spectrum. Over a 2km reach, this is challenging due to problems with dispersion. Narrower channel spacings such as those used in the CW-WDM MSA (multi-source agreement) require temperature control to ensure the wavelengths stay put.
Keeping the symbol rate fixed but doubling the data rate is another option. But adopting the more complex PAM-8 modulation brings its own link challenges.
Another key issue is power. Current 51.2-terabit switches require 400mW of laser launch power (4 x 100mW lasers) per fibre and there are 128 transmit fibers per switch.
“Assuming a wall plug efficiency of 20 per cent, that is around 250W of power dissipation just for the lasers,” says Smyth. “Getting to 4Tbps per fibre appears possible using 16 wavelengths, but the total fiber launch power is 10 times higher, requiring 2.5kW of electrical power per switch just for the lasers.”
In contrast, single-polarisation coherent detection of 16-QAM signals through a typical path loss of 24dB could match that 4Tbps capacity with the original 250W of laser electrical power, he says.
The optimised total laser power improvement for coherent detection versus direct detection as a function of the additional losses in the signal path (the losses not also experienced by the local oscillator). Source: Pilot Photonics
Coherent detection is associated with a high-power digital signal processor (DSP). Are such chips feasible for such a power-sensitive application as co-packaged optics?
Coherent detection adds some DSP complexity, says Smyth, but it has been shown that for pluggable-based intra data centre links using 5nm CMOS silicon, 400-gigabit coherent and direct-detection are comparable in terms of ASIC power but coherent requires less laser power.
“Over time, a similar battle will play out for co-packaged optics. Laser power will become a bigger issue than DSP power,” he says.
The additional signal margin could be used for 10km links, with tens of terabits per fibre and even 80km links at similar per-fibre rates to current direct detection.
“We believe coherent detection in the data centre is inevitable,” says Smyth. “It’s just a question of when.”
Comb-based coherent co-packaged optics
Coherent co-packaged optics brings its own challenges. Coherent detection requires alignment between the signal wavelength and the local oscillator laser in the receiver. Manufacturing tolerances and the effects of ageing in simple laser arrays make this challenging to achieve.
“The wavelengths of a comb laser are precisely spaced, which greatly simplifies the problem,” says Smyth. “And combs bring other benefits related to carrier recovery and lack of inter-channel interference too”.
Pilot Photonics’ comb laser delivers 16 or 32 wavelengths per fibre, up to 8x more than existing solutions. Smyth says the company intends to fit its comb laser inside the OIF’s standardised External Laser Source pluggable form-factor,
The start-up is also developing a coherent ring resonator modulator for its design. The ring modulator is tiny compared with Mach-Zehnder interferometer modulators used for coherent optics.
Pilot Photonics is also developing IP for coherent signal processing. Because its comb laser locks the frequency and phase of the wavelengths generated, the overall control and signal processing can be simplified.
While it will offer the comb laser, the start-up does not intend to develop the DSP IC nor make optical engines itself.
“A strategic partnership with a company with its own manufacturing facilities would be the most effective way of getting this technology to market,” says Smyth.
Nubis' bandwidth-packed tiny optical engine
- Nubis Communications has revealed its ambitions to be an optical input-output (I/O) solutions provider
- Its tiny 1.6-terabit optical engine measures 5mm x 7.5mm
- The optical engine has a power consumption of below 4 picojoule/bit (pJ/b) and a bandwidth density of 0.5 terabits per millimetre.
- “Future systems will be I/O with an ASIC dangling off it.”
Nubis Communications has ended its period of secrecy to unveil an optical engine targeted at systems with demanding data input-output requirements.
The start-up claims its optical engine delivers unmatched bandwidth density measured in terabits per millimetre (T/mm) and power consumption performance metrics.
“In the timeframe of founding the company [in 2020], it became obvious that the solution space [for our product] was machine learning-artificial intelligence,” says Dan Harding, the CEO of Nubis.
Company Background
Nubis has raised over $40 million, with the lead investor being Matrix Partners. Venture capital company Matrix Partners backed Acacia Communications, acquired by Cisco in 2021.
Other Nubis backers are Weili Dai, a co-founder of Marvell Technologies, and Belgium-based imec.xpand.
“We have raised enough money to get to production with our product,” says Harding, who joined Nubis in 2021 from Broadcom.
Peter Winzer is the CTO and founder of the company. Formerly at Nokia Bell Labs, Winzer was the 2018 winner of the Optica (then OSA) and IEEE Photonics Society’s John Tyndall Award for his work on coherent optical communications.
Nubis has 40 staff, mostly engineers.
“As a team, we are multidisciplinary,” says Winzer. The company’s expertise includes silicon photonics, analogue IC design including serialisers/ deserialisers (serdes), packaging – electrical and optical, and software including advanced simulation tools.
“It is all geared towards a systems solution,” says Winzer. “We are not just looking at the PIC [photonic integrated circuit] or the electronics; we have the system and the architecture in mind.”
The input-output challenge
Machine learning workloads continue to grow at a staggering pace, doubling more than twice each year. Not surprisingly, computing systems running such workloads are struggling to keep up.
Scaling such systems not only requires more processing – more graphics processing units (GPUs) – but also networking to connect clusters of GPUs.
What the compute vendors want is any-to-any connectivity between processors and between clusters. This is creating a tremendous input-output challenge in terms of bandwidth density while keeping the power consumption under control.
“Over half the power of that cluster can be taken up by traditional optics,” says Harding. “So it is clear that the industry wants new solutions.”
“Whatever cents-per-gigabit [figure] you use, if you multiply it by the I/O capacity, the number you’ll get is many times that of [the cost of] an ASIC,” adds Winzer. “We say that future systems will be I/O with an ASIC dangling off it.”
Design details
Nubis’ optical engine is a 16 x 112-gigabit design with a footprint of 5mm x 7.5mm.
“Because we have our electronics flip-chipped on top, that’s the entire footprint,” says Winzer. “We maintain that it is the highest density by far of any optical engine.”
Nubis says many parallel fibres can be interfaced to the optical engine despite its tiny size.
Supporting parallel fibres is essential for machine learning systems as the fibres are fanned out to enable any-to-any connectivity.
Nubis’ engine uses a 4 by DR4 fan-out architecture with 36 fibres arranged in a 3×12 array.
Surface coupling in a 2D array interfaces the 36 fibres to the PIC: 32 fibres are for data and four for the external laser light source.
There is only a physical limit to the number of fibres that can be connected if edge coupling is used, says Winzer. But surface coupling in a 2D array means the optical engine delivers 5-10x more density than its competitors.
The start-up also has designed the engine’s electronics: the optical modulator driver and the trans-impedance amplifier (TIA). The electronics use advanced equalisation to boost the electrical channel, given direct drive has demanding requirements, says Harding.
The XT1600 optical module
Nubis’ first product is the XT1600 optical module. Here, a substrate houses the company’s PIC and electronics onto which is packaged a lid containing the optical fibres.
Nubis has developed in-house the packaging and the fibre attach solution.
The substrate is 15x15mm, somewhat larger than the engine. Harding says this is deliberate to support products under development.
The 1.6 terabits – in fact, 16x112Gbps full duplex – module has a 2km reach. Its power consumption is below 4 pJ/b.
The fibres exit the module vertically and bend to the side. “[Going] vertical is good but the 2D is the much more important aspect here,” says Winzer.
A 2D approach is logical, says Nubis. An electrical ball grid array (BGA) all the bottom surface. It makes sense that the optics is similarly massively 2D, especially for designs where its a 100-gigabit electrical signal in and a 100-gigabit optical signal out.
Multiple rings of optical I/O engines can surround the ASIC because the fibres exit vertically. “Nobody else can do that because they are escaping from the [PIC] edge,” says Winzer.
Winzer highlights another benefit of the design.
The Universal Chiplet Interconnect Express (UCIe) specification calls for 2T/mm bandwidth escape density. An optical chiplet can only achieve this if wavelength-division multiplexing (WDM) is used due to the large fibre size. Nubis can achieve this density optically without having to use WDM because of 2D surface coupling.
Doing all-to-all at scale remains a big system challenge. “We’re just a part of that challenge,” says Harding. But for optical I/O to become pervasive in the data centre over the next five years, the optics must be significantly lower power, smaller, and efficient.
“If you crack that 2D nut, you can do many, many great things down the road,” says Winzer. “We’ve solved a huge technology problem that allows us to scale much better than anybody else.”
Status
Nubis has not named its foundry and contract manufacturing partners but says they are large, high-volume manufacturers.
Harding says there are now up to five credible silicon photonic foundries available.
“There was some early product definition which some foundries were better suited to support,“ says Harding. “And there was a robustness of the initial PDKs [process design kits] to get us an early product that was important to us.”
Choosing a contract manufacturer proved easier, given the maturity of the players.
Nubis’ first product has 16 optical channels each at 112 gigabit, but future designs will offer N by 224-gigabit channels.
Meanwhile, the XT1600 optical engine is available for sampling.