Nubis' bandwidth-packed tiny optical engine
- Nubis Communications has revealed its ambitions to be an optical input-output (I/O) solutions provider
- Its tiny 1.6-terabit optical engine measures 5mm x 7.5mm
- The optical engine has a power consumption of below 4 picojoule/bit (pJ/b) and a bandwidth density of 0.5 terabits per millimetre.
- “Future systems will be I/O with an ASIC dangling off it.”
Nubis Communications has ended its period of secrecy to unveil an optical engine targeted at systems with demanding data input-output requirements.
The start-up claims its optical engine delivers unmatched bandwidth density measured in terabits per millimetre (T/mm) and power consumption performance metrics.
“In the timeframe of founding the company [in 2020], it became obvious that the solution space [for our product] was machine learning-artificial intelligence,” says Dan Harding, the CEO of Nubis.
Company Background
Nubis has raised over $40 million, with the lead investor being Matrix Partners. Venture capital company Matrix Partners backed Acacia Communications, acquired by Cisco in 2021.
Other Nubis backers are Weili Dai, a co-founder of Marvell Technologies, and Belgium-based imec.xpand.
“We have raised enough money to get to production with our product,” says Harding, who joined Nubis in 2021 from Broadcom.
Peter Winzer is the CTO and founder of the company. Formerly at Nokia Bell Labs, Winzer was the 2018 winner of the Optica (then OSA) and IEEE Photonics Society’s John Tyndall Award for his work on coherent optical communications.
Nubis has 40 staff, mostly engineers.
“As a team, we are multidisciplinary,” says Winzer. The company’s expertise includes silicon photonics, analogue IC design including serialisers/ deserialisers (serdes), packaging – electrical and optical, and software including advanced simulation tools.
“It is all geared towards a systems solution,” says Winzer. “We are not just looking at the PIC [photonic integrated circuit] or the electronics; we have the system and the architecture in mind.”
The input-output challenge
Machine learning workloads continue to grow at a staggering pace, doubling more than twice each year. Not surprisingly, computing systems running such workloads are struggling to keep up.
Scaling such systems not only requires more processing – more graphics processing units (GPUs) – but also networking to connect clusters of GPUs.
What the compute vendors want is any-to-any connectivity between processors and between clusters. This is creating a tremendous input-output challenge in terms of bandwidth density while keeping the power consumption under control.
“Over half the power of that cluster can be taken up by traditional optics,” says Harding. “So it is clear that the industry wants new solutions.”
“Whatever cents-per-gigabit [figure] you use, if you multiply it by the I/O capacity, the number you’ll get is many times that of [the cost of] an ASIC,” adds Winzer. “We say that future systems will be I/O with an ASIC dangling off it.”
Design details
Nubis’ optical engine is a 16 x 112-gigabit design with a footprint of 5mm x 7.5mm.
“Because we have our electronics flip-chipped on top, that’s the entire footprint,” says Winzer. “We maintain that it is the highest density by far of any optical engine.”
Nubis says many parallel fibres can be interfaced to the optical engine despite its tiny size.
Supporting parallel fibres is essential for machine learning systems as the fibres are fanned out to enable any-to-any connectivity.
Nubis’ engine uses a 4 by DR4 fan-out architecture with 36 fibres arranged in a 3×12 array.
Surface coupling in a 2D array interfaces the 36 fibres to the PIC: 32 fibres are for data and four for the external laser light source.
There is only a physical limit to the number of fibres that can be connected if edge coupling is used, says Winzer. But surface coupling in a 2D array means the optical engine delivers 5-10x more density than its competitors.
The start-up also has designed the engine’s electronics: the optical modulator driver and the trans-impedance amplifier (TIA). The electronics use advanced equalisation to boost the electrical channel, given direct drive has demanding requirements, says Harding.
The XT1600 optical module
Nubis’ first product is the XT1600 optical module. Here, a substrate houses the company’s PIC and electronics onto which is packaged a lid containing the optical fibres.
Nubis has developed in-house the packaging and the fibre attach solution.
The substrate is 15x15mm, somewhat larger than the engine. Harding says this is deliberate to support products under development.
The 1.6 terabits – in fact, 16x112Gbps full duplex – module has a 2km reach. Its power consumption is below 4 pJ/b.
The fibres exit the module vertically and bend to the side. “[Going] vertical is good but the 2D is the much more important aspect here,” says Winzer.
A 2D approach is logical, says Nubis. An electrical ball grid array (BGA) all the bottom surface. It makes sense that the optics is similarly massively 2D, especially for designs where its a 100-gigabit electrical signal in and a 100-gigabit optical signal out.
Multiple rings of optical I/O engines can surround the ASIC because the fibres exit vertically. “Nobody else can do that because they are escaping from the [PIC] edge,” says Winzer.
Winzer highlights another benefit of the design.
The Universal Chiplet Interconnect Express (UCIe) specification calls for 2T/mm bandwidth escape density. An optical chiplet can only achieve this if wavelength-division multiplexing (WDM) is used due to the large fibre size. Nubis can achieve this density optically without having to use WDM because of 2D surface coupling.
Doing all-to-all at scale remains a big system challenge. “We’re just a part of that challenge,” says Harding. But for optical I/O to become pervasive in the data centre over the next five years, the optics must be significantly lower power, smaller, and efficient.
“If you crack that 2D nut, you can do many, many great things down the road,” says Winzer. “We’ve solved a huge technology problem that allows us to scale much better than anybody else.”
Status
Nubis has not named its foundry and contract manufacturing partners but says they are large, high-volume manufacturers.
Harding says there are now up to five credible silicon photonic foundries available.
“There was some early product definition which some foundries were better suited to support,“ says Harding. “And there was a robustness of the initial PDKs [process design kits] to get us an early product that was important to us.”
Choosing a contract manufacturer proved easier, given the maturity of the players.
Nubis’ first product has 16 optical channels each at 112 gigabit, but future designs will offer N by 224-gigabit channels.
Meanwhile, the XT1600 optical engine is available for sampling.
ST makes its first PSM4 optical engine deliveries
What gives Benetti confidence is the demand he is seeing for 100-gigabit transceivers in the data centre. “From my visibility today, the tipping point is 2016,” says Benetti, group vice president and general manager, digital and mixed processes ASIC division at STMicroelectronics.
Flavio Benetti
Benetti and colleagues at ST have spent the last four years working to bring to market the silicon photonics technology that the chip company licensed from Luxtera.
The company has developed a 300mm-wafer silicon photonics production line at its fabrication plant in Crolles that is now up and running. ST also has its first silicon photonics product - a mid-reach PSM4 100-gigabit optical engine - and has just started its very first deliveries.
At the OFC show in March, ST said it had already delivered samples to one unnamed 'customer partner', possibly Luxtera, and Benetti showed a slide of the PSM4 chips as part of a Lumentum transceiver.
Another ST achievement Benetti highlights is the development of a complete supply chain for the technology. In addition to wafer production, ST has developed electro-optic wafer testing. This allows devices to be probed electrically and optically to select working designs before the wafer is diced. ST has also developed a process to 3D-bond chips.
“We have focussed on building an industrial environment, with a supply chain that can deliver hundreds of thousands and millions of devices,” says Benetti.
PSM4 and CWDM4
ST’s first product, the components for a 4x25 gigabit PSM4 transceiver, is a two-chip design.
One chip is the silicon photonics optical engine which integrates the PSM4’s four modulators, four detectors and the grating couplers used to interface the chip to the fibres. The second chip, fabricated using ST’s 55nm BiCMOS process, houses the transceiver’s associated electronics such as the drivers, and trans-impedance amplifiers.
The two chips are combined using 3D packaging. “The 3D packaging consists of the two dies, one copper-pillar bonded to the other,” says Benetti. “It is a dramatic simplification of the mounting process of an optical module.”
The company is also developing a 100-gigabit CWDM4 transceiver which unlike the PSM4 uses four 25-gigabit wavelengths on a single fibre.
The CWDM4 product will be developed using two designs. The first is an interim, hybrid solution that uses an external planar lightwave circuit-based multiplexer and demultiplexer, followed by an integrated silicon photonics design. The hybrid design is being developed and is expected in late 2017; the integrated silicon photonics design is due in 2018.
With the hybrid design, it is not just a question of adding a mux-demux to the PSM4 design. “The four channels are each carrying a different wavelength so there are some changes that need to be done to the PSM4,” says Benetti, adding that ST is working with partners that will provide the mux-demux and do the integration.
We need to have a 100-gigabit solution in high volume for the market, and the pricing pressure that is coming has convinced us that silicon photonics is the right thing to do
Opportunities
Despite the growing demand for 100-gigabit transceivers that ST is seeing, Benetti stresses that these are not 'mobile-phone wafer volumes'. “We are much more limited in terms of wafers,” he says. Accordingly, there is probably only room for one or two large fabs for silicon photonics globally, in his opinion.
So why is ST investing in a large production line? For Benetti, this is an obvious development for the company which has been a provider of electrical ICs for the optical module industry for years.
“ST has entered silicon photonics to provide our customers with a roadmap,” says Benetti. “We need to have a 100-gigabit solution in high volume for the market, and the pricing pressure that is coming has convinced us that silicon photonics is the right thing to do.”
It also offers chip players the possibility of increasing its revenues. “The optical engine integrates all the components that were in the old-fashioned modules so we can increase our revenues there,” he says.
ST is tracking developments for 200-gigabit and 400-gigabit links and is assessing whether there is enough of an opportunity to justify pursuing 200-gigabit interconnects.
For now though, it is seeing strong pricing pressure for 100-gigabit links for reaches of several hundred meters. “We do not think we can compete for very short reach distances,” says Benetti. “We will leave that to VCSELs until the technology can no longer follow.” As link speeds increase, the reach of VCSEL links diminishes. “We will see more room for silicon photonics but this is not the case in the short term,” says Benetti.
Market promise
People have been waiting for years for silicon photonics to become a reality, says Benetti. “My target is to demonstrate it [silicon photonics] is possible, that we are serious in delivering parts to the market in an industrial way and in volumes that have not been delivered before.”
To convince the market, it is not just showing the technological advantages of silicon photonics but the fact that there is a great simplification in constructing the optical module along with the ability to deliver devices in volume. “This is the point,” he says.
Benetti’s other role at ST is overseeing advanced networking ASICs. He argues that over the mid- to long-term, there needs to be a convergence between ASIC and optical connectivity.
“Look at a switch board, for example, you have a big ASIC or two in the middle and a bunch of optical modes on the side,” says Benetti. For him, the two technologies - photonics and ICs - are complementary and the industry’s challenge is to make the two live together in an efficient way.
ECOC 2012 summary - Part 2: Finisar
Gazettabyte completes its summary of key optical announcements at the recent ECOC show held in Amsterdam. In Part 2, Finisar's announcements are detailed.
Part 2
"The general thought with system vendors is that the more they can shrink the in-line equipment into a fewer number of slots, the more slots they have open and available for revenue-generating transceiver and transponder cards"
Rafik Ward, Finisar
Finisar showed its board-mounted parallel optics module in use within a technology demonstrator from data storage firm Xyratex, showcased what it claims is the industry's first two-slot reconfigurable optical add/ drop multiplexer (ROADM) design, unveiled its first CFP2 pluggable transceiver and announced its latest WaveShaper products.
The data storage application uses Finisar's vertical-cavity surface-emitting laser (VCSEL)-based board mounted optical assembly. The optical assembly - or optical engine - comprises 24-channels, 12 transmitters and 12 receivers.
The optical engine sits on the board and is used for such applications as chip-to-chip interconnect, optical backplanes, and dense front panels, and supports a variety of protocols. These include PCI Express, Ethernet and Infiniband as well as proprietary schemes. Indeed the only limit is the VCSEL speed. The optical engine is designed to support traffic up to 28 Gigabit-per-second (Gbps) per channel, once 28 Gigabit VCSELs become available. Finisar have already demonstrated working 28Gbps VCSELs.
The ECOC demonstration showed the optical engine in use within Xyratex's demonstrator storage system. "They are carrying traffic between internal controller cards and the traffic being carried is 12-Gig SAS [serial attached SCSI]," says Rafik Ward, vice president of marketing at Finisar.
As well as the optical engine, the demonstration included polymer waveguides from Vario-optics which connect the optical engine to a backplane connector, built by Huber + Suhner, as well as SAS silicon from LSI.
Finisar first showed the waveguide and connector technologies in a demonstration at OFC 2012. "This is an early prototype but it's a very exciting one," says Ward. "It shows all elements of the ecosystem coming together and running in a live system."
Finisar also showcased what it claims is the industry's first two-slot ROADM line card. The line card was part of a Cisco Systems' platform, according to one analyst shown the demonstration.
The company-designed card uses a high port-count wavelength-selective switch (WSS) that enables both add and drop traffic. "We have built transmit and receive into the same line card using a high port-count device," says Ward. Finisar is not detailing the exact WSS used or how the system is implemented but describes it as a flexible spectrum, 2x1x17 port line card.
The advantage of a denser ROADM line card is that it frees up slots in a system vendor's chassis. A slot can be used for either in-line equipment - WSSes and amplifiers - or terminal equipment that host the transceivers and transponders.
"It is like valuable real-estate," says Ward. "The general thought with system vendors is that the more they can shrink the in-line equipment into a fewer number of slots, the more slots they have open and available for revenue-generating transceiver and transponder cards."
The company also detailed its first CFP2 100 Gigabit optical transceiver. The CFP2 uses a single TOSA comprising four distributed feedback (DFB) lasers, a shared thermo-electric cooler and the multiplexer. The CFP2 consumes under 8W by using the DFBs and an integrated transceiver optical sub-assembly (TOSA).
Oclaro points its laser diodes at new markets
“To succeed in any market ... you need to be the best at something, to have that sustainable differentiator”
Yves LeMaitre, Oclaro
Now LeMaitre is executive vice president at Oclaro, managing the company’s advanced photonics solutions (APS) arm. The APS division is tasked with developing non-telecom opportunities based on Oclaro’s high-power laser diode portfolio, and accounts for 10%-15% of the company’s revenues.
“The goal is not to create a separate business,” says LeMaitre. “Our goal is to use the infrastructure and the technologies we have, find those niche markets that need these technologies and grow off them.”
Recently Oclaro opened a design centre in Tucson, Arizona that adds packing expertise to its existing high-power laser diode chip business. The company bolstered its laser diode product line in June 2009 when Oclaro gained the Newport Spectra Physics division in a business swap. “We became the largest merchant vendor for high-power laser diodes,” says LeMaitre.
The products include single laser chips, laser arrays and stacked arrays that deliver hundred of watts of output power. “We had all that fundamental chip technology,” says LeMaitre. “What we have been less good at is packaging those chips - managing the thermals as well as coupling that raw chip output power into fibre.”
The new design centre is focussed on packaging which typically must be tailored for each product.
Laser diodes
There are three laser types that use laser diodes, either directly or as ‘pumps’:
- Solid-state laser, known as diode-pumped solid-state (DPSS) lasers.
- Fibre laser, where the fibre is the medium that amplifies light.
- Direct diode laser - here the semiconductor diode itself generates the light.
All three types use laser diodes that operate in the 800-980nm range. Oclaro has much experience in gallium arsenide pump-diode designs for telecom that operate at 920nm wavelengths and above.
Laser diode designs for non-telecom applications are also gallium arsenide-based but operate at 800nm and above. They are also scaled-up designs, says LeMaitre: “If you can get 1W on a single mode fibre for telecom, you can get 10W on a multi-mode fibre.” Combining the lasers in an array allows 100-200W outputs. And by stacking the arrays while inserting cooling between the layers, several hundreds of watts of output power are possible.
The lasers are typically sold as packaged and cooled designs, rather than as raw chips. The laser beam can be collimated to precisely deliver the light, or the beam may be coupled when fibre is the preferred delivery medium.
“The laser beam is used to heat, to weld, to burn, to mark and to engrave,” says LeMaitre. “That beam may be coming directly from the laser [diode], or from another medium that is pumped by the laser [diode].” Such designs require specialist packaging, says LeMaitre, and this is what Oclaro secured when it acquired the Spectra Physics division.
Applications
Laser diodes are used in four main markets which Oclaro values at US$800 million a year.
One is the mature, industrial market. Here lasers are used for manufacturing tasks such as metal welding and metal cutting, marking and welding of plastics, and scribing semiconductor wafers.
Another is high-quality printing where the lasers are used to mark large printing plates. This, says LeMaitre, is a small specialist market.
Health care is a growing market for lasers which are used for surgery, although the largest segment is now skin and hair treatment.
The final main market is consumer where vertical-cavity surface-emitting lasers (VCSELs) are used. The VCSELs have output powers in the tens or hundreds of milliwatts only and are used in computer mouse interfaces and for cursor navigation in smartphones.
“These are simple applications that use lasers because they provide reliable, high-quality optical control of the device,” says LeMaitre. “We are talking tens of millions of [VCSEL] devices [a year] that we are shipping right now for these types of applications.”
Oclaro is a supplier of VCSELs for Light Peak, Intel’s high-speed optical cable technology to link electronic devices. “There will be adoptions of the initial Light Peak starting the end of this year or early next year, and we are starting to ramp up production for that,” says LeMaitre. “In the meantime, there are many alternative [designs] happening – the market is extremely active – and we are talking to a lot of players.” Oclaro sells the laser chips for such interface designs; it does not sell optical engines or the cables.
Is Oclaro pursuing optical engines for datacom applications, linking large switch and IP router systems? “We are actively looking at that but we haven’t made any public announcements,” he says.
Market status
LeMaitre has been at Oclaro since 2008 when Avanex merged with Bookham (to become Oclaro). Before that, he was CEO at optical component start-up, LightConnect.
How does the industry now compare with that of a decade ago?
“At that time [of the downturn] the feeling was that it was going to be tough for maybe a year or two but that by 2002 or 2003 the market would be back to normal,” says LeMaitre. “Certainly no-one expected the downturn would last five years.” Since then, nearly all of the start-ups have been acquired or have exited; Oclaro itself is the result of the merger of some 15 companies.
“People were talking about the need for consolidation, well, it has happened,” he says. Oclaro’s main market – optical components for metro and long haul transmission – now has some four main players. “The consolidation has allowed these companies, including Oclaro, to reach a level of profitability which has not been possible until the last two years,” says LeMaitre.
Demand for bandwidth has continued even with the recent economic downturn, and this has helped the financial performance of the optical component companies.
“The need for bandwidth has still sustained some reasonable level of investment even in the dark times,” he says. “The market is not as sexy as it was in those [boom] days but it is much more healthy; a sign of the industry maturing.”
Industry maturity also brings corporate stability which LeMaitre says provides a healthy backdrop when developing new business opportunities.
The industrial, healthcare and printing markets require greater customisation than optical components for telecom, he says, whereas the consumer market is the opposite, being characterised by vastly greater unit volumes.
“To succeed in any market – this is true for this market and for the telecom market – you need to be the best at something, to have that sustainable differentiator,” says LeMaitre. For Oclaro, its differentiator is its semiconductor laser chip expertise. “If you don’t have a sustainable differentiator, it just doesn’t work.”