Intel details its 800-gigabit DR8 optical module
The company earmarks 2023 for its first co-packaged optics product
Intel is sampling an 800-gigabit DR8 in an OSFP pluggable optical module, as announced at the recent OFC virtual conference and show.
“It is the first time we have done a pluggable module with 100-gigabit electrical serdes [serialisers/ deserialisers],” says Robert Blum, Intel’s senior director, marketing and new business. “The transition for the industry to 100-gigabit serdes is a big step.”
The 800-gigabit DR8 module has eight electrical 100-gigabit interfaces and eight single-mode 100-gigabit optical channels in each transmission direction.
Intel demonstrated a prototype 12.8-terabit co-packaged optics design
The attraction of the single-module DR8 design, says Blum, is that it effectively comprises two 400-gigabit DR4 modules. “The optical interface allows you the flexibility that you can break it out into 400-gigabit DR4,” says Blum. “You can also do single 100-gigabit breakouts or you can do 800-gigabit-to-800-gigabit traffic.”
Intel expects volume production of the DR8 in early 2022. Developing a DR8 in a QSFP-DD800 form factor will depend on customer demand, says Blum.
Intel will follow the 800-gigabit DR8 module with a dual 400G FR4, expected later in 2022. The company is also developing a 400-gigabit FR4 module that is expected then.
Meanwhile, Intel is ramping its 200-gigabit FR4 and 400-gigabit DR4 modules.
51.2-terabit co-packaged optics
Intel demonstrated a prototype 12.8-terabit co-packaged optics design, where the optics is integrated alongside its Tofino 2 Ethernet switch chip, at last year’s OFC event.
The company says its first co-packaged optics design will be for 51.2-terabit switches and is scheduled in late 2023. “We see smaller-scale deployments at 51.2 terabits,” says Blum.
Moving the industry from pluggable optical modules to co-packaged optics is a big shift, says Intel. The technology brings clear system benefits such as 30 per cent power savings and lower cost but these must be balanced against the established benefits of using pluggable modules and the need to create industry partnerships for the production of co-packaged optics.
The emergence of 800-gigabit client-side pluggable modules such as Intel’s also means a lesser urgency for co-packaged optics. “You have something that works even if it is more expensive,” says Blum.
Thirty-two 800-gigabit modules can serve a 25.6-terabit switch in a one rack unit (1RU) platform.
However, for Intel, the crossover point occurs once 102.4-terabit switch chips and 200-gigabit electrical interfaces emerge.
“We see co-packaged optics as ubiquitous; we think pluggables will no longer make sense at that point,” says Blum.
FPGA-based optical input-output
Intel published a paper at OFC 2021 highlighting its latest work a part of the U.S. DARPA PIPES programme.
The paper describes a co-packaged optics design that adds 8 terabits of optical input-output (I/0) to its Stratix 10 FPGA. The design uses Ayar Labs’ TeraPHY chiplet for the optical I/O.
The concept is to use optical I/O to connect compute nodes – in this case, FPGAs – that may be 10s or 100s of meters apart.
Intel detailed its first Stratix 10 with co-packaged optical I/O two years ago.
The latest multi-chip package also uses a Stratix 10 FPGA with Intel’s Advanced Interface Bus (AIB), a parallel electrical interface technology, as well as the Embedded Multi-die Interconnect Bridge (EMIB) technology which supports the dense I/O needed to interface the FPGA to the TeraPHY chiplet. The latest design integrates five TeraPHYs compared to the original one that used two. Each chiplet offers 1.6 terabits of capacity such that the FPGA-based co-package has 8 terabits of I/O in total.
Optically enabling Ethernet silicon or an FPGA is part of the industry’s vision to bring optics close to the silicon. Other devices include CPUs and GPUs and machine-learning devices used in computing clusters that require high-density interconnect (see diagram below).
“It is happening first with some of the highest bandwidth Ethernet switches but it is needed with other processors as well,” says Blum.
The Intel OFC 2021 paper concludes that co-packaged optics is inevitable.
Milestones, LiDAR and sensing
Intel has shipped a total of over 5 million 100-gigabit optical modules, generating over $1 billion of revenues.
Blum also mentioned Intel’s Mobileye unit which in January announced its LiDAR-on-a-chip design for autonomous vehicles.
“We have more than 6,000 individual components on this LiDAR photonic integrated circuit,” says Blum. The count includes building blocks such as waveguides, taps, and couplers.
“We have this mature [silicon photonics] platform and we are looking at where else it can be applied,” says Blum.
LiDAR is one obvious example: the chip has dozens of coherent receivers on a chip and dozens of semiconductor optical amplifiers that boost the output power into free space. “You really need to integrate the different functionalities for it to make sense,” says Blum.
Intel is also open to partnering with companies developing biosensors for healthcare and for other sensing applications.
Certain sensors use spectroscopy and Intel can provide a multi-wavelength optical source on a chip as well as ring-resonator technology.
“We are not yet at a point where we are a foundry and people can come but we could have a collaboration where they have an idea and we make it for them,” says Blum.
Enabling 800-gigabit optics with physical layer ICs
Broadcom recently announced a family of 800-gigabit physical layer (PHY) chips. The device family is the company’s first 800-gigabit ICs with 100-gigabit input-output (I/O) interfaces.
Source: Broadcom
Moving from 50-gigabit to 100-gigabit-based I/O enables a new generation of 800-gigabit modules aligned with the latest switch chips.
“With the switch chip having 100-gigabit I/Os, PHYs are needed with the same interfaces,” says Machhi Khushrow, senior director of marketing, physical layer products division at Broadcom.
Broadcom’s latest 25.6 terabit-per-second (Tbps) Tomahawk 4 switch chip using 100-gigabit I/O was revealed at the same time.
800-gigabit PHY devices
The portfolio comprises three 800-gigabit PHY ICs. All operate at a symbol rate of 53 gigabaud, use 4-level pulse amplitude modulation (PAM-4) and are implemented in a 7nm CMOS process.
Two devices are optical PHYs: the BCM87800 and the BCM87802. These ICs are used within 800-gigabit optical modules such as the QSFP-DD800 and the OSFP form factors. The difference between the two chips is that the BCM87802 includes an integrated driver.
The third PHY - the BCM87360 - is a retimer IC used on line cards. Whether the chip is needed depends on the line card design and signal-integrity requirements; for example, whether the line card is used within a pizza box or part of a chassis-based platform.
Source: Broadcom
“If it is a higher-density card that is relatively small, it may only need 15 per cent of the ports with retimers,” says Khushrow. “If the line card is larger, where things fan out to longer traces, retimers may be needed for all the ports.”
All three 800-gigabit PHYs have eight 100-gigabit transmit and eight receive channels (8:8, as shown in the top diagram).
Applications
The optical devices support several 800-gigabit module designs that use either silicon photonics, directly modulated lasers (DMLs) or externally-modulated lasers (EMLs).
The 800-gigabit PHYs support the DR8 module (8 single-mode fibres, 500m reach), two 400-gigabit DR4 (4 single-mode fibres, 500m) or two FR4 in a module (each 4 wavelengths on a single-mode fibre, 2km) as well as the SR8, a parallel VCSEL-based design with a reach of 100m over parallel multi-mode fibre.
Timescales
Given the availability of these PHYs and that 800-gigabit modules will soon appear, will the development diminish the 400-gigabit market opportinity?
“This is independent of 400-gigabit [module] deployments,” says Khushrow.
The hyperscalers are deploying different architectures. There are hyperscalers that are only now transitioning to 200-gigabit modules while others are transitioning to 400- gigabit. They will all transition to 800 gigabit, he says: “How and when they transition are all at different points.”
Some of the hyperscalers deploying 400-gigabit modules are looking at 800 gigabit, and their deployment plans are maybe two to three years out. “We don’t expect 800 gigabit to cannibalise 400 gigabit, at least not in the near term,” he says.
Broadcom says 800-gigabit modules to ship in the second half of this year. “It all depends on how the switch infrastructure, line cards and optics become available,” says Khushrow.
Next developments
The landscape for high-speed networking in the data centre is changing and optics is moving closer to the switch chip, whether it is on-board optics or co-packaged optics.
“People are looking at both options,” says Khushrow.”It depends on the architecture of the data centre whether they use on-board optics or co-packaged optics.”
Meanwhile, the OIF is working on a 200-gigabit electrical interface standard.
Co-packaged optics is challenging and the technology has its own issues whereas optical transceivers are easier to use and deploy, says Khushrow.
Current industry thinking is that some form of co-packaged optics will be used with the adevnt of next-generation 51.2-terabit switch chips. But even with such capacity switches, pluggables will continue to be used, he says.
There will still be a need for PHYs, whether for pluggables, co-packaged designs or on the linecard.
“We will continue to provide those on our roadmap,” says Khushrow. “It is just a matter of what the form factor will be, whether it will be a packaged part or a die part.”
Silicon Photonics spills over into new markets
The market for silicon photonics is set to grow eightfold by 2025. So claims market research firm, Yole Développement, in its latest report on silicon photonics, a technology that enables optical components to be made on a silicon substrate.
Silicon photonics is also being used in new markets although optical transceivers will still account for the bulk of the revenues in 2025.
Source: Yole
Market forecast
“We are entering a phase where we are beyond the tipping point [for silicon photonics],” says Eric Mounier, fellow analyst at Yole. “There is no doubt silicon photonics will grow and will be used beyond the data centre.”
Yole sized the 2019 global silicon photonics market at US $480 million, dominated by sales of optical transceivers for the data centre. In 2025 the forecast is for a $3.9 billion market, with data centre transceivers accounting for over 90 per cent of the market.
Eric Mounier
Revenues from new markets such as 5G optical transceivers, automotive, co-packaged optics, fibre-optic gyroscopes, and biochemical sensors will generate $165 million revenues in 2025.
The Yole report also highlights a maturing supply chain, advances in co-packaged optics, and more silicon photonics start-up announcements in the last year.
“It seems the big data centre operators, telecom players and sensor companies are convinced silicon photonics is a key technology for integration, lower cost and smaller components for interconnect and sensing applications,” says Mounier.
Optical transceivers
Data centre optical transceivers account for the bulk of silicon photonics’ market value and unit volumes.
Three-quarters of revenues in 2019 were for data centre transceivers for reaches ranging from several hundred meters to 2km and 10km. This market for silicon photonics is dominated by two players: Intel and Cisco with its Luxtera acquisition.
“For 100-gigabit transceivers, silicon photonics is probably the most used technology compared to legacy optics,” says Mounier.
The remaining 2019 revenues were from long-haul coherent transceiver sales, a market dominated by Acacia that is being acquired by Cisco.
Other companies involved in the transceiver supply chain include Innolight, Juniper Networks, and Alibaba with its work with Elenion Technologies (Elenion was recently acquired by Nokia). HP is working with several firms to develop its silicon photonics supply chain, from device design to final products.
The rollout of 5G is generating a need for 10-gigabit and 25-gigabit transceivers for distances up to 100m, linking remote radio heads and the baseband unit, part of the 5G radio access network.
Yole forecasts a $61 million 5G transceiver market in 2025.
Co-packaged optics
The packaging of optical input-output with a digital chip, known as co-packaged optics, has made notable progress in the last year.
“We are pretty convinced that co-packaged optics is the next big application for silicon photonics,” says Mounier.
Intel has demonstrated its optics packaged with the Tofino 2 Ethernet switch chip it gained with the Barefoot Networks acquisition. “Talking to Intel, I believe in two to three years from now, there will be the first product,” he says.
Other firms pursuing co-packaged optics include Ranovus, Rockley Photonics, Ayar Labs and Sicoya.
The doubling in Ethernet switch-chip capacity every two years is a key driver for co-packaged optics. Switch chips with 25.6-terabit capacity exist and 51.2-terabit switches will be shipping by 2025.
There will also be eight-hundred-gigabit pluggable transceivers in 2025 but Yole says co-packaged optics offers a systems approach to increasing channel counts to keep pace with growing switch capacities.
Foundries and design houses
More than 10 foundries exist worldwide offering silicon photonics services.
“Foundries are interested in silicon photonics because they see a future opportunity for them to fill their fabs,” says Mounier.
Yole cites how GlobalFoundries is working with Ayar Labs, HP with TSMC, Sicoya with IHP Microelectronics, and Rockley Photonics with VTT Memsfab. TSMC also works with Cisco through its Luxtera acquisition.
Swedish MEMS foundry, Silex Microsystems, is developing a portfolio of silicon photonics technology. “They are working with many players developing telecom photonic platforms,” says Mounier.
There are also several design houses offering photonic design services to companies that want to bring products to market. Examples include VLC Photonics, Luceda, Photon Design and Effect Photonics.
Optical design requires know-how that not all firms have, says Mounier. Such silicon photonics design services recall the ASIC design houses that provided a similar service to the electronics industry some two decades ago.
Sensors
Lidar used for autonomous cars and biochemical chips are two emerging sensor markets embracing silicon photonics. Lidar (light detection and ranging) uses light to sense a vehicle’s surroundings.
“Lidar systems are bulky and expensive and a car needs several, at the front, rear and sides,” says Mounier. “Silicon photonics is an emerging platform for the integration of such devices.”
Two Lidar approaches are using silicon photonics: frequency modulation continuous wave (FMCW) Lidar, also known as coherent Lidar, and an optical phased array.
For coherent Lidar, the transmitted frequency of the laser - represented by the local oscillator - and the reflected signal are mixed coherently. This enables phase and amplitude information to be recovered to determine an object’s position and velocity.
SiLC Technologies has developed a FMCW Lidar chip. Working with Varroc Lighting Systems, the two firms have demonstrated Lidar integrated into a car headlamp.
The second approach - an optical phased array - steers the beam of light without using any moving parts.
Lidar is complex and can be implemented using other technologies besides silicon photonics, says Mounier: “Silicon photonics for Lidar has several advantages but it is not clear why the technology will be used in the car or for robotic vehicles.”
In turn, the emerging economic crisis worldwide will likely delay the development of the autonomous car, he says.
Other sensor developments include silicon photonics-based biosensors from Genalyte that use lasers, micro-ring resonators and detectors to produce fast biological test results. The US company has raised over $90 million in three rounds of funding.
French firm Aryballe produces a tiny photonic IC that acts as an electronic nose (digital olfaction). “Using silicon photonics, you can integrate everything on a chip,” says Mounier. “It needs less packaging and assembly and you get a tiny chip at the end.”
COVID-19
Silicon photonics shipments have been delayed in the first half of 2020 due to the COVID-19 pandemic, says Yole. But the market for silicon photonics will still grow this year albeit not at the originally forecasted 10 per cent.
“Everyone is working from home and there is a need for more networking bandwidth,” says Mounier. There will continued demand for transceivers for the data centre and telecom services.
“Market growth will be positive for telecoms, and markets such as defence and medical will not be much impacted,” he says.
ECOC 2019 industry reflections
Gazettabyte is asking industry figures for their thoughts after attending the recent ECOC show, held in Dublin. In particular, what developments and trends they noted, what they learned and what, if anything, surprised them. Here are the first responses from Huawei, OFS Fitel and ADVA.
James Wangyin, senior product expert, access and transmission product line at Huawei
At ECOC, one technology that is becoming a hot topic is machine learning. There is much work going on to model devices and perform optimisation at the system level.
And while there was much discussion about 400-gigabit and 800-gigabit coherent optical transmissions, 200-gigabit will continue to be the mainstream speed for the coming three-to-five years.
That is because, despite the high-speed ports, most networks are not being run at the highest speed. More time is also needed for 400-gigabit interfaces to mature before massive deployment starts.
BT and China Telecom both showed excellent results running 200-gigabit transmissions in their networks for distances over 1,000km.
We are seeing this with our shipments; we are experiencing a threefold year-on-year growth in 200-gigabit ports.
Another topic confirmed at ECOC is that fibre is a must for 5G. People previously expressed concern that 5G would shrink the investment of fibre but many carriers and vendors now agree that 5G will boost the need for fibre networks.
As for surprises at the show, the main discussion seems to have shifted from high-speed optics to system-level or device-level optimisation using machine learning.
Many people are also exploring new applications based on the fibre network.
For example, at a workshop to discuss new applications beyond 5G, a speaker from Orange talked about extending fibre connections to each room, and even to desktops and other devices. Other operators and systems vendors expressed similar ideas.
Verizon discussed, in another market focus talk, its monitoring of traffic and the speed of cars using fibre deployed alongside roads. This is quite impressive.
We are also seeing the trend of using fibre and 5G to create a fully-connected world.
Such applications will likely bring new opportunities to the optical industry.
Two other items to note.
The Next Generation Optical Transport Network Forum (NGOF) presented updates on optical technologies in China. Such technologies include next-generation OTN standardisation, the transition to 200 gigabits, mobile transport and the deployment of ROADMs. The NGOF also seeks more interaction with the global community.
The 800G Pluggable MSA was also present at ECOC. The MSA is also keen for more companies to join.
Daryl Inniss, director, new business development at OFS Fitel
There were many discussions about co-packaged optics, regarding the growth trends in computing and the technology’s use in the communications market.
This is a story about high-bandwidth interfaces and not just about linking equipment but also the technology’s use for on-board optical interconnects and chip-to-chip communications such as linking graphics processing units (GPUs).
I learned that HPE has developed a memory-centric computing system that improves significantly processing speed and workload capacity. This may not be news but it was new to me. Moreover, HPE is using silicon photonics in its system including a quantum dot comb laser, a technology that will come for others.
As for surprises, there was a notable growing interest in spatial-division multiplexing (SDM). The timescale may be long term but the conversations and debate were lively. Two areas to watch are in proprietary applications such as very short interconnects in a supercomputer and for undersea networks where the hyperscalers quickly consume the capacity on any newly commission link.
Lastly, another topic of note was the use of spectrum outside the C-band and extending the C-band itself to increase the data-carrying capacity of the fibre.
Jörg-Peter Elbers, senior vice president, advanced technology, ADVA
Co-packaging optics with electronics is gaining momentum as the industry moves to higher and higher silicon throughput. The advent of 51.2 terabit-per-second (Tbps) top-of-rack switches looks like a good interception point. Microsoft and Facebook also have a co-packaged optics collaboration initiative.
As for coherent, quo vadis? Well, one direction is higher speeds and feeds. What will the next symbol rate be for coherent after 60-70 gigabaud (GBd)? A half-step or a full-step; incremental or leap-frogging? The growing consensus is a full-step: 120-140 GBd.
Another direction for coherent is new applications such as access/ aggregation networks. Yet cost, power and footprint challenges will have to be solved.
Advanced optical packaging, an example being the OIF IC-TROSA project, as well as compact silicon photonics and next-gen coherent DSPs are all critical elements here.
A further issue arising from ECOC is whether optical networks need to deliver more than just bandwidth.
Latency is becoming increasingly important to address time-sensitive applications as well as for advanced radio technologies such as 5G and beyond.
Additional applications are the delivery of precise timing information (frequency, time of day, phase synchronisation) where the existing fibre infrastructure can be used to deliver additional services.
An interesting new field is the use of the communication infrastructure for sensing, with Glenn Wellbrock giving a presentation on Verizon’s work at the Market Focus.
Other topics of note include innovation in fibres and optics for 5G.
With spatial-division multiplexing, interest in multi-core and multi-mode fibre applications have weakened. Instead, more parallel fibres operating in the linear regime appear as an energy-efficient, space-division multiplexing alternative.
Hollow-core fibres are also making progress, offering not only lower latencies but lower nonlinearity compared to standard fibres.
As for optics for 5G, what is clear is that 5G requires more bandwidth and more intelligence at the edge. How network solutions will look will depend on fibre availability and the associated cost.
With eCPRI, Ethernet is becoming the convergence protocol for 5G transport. While grey and WDM (G.metro) optics, as well as next-generation PON, are all being discussed as optical underlay options. Grey and WDM optics offer an unbundling on the fibre/virtual fibre level whereas (TDM-)PON requires bitstream access.
Another observation is that radio “x-haul” [‘x’ being front, mid or back] will continue to play an important role for locations where fibre is nonexistent and uneconomical.
Lumentum completes sale of certain datacom lines to CIG
Brandon Collings, CTO of Lumentum, talks CIG, 400ZR and 400ZR+, COBO, co-packaged optics and why silicon photonics is not going to change the world.
Lumentum has completed the sale of part of its datacom product lines to design and manufacturing company, Cambridge Industries Group.
The sale will lower the company's quarterly revenues by between $20 million to $25 million. Lumentum also said that it will stop selling datacom transceivers in the next year to 18 months.
The move highlights how fierce competition and diminishing margins from the sale of client-side modules is causing optical component companies to rethink their strategies.
Lumentum’s focus is now to supply its photonic chips to the module makers, including CIG. “From a value-add point of view, there is a lot more value in selling those chips than the modules,” says Brandon Collings, CTO of Lumentum.
400ZR and ZR+
Lumentum will continue to design and sell line-side coherent optical modules, however.
“With coherent, there is a lot of complexity and challenge in the module’s design and manufacture,” says Collings. “We believe we can extract the value we need to continue in that business.”
The emerging 400ZR and 400ZR+ are examples of such challenging coherent interfaces.
The 400ZR specification, developed by the Optical Internetworking Forum (OIF), is a 400-gigabit coherent interface with an 80km reach. The 400 gigabit-per-second (Gbps) line rate will be achieved using a 64-gigabaud symbol rate and a 16-QAM modulation scheme.
>
“[400ZR] is not client-side. Sixty-four gigabaud is very hard to do in such an extremely compact form factor.
”
Module makers will implement the 400ZR interface using client-side pluggable modules such as the QSFP-DD and the OSFP to enable data centre operators to add coherent interfaces directly to their switches.
But implementing 400ZR will be a challenge. “This is not client-side,” says Collings. “Sixty-four gigabaud is very hard to do in such an extremely compact form factor.”
First samples of 400ZR modules are expected by year-end.
The 400ZR+ interface, while not a specification, is a catch-all for a 400-gigabit coherent that exceeds the 400ZR specification. The 400ZR+ will be a multi-rate design that will support additional line rates of 300, 200 and 100Gbps. Such rates coupled with more advanced forward-error correction (FEC) schemes will enable the 400ZR+ to span much greater distances than 80km.
The 400ZR+ interface helps the developers of next-generation coherent DSP chips to recoup their investment by boosting the overall market their devices can address. “It is basically a way of saying I’m going to spend $50 million developing a coherent DSP, and the 400ZR market alone is not big enough for that investment,” says Collings.
Lumentum says there will be some additional functionality that will be possible to fit into a QSFP-DD such that at least one of the ZR+ modes will be supported. But given the QSFP-DD module’s compactness and power constraints, the ZR+ will also be implemented in the CFP2 form factor that has the headroom needed to fully exploit the coherent DSP’s capabilities to also address metro and regional networks.
400ZR+ modules are expected in volume by the end of 2020 or early 2021.
DSP economics
Lumentum will need to source a coherent DSP for its 400ZR/ ZR+ designs as it does not have its own coherent chip. At the recent OFC show held in San Diego, the talk was of new coherent DSP players entering the marketplace to take advantage of the 400ZR/ZR+ opportunity. Collings says he is aware of five DSP players but did not cite names.
NEL and Inphi are the two established suppliers of merchant coherent DSPs. Lumentum (Oclaro) has partnered with Acacia Communications to use its Meru DSP for Lumentum’s CFP2-DCO design, although it is questionable whether Acacia will license its DSP for 400ZR/ ZR+, at least initially.
>
“God forbid if 10 or more players are doing this as no matter how you slice it, people will be losing [money]”
Lumentum and Oclaro also partnered with Ciena to use its WaveLogic Ai for a long-haul module. That leaves room for at least one more provider of a coherent DSP that could be a new entrant or an established system vendor that will license an internal design.
Collings points out that it makes no sense economically to have more than five players. If it takes $50 million to tape out a 7nm CMOS coherent DSP, the five players will invest a total of $250 million. And if the investment cost for the module, photonics and everything else is a comparable amount, that equates to $500 million being spent on the 400-gigabit coherent generation.
As for the opportunity, Collings talks of about a total of up to 500,000 ports a year by 2020. That equates to an investment return in the first year of $1,000 per device sold. “God forbid if 10 or more players are doing this as no matter how you slice it, people will be losing [money].”
Beyond Pluggables
The evolution of optics beyond pluggables was another topic under discussion at OFC.
The Consortium of On-Board Optics (COBO), the developerof an interoperable optical solution that embeds optics on the line card, had a stand at the show and a demonstration of its technology. In turn, co-packaged optics, the stage after COBO in the evolution of optical interfaces that will integrate the optics with the silicon in one package, is also now also on companies' agenda.
Collings explains that COBO came about because the industry thought on-board optics would be needed given the challenge of 400-gigabit pluggables meeting the interface density needed for 12.8-terabit switches . “I shared that opinion four to five years ago,” he says, adding that Lumentum is a member of COBO.
>
“That problem is real. It is a matter of how far the current engineering can go before it becomes too painful.”
But 400-gigabit optics has been engineered to meet the required faceplate density, including ZR for coherent. As a result, COBO is less applicable. “That need to break the paradigm is a lot less,” he says.
That said, Collings says COBO has driven valuable industry discussion given that the data centre is heading in a direction where 32 ports of 800-gigabit interfaces will be needed to get data in and out of next-generation, 25-terabit switches.
“That problem is real,” says Collings. “It is a matter of how far the current engineering can go before it becomes too painful.” Scaling indefinitely what is done today is not an option, he says.
It is possible with the next generation of switch chip to simply use a two-rack-unit box with twice as many 400-gigabit modules. “That has already been done at the 100-gigabit generation that lasted longer because it doubled up the 100-gigabit port count,” he says.
“In the generation after that, you are now asking for stuff that looks very challenging with today’s technology,” he says. “And that is where co-packaging is focused, the 50-terabit switch generation.” Switches using such capacity silicon are expected in the next four years.
But this is where it gets tricky, as co-packaging not only presents significant technical challenges but also will change the supply chain and business models.
Collings points out that hyperscalars do not like making big pioneering investments in new technology, rather they favour buying commodity hardware. “They don’t like risk, they love competition, and they like a healthy ecosystem,” he says.
“There is a lot of talk from the technology direction of how we can solve this problem [using co-packaged optics] but I think on the business side, the riskside, the investment side is putting a lot of pressure on that actually happening,” says Collings. “Where it ends up I don’t honestly know.”
Silicon photonics
One trend evident at OFC was the growing adoption of silicon photonics by optical component companies.
Indeed, the market research firm, LightCounting, in a research note summarising OFC 2019, sees silicon photonics as a must-have technology given co-packaged optics is now clearly on the industry’s roadmap.
However, Collings stresses that Lumentum’s perspective remains unchanged regarding the technology.
“It’s a fabless exercise so we can participate in silicon photonics and, quite frankly, that is why a lot of other companies are participating because the barrier to entry is quite low,” says Collings. “Nevertheless, we look at silicon photonics as another tool in the toolbox: it has advantages in some areas, some significant disadvantages in others, and in some places, it is simply comparable.”
When looking at a design from a system perspective such as a module, other considerations come into play besides the cost of the silicon photonics chip itself. Collings cites the CFP2 coherent module. While the performance of its receiver is good using silicon photonics, the modulator is questionable. You also need a laser and a semiconductor optical amplifier to compensate for silicon photonics higher loss, he says,
The alternative is to use an indium phosphide-based design and that has its own design issues. “What we are finding when you look at the right level is that the two are the same or indium phosphide has the advantage,” says Collings. “And as we go faster, we are finding silicon is not really keeping up in bandwidth and performance.”
As a result, Lumentum is backing indium phosphide for coherent operating at 64 gigabaud.
“A lot of people are talking about silicon photonics because they can talk about it,” says Collings. “It’s not worthless, don’t get me wrong, but its success outside of Acacia has been niche, and Acacia is top notch at doing this stuff.”
Interview: Finisar’s CEO reflects on a notable year
Michael Hurlston has had an eventful 2018.
The year started with him replacing Finisar’s veteran CEO, Jerry Rawls, and it is now ending with Finisar being acquired by the firm II-VI for $3.2 billion.
Finisar is Hurlston’s first experience in the optical component industry, having spent his career in semiconductors. One year in and he already has strong views about the industry and its direction.
Michael Hurlston
“We have seen in the semiconductor industry a period of massive consolidation in the last three to four years,” says Hurlston, in his first interview sinced the deal was announced. “I think it is not that different in optics: scales matters.”
Hurlston says that, right from the start, he recognised the need to drive industry consolidation. “We had started thinking about that fairly deeply at the time the Lumentum-Oclaro acquisition was announced and that gave us more impetus to look at this,” says Hurlston. The result was revealed in November with the announced acquisition of Finisar by II-VI.
“Finisar considered so many deals in the past but could not converge on a solution,” says Vladimir Kozlov, CEO and founder of market research firm, LightCounting. "It needed a new CEO to bring a different perspective. The new II-VI will look more like many diversified semiconductor vendors, addressing multiple markets: automotive, industrial and communications."
“We really have two complementary companies for the most part,” says Hurlston, who highlights VCSELs and reconfigurable optical add-drop multiplexers (ROADMs) as the only product segments where there is overlap. Merging II-VI and Finisar with disparate portfolios further benefits scale, he says.
Chip background
Hurlston’s semiconductor experience was gained at Broadcom and involved Wi-Fi devices. The key lessons he learned there is the importance of offering differentiated products to customers and the need to expand into new application areas.
“Wi-Fi is a standard, a technology, that has rules as you have to interoperate between different chipsets and different producers,” says Hurlston. “But we did find ways to differentiate under a standards umbrella.”
>
“It turns out co-packaging is a great top-line opportunity for optics companies because eventually we will be tasked with pulling together that sub-system”
What he has found, to his surprise, is that it is harder to differentiate in the optical components industry. “What we are trying to do is find spots where we can offer differentiation,” says Hurlston.
Optical components usage needs to also expand into new segments, he says, just as Wi-Fi evolved from a PC-centric technology to home networking and ultimately mobile handsets.
Hurlston cites as an example in the optical components industry how VCSELs are now being used for 3D sensing in handsets. There are also emerging opportunities in automotive and the data centre.
For the automative market, applications include in-cabin sensing to assist drivers and LIDAR (laser detection and ranging) to help vehicles build up an image of their surroundings in real-time. “LIDAR is further out but it is a significant opportunity,” says Hurlston.
For data centres, a key opportunity silicon co-packaging: bringing optics closer to switch silicon.
Currently, switch platform use pluggable optical modules on the faceplate to send and receive data. But with switch silicon capacity doubling every two years, the speed and density of the input-output means optics will have to get closer to the switch silicon.
On-board optics - as promoted by the Consortium for On-Board Optics (COBO) - is one option. Another is co-packaged optics, where the optics and silicon are placed in the same package.
“It turns out co-packaging is a great top-line opportunity for optics companies because eventually we will be tasked with pulling together that sub-system,” says Hurlston. “The integration of the switch chip and optics is something that will be technically difficult and necessitate differentiation.”
Challenges
As well as the issue of acquisitions, another area Hurlston has tackled in his short tenure is Finisar’s manufacturing model and how it can be improved.
“Finisar is a technology company at heart but the life-blood of the company is manufacturing,” he says.
Manufacturing is also one area where there is a notable difference between chips and optics. “There are manufacturing complexities with semiconductors and semiconductor process but optics takes it to a whole different level,” he says.
This is due to the manufacturing complexity of optical transceiver which Finisar’s CEO likens to manufacturing a mobile phone. There are chips that need a printed circuit board onto which are also added optical subassemblies housing such components as lasers and photo-detectors.
“Part of it [the complexity] is the human labour - the human touch - that is involved in the manufacturing and assembling of these transceivers ” he says. Finisar says its laser fab employs several hundred people whereas its optical transceiver factories employ thousands: 5,000 staff in Malaysia and some 5,500 in China.
“Our manufacturing model has been where I’ve spent a lot of time,” says Hurston. Some efficiencies have been gained but not nearly as much as he initially hoped.
Consolidation
One of the issues that has hindered greater industry consolidation has been the need for synergy between companies. A semiconductor company will only acquire or merge with another semiconductor company, and the same with a laser company looking for another laser player, he says. “What I admire about II-VI is that they are pretty bold,” says Hurlston. “What II-VI did is go after something that is not overlapping.”
He believes the creation of such broad-based suppliers is something the optics industry will have to do more of: “The transceiver guys are going to have to go after different areas of the value chain.”
In most mature industries, three large diversified companies typically dominate the marketplace. Given Lumentum’s acquisition of Oclaro has just closed and II-VI’s acquisition of Finisar is due to be completed in mid-2019, will there be another large deal?
“This is a big industry and the opportunity today and going forward is big,” says Hurlston. But there are so many players in different parts of the supply chain such that he is unsure whether these niche companies will survive in the long run.
“Whether there will be three, four or five large players, I don’t know,” he says. “But we are definitely going to see fewer; this [II-VI - Finisar deal] isn't the last transaction that drives industry consolidation.”
>
“Whether there will be three, four or five large players, I don’t know but we are definitely going to see fewer”
How will Finisar make optical transceivers in such a competitive marketplace, that includes an increasing number of Chinese entrants, while delivering gross margins that meet Wall Street expectations?
Finisar does have certain advantages, he says, such as making its own lasers. “We also make our own semiconductors, a lot of the semiconductor solutions the Chinese guys have are sourced,” he says. “That gives us an inherent advantage.”
Having its own manufacturing facilities in the Far East means that Chinese players have no inherent manufacturing advantage there. However, he admits that the gross margin expected of Finisar is higher that its Chinese competitors.
This is why Finisar’s CEO stresses the need to pursue pockets of differentiation and why the company has to be first to market in important productareas that all players will target. “We historically have not been first to market,” he says. “We have made adjustments in the last year in our time-to-market and our ability to get to big products transitions that will be hyper-competitive first.”
Hurston expresses some satisfaction in the improved revenues and gross margins as reported in Finisar’s last two quarters’ results, albeit these quarters coming after what he calls ‘a low base’.
“We have also made significant progress in 3D sensing that has been a big challenge for us,” he says.
What next?
Hurlston says he hopes to have a role in the new company once the deal closes.
“But If I don’t, I’ve really enjoyed working with the [Finisar] team and in this space,” he says. “It’s been a bit of a learning curve but I’ve learnt a couple of tricks. Hopefully there will be another opportunity to apply some of that learning to a job elsewhere.”
COBO: specification work nearing completion
The Consortium for On-board Optics (COBO) is on target to complete its specifications work by the year end. The work will then enter a final approval stage that will take up to a further three months.
On-board optics, also known as mid-board or embedded optics, have been available for years but vendors have so far had to use custom products. The goal of COBO, first announced in March 2015 and backed by such companies as Microsoft, Cisco Systems, Finisar and Intel, is to develop a technology roadmap and common specifications for on-board optics to ensure interoperability.
Brad Booth (pictured), the chair of COBO and principal architect for Microsoft’s Azure Global Networking Services, says that bringing optics inside systems raises a different set of issues compared to pluggable optical modules used on the front panel of equipment. “If you have a requirement for 32 ports on a faceplate, you know mechanically what you can build,” says Booth.
With on-board optics, the focus is less about size considerations and more about the optical design itself and what is needed to make it work. There is also more scope to future-proof the design, something that can not be done so much with pluggable optics, says Booth.
COBO is working on a 400-gigabit optical module based on the 8-by–50 gigabit interface. The focus in recent months has been on defining the electrical connector that will be needed. The group has narrowed down the choice of candidates to two and the final selection will be based on the connector's signal integrity performance and manufacturability. Also being addressed is how two such modules could be placed side-by-side to create an 800-gigabit (16-by–50 gigabit) design.
COBO’s 400-gigabit on-board optics will support multi-mode and single-mode fibre variants. “When we do a comparison with what the pluggable people are pushing, there are a lot of pluggables that won’t be able to handle the power envelope,” says Booth.
There is no revolutionary change that goes on with technology, it all has to be evolutionary
On-board optics differs from a pluggable module in that the optics and electronics are not confined within a mechanical enclosure and therefore power dissipation is less of an design issue. But by supporting different fibre requirements and reaches new design issues arise. For example, when building a 16-by–50 gigabit design, the footprint is doubled and COBO is looking to eliminate the gap between the two such that a module can be plugged in that is either 8- or 16-lanes wide.
COBO is also being approached about supporting other requirements such as coherent optics for long-distance transmission. A Coherent Working Group has been formed and will meet for the first time in December in Santa Barbara, California. Using on-board optics for coherent avoids the power constraint issues associated with using a caged pluggable module.
On-board optics versus co-packaging
On-board optics is seen as the next step in the evolution of optics as it moves from the faceplate onto the board, closer to the ASIC. There is only so many modules that can fit on a faceplate. The power consumption also raises as the data rate of a pluggable modules increases, as does the power associated with driving faster electrical traces across the board.
Using on-board optics shortens the trace lengths by placing the optics closer to the chip. The board input-output capacity that can be supported also increases as it is fibres not pluggable optics that reside on the front panel. Ultimately, however, designers are already exploring the combining of optics and the chip using a system-in-package design, also known as 2.5D or 3D chip packaging.
Booth says discussions have already taken place between COBO members about co-packaged optics. But he does not expect system vendors to stay with pluggable optics and migrate directly to co-packaging thereby ignoring the on-board optics stage.
“There is no revolutionary change that goes on with technology, it all has to be evolutionary,” says Booth, who sees on-board optics as the next needed transition after pluggables. “You have to have some pathway to learn and discover, and figure out the pain points,” he says. “We are going to learn a lot when we start the deployment of COBO-based modules.”
Booth also sees on-board optics as the next step in terms of flexibility.
When pluggable modules were first introduced they were promoted as allowing switch vendors to support different fibre and copper interfaces on their platforms. The requirements of the cloud providers has changed that broad thinking, he says: “We don’t need that same level of flexibility but there is still a need for suporting different styles of optical interfaces on a switch.”
There are not a lot of other modules that can do 600 gigabit but guess what? COBO can
For example, one data centre operator may favour a parallel fibre solution based on the 100-gigabit PSM4 module while another may want a 100-gigabit wavelength-division multiplexing (WDM) solution and use the CWDM4 module. “This [parallel lane versus WDM] is something embedded optics can cater for,” says Booth.
Moving to a co-packaged design offers no such flexibility. What can a data centre manager do when deciding to change from parallel single-mode optics to wavelength-division multiplexing when the optics is already co-packaged with the chip? “Also how do I deal with an optics failure? Do I have to replace the whole switch silicon?” says Booth. We may be getting to the point where we can embed optics with silicon but what is needed is a lot more work, a lot more consideration and a lot more time, says Booth.
Status
COBO members are busy working on the 400-gigabit embedded module, and by extension the 800-gigabit design. There is also ongoing work as to how to support technologies such as the OIF’s FlexEthernet. Coherent designs will soon support rates such as 600-gigabit using a symbol rate of 64 gigabaud and advanced modulation. “There are not a lot of other modules that can do 600 gigabits but guess what? COBO can,” says Booth.
The good thing is that whether it is coherent, Ethernet or other technologies, all the members are sitting in the same room, says Booth: “It doesn’t matter which market gets there first, we are going to have to figure it out.”
Story updated on October 27th regarding the connector selection and the Coherent Working Group.
Heterogeneous integration comes of age
Silicon photonics luminaries series
Interview 7: Professor John Bowers
August has been a notable month for John Bowers.
Juniper Networks announced its intention to acquire Aurrion, the US silicon photonics start-up that Bowers co-founded with Alexander Fang. And Intel, a company Bowers worked with on a hybrid integration laser-bonding technique, unveiled its first 100-gigabit silicon photonics transceivers.
Professor John BowersBower, a professor in the Department of Electrical and Computer Engineering at the University of California, Santa Barbara (UCSB), first started working in photonics in 1981 while at AT&T Bell Labs.
When he became interested in silicon photonics, it still lacked a good modulator and laser. "If you don't have a laser and a modulator, or a directly modulated laser, it is not a very interesting chip,” says Bowers. "So I started thinking how to do that."
Bowers contacted Mario Paniccia, who headed Intel’s silicon photonics programme at the time, and said: “What if we can integrate a laser? I think there is a good way to do it.” The resulting approach, known as heterogeneous integration, is one that both Intel and Aurrion embraced and since developed.
This is a key Bowers trait, says Aurrion co-founder, Fang: he just knows what problems to work on.
"John came up with the concept of the hybrid laser very early on," says Fang. "Recall that, at that time, silicon photonics was viewed as nothing more than people making plasma-effect phase shifters and simple passive devices. John just cut to the chase and went after combining III-V materials with silicon."
If you look at the major companies with strong photonics activities, you’ll find a leader in that group that was developed under John’s training
Fang also highlights Bowers' management skills. “John can pick players and run teams,” says Fang, who describes himself as one of those privileged to graduate out of Bowers’ research group at UCSB.
“You find yourself in an environment where John picks a team of sharp folk with complementary skills and domain expertise to solve a problem that John determines as important and has some insight on how to solve it,” says Fang. “If we look like we are going to drive off the road, he nudges with a good mix of insight, fear, and humour.”
It has resulted in some of the best trained independent thinkers and leaders in the industry, says Fang: “If you look at the major companies with strong photonics activities, you’ll find a leader in that group that was developed under John’s training”.
Silicon photonics
Bowers defines silicon photonics as photonic devices on a silicon substrate fabricated in a CMOS facility.
“Silicon photonics is not about using silicon for everything; that misses the point,” says Bowers. “The key element is using silicon as a substrate - 12-inch wafers and not 2- or 3-inch wafers - and having all the process capability a modern silicon CMOS facility brings.” These capabilities include not just wafer processing but also advanced testing and packaging.
The world is about to change and I don't think people have quite figured that out
“If you go to an advanced packaging house, they don't do 6-inch wafers and I don't know of indium phosphide and gallium arsenide wafers larger than 6 inches,” says Bowers. “The only solution is to go to silicon; that is the revolution that hasn't happened yet but it is happening now.”
Bowers adds that everything Aurrion does, there is automated test along the way. "And I think you have others; Luxtera has done a great job as well at wafer-level test and packaging," he says. "The world is about to change and I don't think people have quite figured that out."
Working with Intel was an eye-opener for Bowers, especially the process controls it applies to chip-making.
“They worry about distributions and yields, and it is clear why there are seven billion transistors on a chip and that chip will yield,” says Bowers. “When you apply that to photonics, it will take it to a whole new level.” Indeed, Bowers foresees photonics transfering to silicon.
Bowers highlights the fairly complex chips now being developed using silicon photonics.
“We have done a 2D scanner - a 32-element phased array - something one could never do in optics unless it was integrated all on one chip,” he says. The phased-array chip comprises 160 elements and is physically quite large.
This is another benefit of using 12-inch silicon wafers and fabricating the circuits in a CMOS facility. “You are not going to cost-effectively do that in indium phosphide, which I've worked on for the last 30 years,” says Bowers.
Another complex device developed at UCSB is a 2.54-terabit network-on-a-chip. “This is a larger capacity than anyone has done on any substrate,” he says.
Infinera’s latest photonic integrated circuit (PIC), for example, has a transport capacity of up to 2.4 terabit-per-second. That said, Bowers stresses that the network-on-a-chip is a research presentation while Infinera’s PIC is a commercial device.
Heterogeneous integration
Heterogeneous integration involves bonding materials such as III-V compounds onto silicon.
Bowers first worked on III-V bonding with HP to make longer wavelength - 1310nm and 1550nm - VCSELs. “We had been bonding indium phosphide and gallium arsenide to solve a fundamental problem that indium phosphide does not make good mirrors,” he says. “So I was pretty confident we could bond III-V to silicon to add gain to silicon photonics to then add all the laser capability.”
Bonding to silicon is attractive as it enables the integration of optical features that haven't been widely integrated onto any other platform, says Bowers. These include not only lasers but other active devices such as modulators and photo-detectors, as well as passive functions such as isolators and circulators.
One concern raised about heterogeneous integration and the use of III-V materials is the risk of contamination of a CMOS fabrication line.
Bowers points out that the approached used does not impact the front end of the fabrication, where silicon wafers are etched and waveguides formed. The III-V material is bonded to the wafer at the fab’s back end, the stage where metallisation occurs when making a CMOS chip.
The leading chipmakers are also experimenting with III-V materials to create faster digital devices due to their higher electron mobility. “This is part of the natural evolution of CMOS,” he says. It remains unclear if this will be adopted, but it is possible that a 5nm CMOS node will use indium phosphide.
“All the CMOS houses are doing lots of work on III-V and silicon,” says Bowers. “They have figured out how to control that contamination issue.”
New capabilities
Bowers and his team have already demonstrated the integration of new optical functions on silicon.
“Neither silicon nor indium phosphide has an isolator and one always has to use an external YIG (yttrium iron garnet) isolator to reduce the reflection sensitivity of things like widely tunable lasers,” says Bower.
His team has developed a way to bond a YIG onto silicon using the same techniques it uses for bonding III-V materials. The result is an integrated isolator device with 32dB isolation and a 2dB insertion loss, a level of performance matching those of discrete isolators.
Incorporating such functionality onto silicon creates new possibilities. “We have a paper coming out that features a 6-port circulator,” says Bowers. “It is not a tool that the community can use yet because it has never been made before but we can do that on silicon now,” he says. “That is a good new capability.”
Superior performance
Bowers stresses that heterogeneous integration can also result in optical performance superior to a III-V design alone. He cites as an example how using a silicon nitride waveguide, with its lower loss that indium phosphide or gallium arsenide, can create high-quality Q-resonators.
A Q-resonator can be viewed as a form of filter. Bowers' group have demonstrated one with a Q of 80 million. “That makes it very sensitive to a variety of things,” he says. One example is for sensors, using a Q resonator with a laser and detector to form a spectrometer.
His researchers have also integrated the Q resonator with a laser to make a widely tunable device that has a very narrow line-width: some 40kHz wide. This is a narrower than the line-width of commercially-available tunable lasers and exceeds what can be done with indium phosphide alone, he says.
Challenges
Bowers, like other silicon photonics luminaries, highlights the issues of automated packaging and automated testing, as important challenges facing silicon photonics. “Taking 10,000s of transceivers and bringing all the advanced technology - not just processing but test and packaging - that are being developed for cell phones,” he says.
Too much of photonics today is based on gold boxes and expensive transceivers. “Where Aurrion and Intel are going is getting silicon photonics to the point where photonics will be ubiquitous, cheap and high yielding,” he says. This trend is even evident with his university work. The 400-element 2.54-terabit network-on-a-chip has very high laser yields, as are its passive yields, he says.
“So, effectively, what silicon photonics can do is going up very rapidly,“ says Bowers. “If you can put it in the hands of a real CMOS player like Intel or the companies that Aurrion uses, it is going to take photonics to a whole new area that people would not have thought possible in terms of complexity.”
Yet Bowers is also pragmatic. “It still takes time,” he says. “You can demonstrate an idea, but it takes time to make it viable commercially.”
He points to the recently announced switch from Oracle that uses mid-board optics. “That is a commercial product out there now,” he says. “But is it silicon photonics? No, it is VCSEL-based; that is the battle going on now.”
VCSELs have won the initial battle in the data centre but the amount of integration the technology can support is limited. Once designers move to wavelength-division multiplexing to get to higher capacities, where planar technology is required to combine and separate the different wavelengths efficiently, that is when silicon has an advantage, he says.
The battle at 100 gigabit between VCSELs and silicon photonics is also one that Bowers believes silicon photonics will eventually win. But at 400 gigabit and one terabit, there is no way to do that using VCSELs, he says.
Status
The real win for silicon photonics is when optics moves from transceivers at the edge of the board to mid-board and eventually are integrated with a chip in the same package, he says.
Advanced chips such as switch silicon for the data centre are running into an input-output problem. There are only so many 25 gigabit-per-second signals a chip can support. Each signal, sent down a trace on a printed circuit board, typically requires equalisation circuitry at each end and that consumes power.
Most of the photonics industry has focused on telecom and datacom, and justifiably so. The next big thing will happen in the area of sensors.
A large IC packaged as a ball grid array may have as many as 5,000 bumps (balls) that are interfaced to the printed circuit board. Using photonics can boost the overall bandwidth coming on and off the chip.
“With photonics, and in particular when we integrate the laser as well as the modulator, the world doesn't see it as a photonics chip, it's an electronics chip, it just turns out that some of those bumps are optical ones and they provide much more efficient transmission of data and at much lower power,” say Bowers. A 100 terabit of even a 1000 terabit - a petabit - switch chip then becomes possible. This is not possible electrically but it is possible by integrating photonics inside the package or on the chip itself, he says.
“That is the big win eventually and that is where we help electronics extend Moore’s law,” says Bowers.
And as silicon photonics matures, other applications will emerge - More than Moore’s law - like the use of photonics for sensors.
“Most of the photonics industry has focused on telecom and datacom, and justifiably so,” says Bowers. “The next big thing will happen in the area of sensors.”
Professor Bowers was interviewed before the Juniper Networks announcement