CW-WDM MSA charts a parallel path for optics
Artificial intelligence (AI) and machine learning have become an integral part of the businesses of the webscale players.
The mega data centre players apply machine learning to the treasure trove of data collected from users to improve services and target advertising.
Chris Cole
They can also use their data centres to offer cloud-based AI services.
Training neural networks with data sets is so intensive that it is driving new processor and networking requirements.
It is also impacting optics. Optical interfaces will need to become faster to cope with the amount of data, and that means interfaces with more parallel channels.
Anticipating these trends, a group of companies has formed the Continuous-Wave Wavelength Division Multiplexing (CW-WDM) multi-source agreement (MSA).
The CW-WDM MSA will specify lasers sources and the wavelength grids they use. The lasers will operate in the O-band (1260nm-1360nm) used for datacom optics.
The MSA is defining eight, 16 and 32 channels and will build on work done by the ITU-T and the IEEE.
This is good news for the laser manufacturers, says Chris Cole, Chair of CW-WDM MSA (pictured), given they have already shipped millions of lasers for datacom.
“In general, lasers are typically the hardest thing,” he says.
Wavelength count
The majority of datacom pluggable modules deployed today use either one or four optical channels. “When I started in optics 20 years ago it was all about single wavelengths,” says Cole.
Four channels were first used successfully for 40-gigabit interfaces. “That is when we introduced coarse wavelength-division multiplexing (CWDM),” says Cole.
Four wavelengths are the standard approach for 100, 200 and 400-gigabit optical modules. Spreading data across four channels simplifies the design of the electrical and optical interfaces.
“But we are ready to move on because the ability to increase parallel channels - be it parallel fibres or wavelengths - is much greater than the ability to push speed,” says Cole. “If all we do is rely on a four-wavelength paradigm and we keep pushing speed, we will run into a brick wall.”
Integration
Adopting more parallel channels will have two consequences on the optics, says Cole.
One is that photonic integration will become the only practical way to build multi-channel designs. Eight-channel designs are possible using discrete components but it won’t be cost-competitive for designs of 16 or more channels.
“It has to be photonic integration because as you get to eight and later, 16 and 32 wavelengths, it is not supportable in a small size with conventional approaches,” says Cole.
The MSA favours silicon photonics integration but indium phosphide or polymer integration platforms could be used.
The MSA will also cause wavelengths to be packed far more closely than the 20nm used for CWDM. Techniques now exist that enable tighter wavelength spacings without needing dedicated cooling.
One approach is separating the laser from the silicon chip - a switch chip or processor - that generates a lot of heat. Here, light from the source is fed to the optics over a fibre such that temperature control is more straightforward because the laser and chip are separated.
Cole also highlights the athermal silicon photonics of Juniper Networks that controls wavelength drift on the grid without requiring a thermo-electric cooler. Juniper gained the technology with its Aurrion acquisition in 2016.
Specification work
“Using the O-band has a lot of advantages,” says Cole. “That is where all the datacom optics are.”
The optical loss in the O-band may be double that of the C-band but this is not an issue for datacom’s short spans.
The MSA is to define a technology roadmap rather than a specific product, says Cole. First-generation products will use eight wavelengths followed by 16- and then 32-wavelength designs. Sixty-four and even 128 channel counts will be specified once the technology is established.
“Initially we did [specify 64 and 128 channels] but we took it out,” says Cole. “We’ll know a lot more if we are successful over three generations; we’ll figure out what we need to do when we get to that point.”
The MSA is proposing two bands, one 18nm wide (1291nm-1309nm) and the other 36nm wide (1282nm-1318nm). Eight, 16 and 32 wavelengths are assigned across both bands.
“It’s smack in the middle of the CWDM4 grid which is the largest shipping laser grid ever, and it is smack on top of the LWDM4 grid [used by -LR4 modules] which is the next highest grid to ship in volume,” says Cole.
The MSA will also specify continuous-wave laser parameters such as the output power, spectral width, variation in power between the wavelengths, and allowable wavelength shift.
Members
Cole started work on the CW-WDM MSA in collaboration with Ayar Labs while he was still at II-IV. Now at Luminous Computing, Cole, along with MSA editor Matt Sysak of Ayar Labs, and associate editor Dave Lewis of Lumentum, are preparing the first MSA draft and have solicited comments from members as to what to include in the specifications.
The MSA has 11 promoter members: Arista, Ayar Labs, CST Global, imec, Intel, Lumentum, Luminous Computing, MACOM, Quintessent, Sumitomo Electric, and II-VI.
The MSA has created a new observer member status to get input from companies that otherwise would be put off joining an MSA due to the associated legal requirements.
“So we have an observer category that if someone is serious and they want to see a subset of the material the MSA is working on and provide feedback, we welcome that,” says Cole.
The observer members are AMF, Axalume, Broadcom, Coherent Solutions, Furukawa Electric, GlobalFoundries, Keysight Technologies, NeoPhotonics, NVIDIA, Samtec, Scintil Photonics, and Tektronix.
“This MSA is meant to be inclusive, and it is meant to foster innovation and foster as broad an industry contribution as possible,” concludes Cole.
Further information
The CW-WDM MSA has several documents and technical papers on its website. The first document is the CW-WDM MSA grid proposal while the rest are technical papers addressing developments and applications driving the need for high-channel-count optical interfaces.
Companies gear up to make 800 Gig modules a reality
Nine companies have established a multi-source agreement (MSA) to develop optical specifications for 800-gigabit pluggable modules.
The MSA has been created to address the continual demand for more networking capacity in the data centre, a need that is doubling roughly every two years. The largest switch chips deployed have a 12.8 terabit-per-second (Tbps) switching capacity while 25.6-terabit and 51-terabit chips are in development.
“The MSA members believe that for 25.6Tbps and 51.2Tbps switching silicon, 800-gigabit interconnects are required to deliver the required footprint and density,” says Maxim Kuschnerov, a spokesperson for the 800G Pluggable MSA.
A 1-rack-unit (1RU) 25.6-terabit switch platform will use 32, 800-gigabit modules while a 51.2-terabit 2RU platform will require 64.
The MSA has been founded now to ensure that there will be optical and electrical components for 800-gigabit modules...
Motivation
The founding members of the 800G MSA are Accelink, China Telecommunication Technology Labs, H3C, Hisense Broadband, Huawei Technology, Luxshare, Sumitomo Electric Industries, Tencent, and Yamaichi Electronics. Baidu, Inphi and Lumentum have since joined the MSA.
The MSA has been founded now to ensure that there will be optical and electrical components for 800-gigabit modules when 51.2-terabit platforms arrive in 2022.
And an 800-gigabit module will be needed rather than a dual 400-gigabit design since the latter will not be economical.
“Historically, the cost of optical short-reach interfaces has always scaled with laser count,” says Kuschnerov. “Pluggables with 8, 10 or 16 lasers have never been successful in the long run.”
He cites such examples as the first 100-gigabit module implemented using 10×10-gigabit channels, and the early wide-channel 400 Gigabit Ethernet designs such as the SR16 parallel fibre and the FR8 specifications. The yield for optics doesn’t scale in the same way as CMOS for parallel designs, he says.
That said, the MSA will investigate several designs for the different reaches. For 100m, 8-channel and 4-channel parallel fibre designs will be explored while for the longer reaches, single-fibre coarse wavelength division multiplexing (CWDM) technology will be used.
Shown from left to right are a PSM8 and a PSM4 module for 100m spans, and the CWDM4 design for 500m and 2km reaches. Source: 800G Pluggable MSA.
“Right now, we are discussing several technical options, so there’s no conclusion as to which design is best for which reach class,” says Kuschnerov.
The move to fewer channels is similar to how 400 Gigabit Ethernet modules have evolved: the 8-channel FR8 and LR8 module designs address early applications but, as demand ramp, they have made way for more economical four-channel FR4 and LR4 designs.
Specification work
The MSA will focus on several optical designs for the 800G Pluggable MSA, all using 112Gbps electrical input signals.
The first MSA design, for applications up to 100m, will explore 8×100-gigabit optical channels as a fast-to-market solution. This is a parallel single-mode 8-channel (PSM8) design, with each 100-gigabit channel carried over a dedicated fibre. The module will use 16 fibres overall: eight for input and eight for output. The MSA will also explore a PSM4 design – ‘the real 800G’ – where each fibre carries 200 gigabits.
The CWDM designs, for 500m and 2km, will require a digital signal processor (DSP) to implement four-level pulse-amplitude modulation (PAM4) signalling that generates the 200-gigabit channels. An optical multiplexer and demultiplexer will also be needed for the two designs.
The MSA will explore the best technologies for each of the three spans. The modulation technologies to be investigated include silicon photonics, directly modulated lasers (DML) and externally modulated lasers (EML).
Challenges
The MSA foresees several technical challenges at 800 gigabits.
One challenge is developing 100-gigabaud direct-detect optics needed to generate the four 200 gigabit channels using PAM4. Another is fitting the designs into a QSFP-DD or OSFP pluggable module while meeting their specified power consumption limitations. A third challenge is choosing a low-power forward error correction scheme and a PAM4 digital signal processor (DSP) that meet the MSA’s performance and latency requirements.
“We expect first conclusions in the fourth quarter of 2020 with the publication of the first specification,” says Kuschnerov.
The 800G Pluggable MSA is also following industry developments such as the IEEE proposal for the 8×100-gigabit SR8 over multi-mode fibre that uses VCSELs. But the MSA believes VCSELs represent a higher risk.
“Our biggest challenge is creating sufficient momentum for the 800-gigabit ecosystem, and getting key industry contributors involved in our activity,” says Kuschnerov.
Arista Networks, the switch vendor that has long promoted 800-gigabit modules, says it has no immediate plans to join the MSA.
“But as one of the supporters of the OSFP MSA, we are aligned in the need to develop an ecosystem of technology suppliers for components and test equipment for OSFP pluggable optics at 800 gigabits,” says Martin Hull, Arista’s associate vice president, systems engineering and platforms.
Hull points out that the OSFP pluggable module MSA was specified with 800 gigabits in mind.
Next-generation Ethernet
The fact that there is no 800 Gigabit Ethernet standard will not hamper the work, and the MSA cannot wait for the development of such a standard.
“The IEEE is in the bandwidth assessment stage for beyond 400-gigabit rates and we haven’t seen too many contributions,” says Kuschnerov. The IEEE would then need to start a Call For Interest and define an 800GbE Study Group to evaluate the technical feasibility of 800GbE. Only then will an 800GbE Task Force Phase start. “We don’t expect the work on 800GbE in IEEE to progress in line with our target for component sampling,” says Kuschnerov. First prototype 800G MSA modules are expected in the fourth quarter of 2021.
Arista’s Hull stresses that an 800GbE standard is not needed given that 800-gigabit modules support standardised rates based on 2×400-gigabit and 8×100-gigabit.
Moreover, speed increments for Ethernet are typically more than 2x. “That would suggest an expectation for 1 Terabit Ethernet (TbE) or 1.6TbE speeds,” says Hull. This was the case with the bandwidth transition from 10GbE to 40GbE (4x), and 40GbE to 100GbE (2.5x).
“It would be unusual for Ethernet’s evolution to slow to a 2x rate and make 800 Gigabit Ethernet the next step,” says Hull. “The introduction of 112Gbps serdes allows for a doubling of input-output (I/O) on a per-physical interface but this is not the next Ethernet speed.”
Pluggable versus co-packaged optics
There is an ongoing industry debate as to when switch vendors will be forced to transition from pluggable optics on the front panel to photonics co-packaged with the switch ASIC.
The issue is that with each doubling of switch chip speed, it becomes harder to get the data on and off the chip at a reasonable cost and power consumption. Driving the ever faster signals from the chip to the front-panel optics is also becoming challenging.
Packaging the optics with the switch chip enables the high-speed serialiser-deserialiser (serdes), the circuitry that gets data on and off the chip, to be simplified; no longer will the serdes need to drive high-speed signals across the printed circuit board (PCB) to the front panel. Adopting co-packaged optics simplifies the PCB design, constrains the switch chip’s overall power consumption given how hundreds of serdes are used, and reduces the die area reserved for the serdes.
But transitioning to co-packaged optics represents a significant industry shift.
The consensus at a panel discussion at the OFC show, held in March, entitled Beyond 400G for Hyperscaler Data Centres, was that the use of front-panel pluggable optics will continue for at least two more generations of switch chips: at 25.6Tbps and at 51.2Tbps.
It is a view shared by the 800G Pluggable MSA and one of its motivating goals.
“The MSA believes that 800-gigabit pluggables are technically feasible and offer clear benefits versus co-packaging,” says Kuschnerov. “As long as the industry can support pluggables, this will be the preferred choice of the data centre operators.”
It has always paid off to bet on the established technology as long as it is technically feasible due to the sheer amount of investment already made, says Kuschnerov.
Major shifts in interconnects such as coherent replacing direct detect, or PSM/ CWDM pushing out VCSELs, or optics replacing copper have happened only when legacy technologies approach their limits and which can’t be overcome easily, he says: “We don’t believe in such fundamental limitations for 800-gigabit pluggables.”
So when will the industry adopt co-packaged optics?
“We believe that beyond 51.2Tbps there is a very high risk surrounding the serdes and thus co-packaging might become necessary to overcome this limitation,” says Kuschnerov.
Switch-chip-maker, Broadcom, has said that co-packaged optics will be adopted alongside pluggables, enabling the hyperscalers to lessen the risk of the new technology’s introduction. Broadcom believes that co-packaged optics solutions will appear as early as the advent of 25.6-terabit switch chips.
An earlier transitional introduction is also a view shared by Hugo Saleh, vice president of marketing and business development at silicon photonics specialist at Ayar Labs, which recently unveiled its optical I/O chiplet technology is being co-packaged with Intel’s Stratix 10 FPGA.
Saleh says the consensus is that the node past 51.2Tbps must use in-packaged optics. But he also expects overlap before then, especially for high-end and custom solutions.
“It [co-packaged optics] is definitely coming, and it is coming sooner than some folks expect,” says Saleh.
Several companies have contacted the MSA since its 800-gigabit announcement. The 800G MSA is also in discussion with several component and module vendors that are about to join, from Asia and elsewhere. Inphi and Lumentum have joined since the MSA was announced.
Discussions have started with system vendors and hyperscale data center operators; Baidu is one that has since signed up.
Ayar Labs and Intel add optical input-output to an FPGA
Start-up Ayar Labs, working with Intel, has interfaced its TeraPHY optical chiplet to the chip giant’s Stratix10 FPGA.
Hugo SalehIntel has teamed with several partners in addition to Ayar Labs for its FPGA-based silicon-in-package design, part of the US Defense Advanced Research Projects Agency’s (DARPA) project.
Ayar Labs used the Hot Chips conference, held in Palo Alto, California in August, to detail its first TeraPHY chiplet product and its interface to the high-end FPGA.
Origins
Ayar Labs was established to commercialise research that originated at MIT. The MIT team worked on integrating both photonics and electronics on a single die without changing the CMOS process.
The start-up has developed such building-block optical components in CMOS as a vertical coupler grating and a micro-ring resonator for modulation, while the electronic circuitry can be used to control and stabilise the ring resonator’s operation.
Ayar Labs has also developed an external laser source that provides an external light source that can power up to 256 optical channels, each operating at either 16, 25 or 32 gigabits-per-second (Gbps).
The company has two strategic investors: Intel Capital, the investment arm of Intel, and semiconductor firm GlobalFoundries.
The start-up received $24 million in funding late last year and has used the funding to open a second office in Santa Clara, California, and double its staff to about 40.
Markets
Ayar Labs has identified four markets for its silicon photonics technology.
The first is the military, aerospace and government market segment. Indeed, the Intel FPGA system-in-package is for a phased-array radar application.
Two further markets are high-performance computing and artificial intelligence, and telecommunications and the cloud.
Computer vision and advanced driver assisted systems is the fourth market segment. Here, the start-up’s expertise in silicon photonics is not for optical I/O but a sensor for LIDAR, says Hugo Saleh, Ayar Labs’ vice president of marketing and business development.
Stratix 10 system-in-package
The Intel phased-array radar system-in-package is designed to takes in huge amounts of RF data that is down-converted and digitised using an RF chiplet. The data is then pre-processed on the FPGA and sent optically using Ayar Labs’ TeraPHY chiplets for further processing in the cloud.
“To digitise all that information you need multiple TeraPHY chiplets per FPGA to pull the information back into the cloud,” says Saleh. A single phased-array radar can use as many as 50,000 FPGAs.
Such a radar design can be applied to civilian and to military applications where it can track 10,000s of objects.
Moreover, it is not just FPGAs that the TeraPHY chiplet can be interfaced to.
Large aerospace companies developing flight control systems also develop their own ASICs. “Almost every single aerospace company we have talked to as part of our collaboration with Intel has said they have custom ASICs,” says Saleh. “They want to know how they can procure, package and test the chiplets and bring them to market.”
It is one thing to integrate a chiplet but photonics is tricky
TeraPHY chiplet
Two Intel-developed technologies are used to interface the TeraPHY chiplet to the Stratix 10 FPGA.
The first is Intel’s Advanced Interface Bus (AIB), a parallel electrical interface technology. The second is the Embedded Multi-die Interconnect Bridge (EMIB) which supports the dense I/O needed to interface the main chip, in this case, the FPGA to a chiplet.
EMIB is a sliver of silicon designed to support I/O. The EMIBs are embedded in an organic substrate on which the dies sit; one is for each chiplet-FPGA interface. The EMIB supports various bump pitches to enable dense I/O connections.
Ayar Labs’ first TeraPHY product uses 24 AIB cells for its electrical interface. Each cell supports 20 channels, each operating at 2Gbps. The result is that each cell supports 40Gbps and the overall electrical bandwidth of the chiplet is 960 gigabits.
The TeraPHY’s optical interface uses 10 transmitter-receiver pairs, each pair supporting 8 optical channels that can operate at 16Gbps, 25Gbps or 32Gbps. The result is that the TeraPHY support a total optical bandwidth ranging from 1.28Tbps to 2.56Tbps.
The optical bandwidth is deliberately higher than the electrical bandwidth, says Saleh: “Just because you have ten [transmit/ receive] macros on the die doesn’t mean you have to use all ten.”
Also, the chiplet supports a crossbar switch that allows one-to-many connections such that an electrical channel can be sent out on more than one optical interface and vice versa.
For the Intel FPGA system-in-package, two TeraPHY chiplets are used, each supporting 16Gbps channels such that the chiplet’s total optical I/O is up to 5.12 terabits.
Ramifications
Saleh stresses the achievement in integrating optics in-package: “It is one thing to integrate a chiplet but photonics is tricky.”
Ayar Labs flip-chips its silicon and etches on the backside. “Besides all the hard work that goes into figuring how to do that, and keeping it hermetically sealed, you still have to escape light,” he says. “Escaping light out of the package that is intended to be high volume requires significant engineering work.” This required working very closely with Intel’s packaging department.
Now the challenge is to take the demonstrator chip to volume manufacturing.
Saleh also points to a more fundamental change that will need to take place with the advent of chip designs using optical I/O.
Over many years compute power in the form of advanced microprocessors that incorporate ever more CPU cores has doubled every two years or so. In contrast, I/O has advanced at a much slower pace – 5 or 10 per cent annually.
This has resulted in application software for high-performance computing being written to take this BW-compute disparity into account, reducing the number of memory accesses and minimising I/O transactions.
“Software now has to be architected to take advantage of all this new performance and all this new bandwidth,” he says. “We are going to see tremendous gains in performance because of it.”
Ayar Labs says it is on schedule to deliver its first TeraPHY chiplet product in volume to lead customers by the second half of 2020.
Books in 2018
Gazettabyte has asked various industry executives to discuss the books they have read in 2018. Here, Valery Tolstikhin and Alexandra Wright-Gladstein give their recommendations.
Valery Tolstikhin, president and CEO of Intengent, a consultancy
I read too many technical and business texts during the day so I leave my bedtime for more human reading.
Valery Tolstikhin
This year I wasn’t too lucky with fiction books but I did read some great non-fiction ones: Homo Deus: A Brief History of Tomorrow by Yuval Noah Harari, Jordan Peterson’s 12 Rules for Life: An Antidote to Chaos and Leonardo Da Vinci by Walter Isaacson.
All three titles are bestsellers and do not need an introduction, but still.
Harari’s book - the second in the series, and there is a third already published - encourages you to think of the big issues by detaching yourself from everyday routines and trivia.
Peterson’s book is about how to make yourself comfortable with these very routines and trivia while remaining at peace with the big issues. The book is also music to the ears of conservatives.
Isaacson’s book is as much about Leonardo da Vinci as it is about human’s aspiration for harmony, which extends from the arts to physics theories to iPhone design.
I highly recommend all three.
Alexandra Wright-Gladstein, co-founder of Ayar Labs
I'd recommend Measure What Matters: How Google, Bono and the Gates Foundation Rock the World with OKRs, by John Doerr, and Daring to Drive: A Saudi Woman's Awakening by Manal al-Sharif.
Alexandra Wright-Gladstein
Measure what Matters is a great overview of how several of the top companies of our time use the management method known as OKRs (objectives and key results), first developed by Andy Grove of Intel, to motivate large teams to accomplish impressive goals.
John Doerr learned the method early in his career while at Intel. Then, when he became a VC investor, he started teaching the method to the companies he invested in, including Google.
It is great that the method is now available for the rest of us.
Daring to Drive is just a wonderful story, a page-turner I could not put down. It is the autobiography of a woman who was raised in a conservative part of Saudi Arabia, who eventually revolted by driving a car (an illegal act for women in Saudi Arabia) and putting a video of her doing so on YouTube.
The book came out last year. This year we felt the impact of her life's work and the book when the Saudi government legalised driving for women - an incredible win for Manal and her community.
Ayar Labs advances I/O and pens GlobalFoundries deal
Silicon photonics start-up, Ayar Labs, has entered into a strategic agreement with semiconductor foundry, GlobalFoundries.
Alexandra Wright-GladsteinAyar Labs will provide GlobalFoundries with its optical input-output (I/O) technology. In return, the start-up will gain early access to the foundry’s 45nm CMOS process being tailored for silicon photonics.
GlobalFoundries has also made an investment in the start-up for an undisclosed fee.
“We gain, first and foremost, a close relationship with GlobalFoundries as we qualify our product for customers,” says Alexandra Wright-Gladstein, co-founder and CEO of Ayar Labs. “That will help us speed up availability of our product and have their weight of support behind us.”
Strategy
Ayar Labs is bringing to market technology developed by academics originally at MIT. The research group developed a way to manufacture silicon photonics components using a standard silicon-on-insulator (SOI) CMOS process. The research work resulted in a novel dual-core RISC-V microprocessor demonstrator that used optical I/O to send and receive data, work that was published in the Nature science journal in December 2015.
Ayar Labs is using its optical I/O technology to address the high-performance computing and data centre markets. The optical I/O reaches up to 2km, from chip-to-chip communications to linking equipment between the buildings of a large data centre.
The start-up will offer a die - chiplet - that can be integrated within a multi-chip module, as well as a high-capacity 3.2-terabit optical module.
“We are aggregating the capacity of 4, 8 or 16 pluggable transceivers into a single module to share the cost of production at such high data rates,” says Wright-Gladstein. “This makes us competitive [for applications] where a pluggable transceiver is not.” Offering a chiplet and a high-density optical module on a board will bring to the marketplace the benefits companies are looking for if they are to move from copper to optics, she says.
Ayar Labs will also license its technology. “Our goal is to create an ecosystem for optical I/O for chips,” says Wright-Gladstein.
Technology
Ayar Labs has been a customer of GlobalFoundries for several years, using its existing 45nm SOI CMOS process to make devices as part of the foundry’s multi-project wafer service. The start-up will use the same 45nm CMOS process to make its first product. The CEO points out that using an unmodified electronics process introduces tight design constraints; no new materials can be introduced or layer thicknesses modified.
The start-up will also support GlobalFoundries in the development of its 45nm CMOS process optimised for silicon photonics. “The new process is more geared to traditional applications of optics such as optical transceivers for longer-distance communications,” says Wright-Gladstein.
Our goal is to create an ecosystem for optical I/O for chips
The intellectual property of Ayar Labs includes a micro-ring resonator optical modulator that is tiny compared to a Mach-Zehnder modulator. An issue with a micro-ring resonator is its sensitivity to temperature and manufacturing variances. Ayar’s Labs ability to design the ring resonator using standard CMOS means control circuitry can be added to ensure the modulator’s stability.
Ayar Labs has advanced its technology since the publication of the 2015 Nature paper. It has changed the operating wavelength of its optics from 1180nm to the standard 1310nm. It has also increased the speed of optical transmission from 2.5 to 25 gigabits-per-second (Gbps). The start-up expects to be able to extend the data rate to 50Gbps and even 100Gbps using 4-level pulse-amplitude modulation (PAM-4). The company has already demonstrated PAM-4 technology working with its optics.
The company also has wavelength-division multiplexing technology, using 8 wavelengths on a fibre; the original microprocessor demonstrator used only one wavelength. “We have 8 [micro-resonator] rings that lock on the transmit side and 8 rings that lock on the receive side,” says Wright-Gladstein. The company expects to extend the number of working wavelengths to 16 and even 32.
“We believe this is the process of the future because it can scale,” she says.
A factor of 10
Wright-Gladstein says its technology delivers a tenfold improvement using several metrics when compared to copper interconnect.
Typically a 25Gbps electrical interface will occupy 1 mm2 of chip area whereas Ayar Labs can fit more - potentially much more - than 250Gbps. The use of WDM technology also means that the amount of data passing the chip’s edge is at least 10 times greater.
The energy efficiency for the I/O is also between 5 times and 20 times greater than copper
The latency - how long it takes a signal to arrive at the receiver from the transmitter - is also improved tenfold. The fastest electrical interfaces at 56Gbps that use PAM-4 require forward-error correction which adds 100ns to the latency. Sending light 3m between racks takes 10ns, a tenth of the time. And more wavelengths can be added rather than using PAM-4 to avoid adversely impacting latency. “That matters for HPC customers,” she says.
The energy efficiency for the I/O is also between 5 times and 20 times greater than copper.
Ayar Labs has also developed an integrated laser module that provides the light sources for its optical I/O. Multiple lasers are integrated on a single die and the module outputs several wavelengths of light on several fibres.
The start-up claims the overall optical I/O design is simplified as there is no attachment of laser dies to the silicon and there are no attached driver chips. The result is a die that is flip-chip-attached allowing the use of standard high-volume CMOS packaging techniques.
First samples are expected sometime this year, with general product availability starting in 2019.
Meanwhile, GlobalFoundries is expected to offer the optical I/O as part of its 45nm silicon photonics process library in 2019.
Reflections on OFC 2017
Mood, technologies, notable announcements - just what are the metrics to judge the OFC 2017 show held in Los Angeles last week?
It was the first show I had attended in several years and the most obvious changes were how natural the presence of the internet content providers now is alongside the telecom operators, as well as systems vendors exhibiting at the show. Chip companies, while also present, were fewer than before.
Source: OSA
Another impression were the latest buzz terms: 5G, the Internet of Things and virtual reality-augmented reality. Certain of these technologies are more concrete than others, but their repeated mention suggests a consensus that the topics are real enough to impact optical components and networking.
It could be argued that OFC 2017 was the year when 400 Gigabit Ethernet became a reality
The importance of 5G needs no explanation while the more diffuse IoT is expected to drive networking with the huge amounts of data it will generate. But what are people seeing about virtual reality-augmented reality that merits inclusion alongside 5G and IoT?
Another change is the spread of data rates. No longer does one rate represent the theme of an OFC such as 40 Gigabits or 100 Gigabits. It could be argued that OFC 2017 was the year when 400 Gigabit Ethernet became a reality but there is now a mix of relevant rates such as 25, 50, 200 and 600 gigabits.
Highlights
There were several highlights at the show. One was listening to Jiajin Gao, deputy general manager at China Mobile Technology, open the OIDA Executive Forum event by discussing the changes taking place in the operator's network. Gao started by outlining the history of China Mobile's network before detailing the huge growth in ports at different points in the network over the last two years. He then outlined China Mobile's ambitious rollout of new technologies this year and next.
China's main three operators have 4G and FTTx subscriber numbers that dwarf the rest of the world. Will 2017 eventually be seen as the year when the Chinese operators first became leaders in telecom networking and technologies?
The Executive Forum concluded with an interesting fireside discussion about whether the current optical market growth is sustainable. The consensus among representatives from Huawei, Hisense, Oclaro and Macom was that it is; that the market is more varied and stable this time compared to the boom and bust of 1999-2001. As Macom’s Preetinder Virk put it: "The future has nothing to do with the past". Meanwhile, Huawei’s Jeffrey Gao still expects strong demand in China for 100 gigabits in 2017 even if growth is less strong than in 2016. He also expects the second quarter this year to pick up compared to a relatively weak first quarter.
OFC 2017 also made the news with an announcement that signals industry change: Ciena's decision to share its WaveLogic Ai coherent DSP technology with optical module vendors Lumentum, Oclaro and NeoPhotonics.
The announcement can be viewed several ways. One is that the initiative is a response to the success of Acacia as a supplier of coherent modules and coherent DSP technology. System vendors designed their own coherent DSP-ASICs to differentiate their optical networking gear. This still holds true but the deal reflects how the progress of merchant line-side optics from the likes of Acacia is progressing and squeezing the scope for differentiation.
The deal is also a smart strategic move by Ciena which, through its optical module partners, will address new markets and generate revenues as its partners start to sell modules using the WaveLogic Ai. The deal also has a first-mover advantage. Other systems vendors may now decide to offer their coherent DSPs to the marketplace but Ciena has partnerships with three leading optical module makers and is working with them on future DSP developments for pluggable modules.
The deal also raises wider questions as to the role of differentiated hardware and whether it is subtly changing in the era of network function virtualisation, or whether it is a reflection of the way companies are now collaborating with each other in open hardware developments like the Telecom Infra Project and the Open ROADM MSA.
Another prominent issue at the show is the debate as to whether there is room for 200 Gigabit Ethernet modules or whether the industry is best served by going straight from 100 to 400 Gigabit Ethernet.
Facebook and Microsoft say they will go straight to 400 gigabits. Cisco agrees, arguing that developing an interim 200 Gigabit Ethernet interface does not justify the investment. In contrast, Finisar argues that 200 Gigabit Ethernet has a compelling cost-per-bit performance and that it will supply customers that want it. Google supported 200 gigabits at last year’s OFC.
Silicon photonics
Silicon photonics was one topic of interest at the show and in particular how the technology continues to evolve. Based on the evidence at OFC, silicon photonics continues to progress but there were no significant developments since our book (co-written with Daryl Inniss) on silicon photonics was published late last year.
One of the pleasures of OFC is being briefed by key companies in rapid succession. Intel demonstrated at its booth its silicon photonics products including its CWDM4 module which will be generally available by mid-year. Intel also demonstrated a 10km 4WDM module. The 4WDM MSA, created last year, is developing a 10km reach variant based on the CWDM4, as well as 20km and 40km based designs.
Meanwhile, Ranovus announced its 200-gigabit CFP2 module based on its quantum dot laser and silicon photonics ring resonator technologies with a reach approaching 100km. The 200 gigabit is achieved using 28Gbaud optics and PAM-4.
Elenion Technologies made several announcements including the availability of its monolithically integrated coherent modulator receiver after detailing it was already supplying a 200 gigabit CFP2-ACO to Coriant. The company was also demonstrating on-board optics and, working with Cavium, announced a reference architecture to link network interface cards and switching ICs in the data centre.
I visited Elenion Technologies in a hotel suite adjacent to the conference centre. One of the rooms had enough test equipment and boards to resemble a lab; a lab with a breathtaking view of the hills around Los Angeles. As I arrived, one company was leaving and as I left another well-known company was arriving. Elenion was using the suite to demonstrate its technologies with meetings continuing long after the exhibition hall had closed.
Two other silicon photonics start-ups at the show were Ayar Labs and Rockley Photonics.
Ayar Labs in developing a silicon photonics chip based on a "zero touch" CMOS process that will sit right next to complex ASICs and interface to network interface cards. The first chip will support 3.2 terabits of capacity. The advantage of the CMOS-based silicon photonics design is the ability to operate at high temperatures.
Ayar Labs is using the technology to address the high-bandwidth, low-latency needs of the high-performance computing market, with the company expecting the technology to eventually be adopted in large-scale data centres.
Rockley Photonics shared more details as to what it is doing as well as its business model but it is still to unveil its first products.
The company has developed silicon photonics technology that will co-package optics alongside ASIC chips. The result will be packaged devices with fibre-based input-output offering terabit data rates.
Rockley also talked about licensing the technology for a range of applications involving complex ICs including coherent designs, not just for switching architectures in the data centre that it has discussed up till now. Rockley says its first product will be sampling in the coming months.
Looking ahead
On the plane back from OFC I was reading The Undoing Project by Michael Lewis about the psychologists Danny Kahneman and Amos Tversky and their insights into human thinking.
The book describes the tendency of people to take observed facts, neglecting the many facts that are missed or could not be seen, and make them fit a confident-sounding story. Or, as the late Amos Tversky put it: "All too often, we find ourselves unable to predict what will happen; yet after the fact, we explain what did happen with a great deal of confidence. This 'ability' to explain that which we cannot predict, even in the absence of any additional information, represents an important, though subtle, flaw in our reasoning."
So, what to expect at OFC 2018? More of the same and perhaps a bombshell or two. Or to put it another way, greater unpredictability based on the impression at OFC 2017 of an industry experiencing an increasing pace of change.
Silicon photonics adds off-chip comms to a RISC-V processor
"For the first time a system - a microprocessor - has been able to communicate with the external world using something other than electronics," says Vladimir Stojanovic, associate professor of electrical engineering and computer science at the University of California, Berkeley.
Vladimir Stojanovic
The microprocessor is the result of work that started at MIT nearly a decade ago as part of a project sponsored by the US Defense Advanced Research Projects Agency (DARPA) to investigate the integration of photonics and electronics for off-chip and even intra-chip communications.
The chip features a dual-core 1.65GHz RISC-V open instruction set processor and 1 megabyte of static RAM and integrates 70 million transistors and 850 optical components.
The work is also notable in that the optical components were developed without making any changes to an IBM 45nm CMOS process used to fabricate the processor. The researchers have demonstrated two of the processors communicating optically, with the RISC core on one chip reading and writing to the memory of the second device and executing programs such as image rendering.
This CMOS process approach to silicon photonics, dubbed 'zero-change' by the researchers, differs from that of the optical industry. So far silicon photonics players have customised CMOS processes to improve the optical components' performance. Many companies also develop the silicon photonics separately, using a trailing-edge 130nm or 90nm CMOS process while implementing the driver electronics on a separate chip using more advanced CMOS. That is because photonic devices such as a Mach-Zehnder modulator are relatively large and waste expensive silicon real-estate if implemented using a leading-edge process.
IBM is one player that has developed the electronics and optics on one chip using a 90nm CMOS process. However, the company says that the electronics use feature sizes closer to 65nm to achieve electrical speeds of 25 gigabit-per-second (Gbps), and being a custom process, it will only be possible to implement 50-gigabit rates using 4-level pulse amplitude modulation (PAM-4).
We are now reaping the benefits of this very precise process which others cannot do because they are operating at larger process nodes
"Our approach is that photonics is sort of like a second-class citizen to transistors but it is still good enough," says Stojanovic. This way, photonics can be part of an advanced CMOS process.
Pursuing a zero-change process was first met with skepticism and involved significant work by the researchers to develop. "People thought that making no changes to the process would be super-restrictive and lead to very poor [optical] device performance," says Stojanovic. Indeed, the first designs produced didn't work. "We didn't understand the IBM process and the masks enough, or it [the etching] would strip off certain stuff we'd put on to block certain steps."
But the team slowly mastered the process, making simple optical devices before moving on to more complex designs. Now the team believes its building-block components such as its vertical grating couplers have leading-edge performance while its ring-resonator modulator is close to matching the optical performance of designs using custom CMOS processes.
"We are now reaping the benefits of this very precise process which others cannot do because they are operating at larger process nodes," says Stojanovic.
Silicon photonics design
The researchers use a micro ring-resonator for its modulator design. The ring-resonator is much smaller than a Mach-Zehnder design and is 10 microns in diameter. Stojanovic says the dimensions of its vertical grating couplers are 10 to 20 microns while its silicon waveguides are 0.5 microns.
Photonic components are big relative to transistors, but for the links, it is the transistors that occupy more area than the photonics. "You can pack a lot of utilisation in a very small chip area," he says.
A key challenge with a micro ring-resonator is ensuring its stability. As the name implies, modulation of light occurs when the device is in resonance but this drifts with temperature, greatly impairing its performance.
Stojanovic cites how even the bit sequence can affect the modulator's temperature. "Given the microprocessor data is uncoded, you can have random bursts of zeros," he says. "When it [the modulator] drops the light, it self-heats: if it is modulating a [binary] zero it gets heated more than letting a one go through."
The researchers have had to develop circuitry that senses the bit-sequence pattern and counteracts the ring's self-heating. But the example also illustrates the advantage of combining photonics and electronics. "If you have a lot of transistors next to the modulator, it is much easier to tune it and make it work," says Stojanovic.
A prototype set-up of the chip-to-chip interconnect using silicon photonics. Source: Vladimir Stojanovic
Demonstration
The team used two microprocessors - one CPU talking to the memory of the second chip 4m away. Two chips were used rather than one - going off-chip before returning - to prove that the communication was indeed optical since there is also an internal electrical bus on-chip linking the CPU and memory. "We wanted to demonstrate chip-to-chip because that is where we think the biggest bang for the buck is," says Stojanovic.
In the demonstration, a single laser operating at 1,183nm feeds the two paths linking the memory and processor. Each link is 2.5Gbps for a total bandwidth of 5Gbps. However the microprocessor was clocked at one-eightieth of its 1.65GHz clock speed because only one wavelength was used to carry data. The microprocessor design can support 11 wavelengths for a total bandwidth of 55Gbit/s while the silicon photonics technology itself will support between 16 and 32 wavelengths overall.
The group is already lab-testing a new iteration of the chip that promises to run the processor at full speed. The latest chip also features improved optical functions. "It has better devices all over the place: better modulators, photo-detectors and gratings; it keeps evolving," says Stojanovic.
We can ship that kind of bandwidth [3.2 terabits] from a single chip
Ayar Labs
Ayar Labs is a start-up still in stealth mode that has been established to use the zero-change silicon photonics to make interconnect chips for platforms in the data centre.
Stojanovic says the microprocessor demonstrator is an example of a product that is two generations beyond existing pluggable modules. Ayar Labs will focus on on-board optics, what he describes as the next generation of product. On-board optics sit on a card, close to the chip. Optics integrated within the chip will eventually be needed, he says, but only once applications require greater bandwidth and denser interfaces.
"One of the nice things is that this technology is malleable; it can be put in various form factors to satisfy different connectivity applications," says Stojanovic.
What Ayar Labs aims to do is replace the QSFP pluggable modules on the face plate of a switch with one chip next to the switch silicon that can have a capacity of 3.2 terabits. "We can ship that kind of bandwidth from a single chip," says Stojanovic.
Such a chip promises cost reduction given how a large part of the cost in optical design is in the packaging. Here, packaging 32, 100 Gigabit Ethernet QSFP modules can be replaced with a single optical module using the chip. "That cost reduction is the key to enabling deeper penetration of photonics, and has been a barrier for silicon photonics [volumes] to ramp," says Stojanovic.
There is also the issue of how to couple the laser to the silicon photonics chip. Stojanovic says such high-bandwidth interface ICs require multiple lasers: "You definitely don't want hundreds of lasers flip-chipped on top [of the optical chip], you have to have a different approach".
Ayar Labs has not detailed what it is doing but Stojanovic says that its approach is more radical than simply sharing one laser across a few links, "Think about the laser as the power supply to the box, or maybe a few racks," he says.
The start-up is also exploring using standard polycrystalline silicon rather than the more specialist silicon-on-isolator wafers.
"Poly-silicon is much more lossy, so we have had to do special tricks in that process to make it less so," says Stojanovic. The result is that changes are needed to be made to the process; this will not be a zero-change process. But Stojanovic says the changes are few in number and relatively simple, and that it has already been shown to work.
Having such a process available would allow photonics to be added to transistors made using the most advanced CMOS processes - 16nm and even 7nm. "Then silicon-on-insulator becomes redundant; that is our end goal,” says Stojanovic.
Further information
Single-chip microprocessor that communicates directly using light, Nature, Volume 528, 24-31 December 2015