Webinar: Scaling AI clusters with optical interconnects
A reminder that this Thursday, September 14th, 8:00-9:00 am PT, I will be taking part in a webcast as part of the OCP Educational Webinar Programme that explores the future of AI computing with optical interconnects.
Data and computation drive AI success, and the hyperscaler are racing to build massive AI accelerator-based compute clusters. The impact of large language models and ChatGPT has turbocharged this race. Scaling demands innovation in accelerator chips, node linkages, fabrics, and topology.
For this webinar, industry experts will discuss the challenge of scaling AI clusters. The other speakers include Cliff Grossner Ph.D., Yang Chen, and Bob Shine. To register, please click here
Graphene prototype modulator shown working at 10 Gigabit
- Imec's graphene electro-absorption modulator works at 10 Gigabit-per-second
- The modulator is small and has be shown to be thermally stable
- Much work is required to develop the modulator commercially
Cross-section of the graphene electro-absorption modulator. The imec work was first detailed in a paper at the IEDM conference held in December 2014 in San Francisco. Source: imec
Imec has demonstrated an optical modulator using graphene operating at up to 10 Gigabit. The Belgium nano-electronics centre is exploring graphene - carbon atoms linked in a 2D sheet - as part of its silicon photonics research programme investigating next-generation optical interconnect. Chinese vendor Huawei joined imec's research programme late last year.
Several characteristics are sought for a modulator design. One is tiny dimensions to cram multiple interfaces in as tight a space as possible, as required for emerging board-to-board and chip-to-chip optical designs. Other desirable modulator characteristics include low power consumption, athermal operation, the ability to operate over a wide range of wavelengths, high speed (up to 50 Gbps) and ease of manufacture.
Imec's interest in graphene stems from the material's ability to change its light-absorbing characteristics over a wide spectral range. "Graphene has a high potential for a wide-band modulator solution and also for an athermal design," says Joris Van Campenhout, programme director for optical I/O at imec.
Source: Gazettabyte
Modulation
For optical modulation, either a material's absorption coefficient or its refractive index is used. Silicon photonics has already been used to implement Mach-Zehnder interferometer and ring resonator modulators. These designs modifying their refractive index and use interference to induce light intensity modulation.
"Mach-Zehnder modulators have been optimised dramatically over the last decade," says Van Campenhout. "They can generate at very high bit rates but they are still pretty big - 1mm or longer - and that prevents further scaling."
Ring resonators are more compact and have been shown working at up to 50 Gigabit. "But they are resonant devices; they are wavelength-specific and thermally dependent," says Van Campenhout. "A one degree change can detune the ring resonance from the laser's wavelength."
The other approach, an electro-absorption modulator, uses an electric field to vary the absorption coefficient of the material and this is the graphene modulator approach imec has chosen.
Electro-absorption modulators using silicon germanium meet the small footprint requirement, have a small capacitance and achieve broadband operation. Capacitance is an important metric as it defines the modulator's maximum data rate as well as such parameters as insertion loss (how many dBs of signal are lost passing through the modulator) and the extinction ratio (a measure of the modulator's on and off intensity).
"Silicon germanium offers a pretty decent modulation quality," says Van Campenhout but the wavelength drifts with temperature. Thermal drift is something that graphene appears to solve.
Imec's graphene electro-absorption modulator comprises a 50 micron graphene-oxide-silicon capacitor structure residing above a silicon-on-insulator rib waveguide. The waveguides are implemented using a 200mm wafer whereas the graphene is grown on a copper substrate before being placed on the silicon die. Van Campenhout refers to the design as hybrid or heterogenous silicon photonics.
The graphene modulator exhibits a low 4dB insertion loss and an extinction ratio of 2.5dB. The device's performance is stable over a broad spectrum: an 80nm window centred around the 1550nm wavelength. The performance of up to 10Gbps was achieved over a temperature range of 20-49°C.
"The key achievement is that we have been able to show that you can operate at 10 Gigabit with very clean modulation eye diagrams," says Van Campenhout. However, much work is needed before the device becomes a viable technology.
Source: Gazettabyte, imec
What next?
Imec has modelled the graphene modulator using a simple resistor-capacitor circuit. "We have been able to identify sources of capacitance and resistance," says Van Campenhout. "We can now better optimise the design for speed or for efficiency."
The speed of the modulator is dictated by the resistance-capacitance product. Yet the higher the capacitance, the greater the efficiency: the better the extinction ratio and the lower the insertion loss. "So it comes down to reducing the resistance," says Van Campenhout. "We think we should be able to get to 25 Gigabit."
With the first prototype, the absorption effect induced by the electric field is achieved between a single graphene plate and the silicon. Imec plans to develop a design using two graphene plates. "If two slabs of graphene are used, we expect to double the effect," says Van Campenhout. "All the charge on both plates of the capacitor will contribute to the modulation of the absorption."
However the integration is more difficult with two plates, and two metal contacts to graphene are needed. "This is still a challenge to do," says Van Campenhout.
Imec has also joined the Graphene Flagship, the European €1 billion programme that spans materials production, components and systems. "One of the work packages is to show you can process on a manufacturing scale graphene-based devices in a CMOS pilot line," he says. Another consideration is to use silicon nitride waveguides rather than silicon ones as these can be more easily deposited.
One challenge still to be overcome is the development of an efficient graphene-based photo-detector. "If this technology is ever going to be used in a real application, there should be a much more efficient graphene photo-detector being developed," says Van Campenhout.
Huawei joins imec to research silicon photonics
Huawei has joined imec, the Belgium nano-electronics research centre, to develop optical interconnect using silicon photonics technology. The strategic agreement follows Huawei's 2013 acquisition of former imec silicon photonics spin-off, Caliopa.
Source: Gazettabyte
“Having acquired cutting-edge expertise in the field of silicon photonics thanks to our acquisition of Caliopa last year, this partnership with imec is the logical next move towards next-generation optical communication,” says Hudson Liu, CEO at Huawei Belgium.
Imec's research focus is to develop technologies that are three to five years away from production. "Imec works with leading IC manufacturers and fabless companies in the field of CMOS fabrication," says Philippe Absil, department director for 3D and optical technologies at imec. "One of the programmes with our co-partners is about optical interconnect and silicon photonics, and Huawei is one of the participating companies."
Imec's research concentrates on board-to-board and chip-to-chip interconnect. The optical interconnect work includes increasing interface bandwidth density, reducing power consumption, and achieving thermal stability and system-cost reduction.
The research centre has demonstrated high-bandwidth interfaces as part of work with Chiral Photonics that makes multi-core fibre. Imec has developed a 2D ring of grating couplers that allow coupling between the silicon photonics chip and Chiral's 61-core fibre. "A grating coupler is a sub-wavelength structure that diffracts the light from a waveguide in a vertical direction towards the fibre above the chip," says Absil. This contrasts to traditional edge coupling to a device, achieved by dicing or cleaving a facet on the waveguide, he says.
Another research focus is how to reduce device power consumption and achieve thermal stability. One silicon photonics component that dictates the overall power consumption is the modulator, says Absil. "The Mach-Zehnder modulator is known to consume significant amounts of power for chip-to-chip distances," he says. "The alternative is to use resonating-based modulators but these have to be thermally controlled, and that has an associated power consumption."
Imec is looking at ways to reduce the thermal control needed and is investigating the addition of materials to silicon to create resonator modulators that do away with the need for heating.
The system-cost reduction work looks at packaging. "Eventually, we want to get the optical transceiver inside a host IC," says Absil. "That package has to enable an optical pass-through, whether it is fibre or an optically-transparent package." Such a requirement differs from established CMOS packaging technology. "The programme is also looking to explore new types of packaging for enabling this optical pass-through," he says.
Absil says certain programme elements are two years away from being completed. "In the programme, we have topics that are closer to being adopted and some that are further away, maybe even to 2020."
Multi-project wafer service
Imec is part of the a consortium of EC research institutes that provide low-cost access to companies that don't have the means to manufacture their own silicon photonics designs. Known as Essential, the EC's Seventh Framework (FP7) programme is an extension of the ePIXfab silicon photonics multi-project wafer initiative. "Imec is offering one flavour of the technology, Leti is also offering a flavour, and then there is IHP and VTT," says Absil. Once the Essential FP7 project is completed, the service will be continued by the Europractice IC service.
Has imec seen any growth now that the funding for OpSIS, the multi-project wafer provider, has come to an end? "We see decent contributions but I wouldn't say it is exponential growth," says Absil, who notes that the A*STAR Institute of Microelectronics in Singapore that OpSIS used continues to offer a multi-project wafer service.
Status of silicon photonics
Despite announcements from Acacia and Intel, and Finisar revealing at ECOC '14 that it is now active in silicon photonics, 2014 has been a quiet year for the technology.
"Right now it is a bit quiet because companies are investing in development," says Absil. "There is not so much incentive to publish this work." Another factor he cites for the limited news is that there are vertically-integrated vendors that are putting the technology in their servers rather than selling silicon-photonics products directly.
"This is only first generation," says Absil. "As it picks up, there will be more incentive to work on a second generation of silicon photonics which will depart from what we know from the early work published by Intel and Luxtera."
The opportunities this next-generation technology will offer are 'quite exciting', says Absil.
Fibre-to-the-NPU: optics reshapes the IP core router
Start-up Compass Electro-Optical Systems has announced an IP core router based on a chip with a Terabit-plus optical interface.
Asaf Somekh, vice president of marketing, showing Gazettabyte Compass-EOS's novel icPhotonics chip
Having an optical interface linking directly to the chip, which includes a merchant network processor, simplifies the system design and enables router features such as real output queuing. The r10004 IP router is in production and is already deployed in an operator's network.
The company's icPhotonics chip integrates 168, 8 Gigabit VCSELs and 168 photodetectors for a bandwidth of 1.344 Terabit-per-second (Tbps) each direction. Eight of the chips are connected in a full mesh, doing away with the need for a router's switch fabric and mid-plane used to interconnect the router cards.
The resulting architecture saves power, space and cost, says Asaf Somekh, vice president of marketing at Compass-EOS. The start-up estimates that its platform's total cost of ownership over five years is a quarter to a third of existing IP core routers.
The high-bandwidth optical links will also be used to connect multiple platforms, enabling operators to add routing resources as required. Compass-EOS is coming to market with a 6U-high standalone platform but says it will scale up to 21 platforms to appear as one logical router.
The 800Gbps-capacity r10004 comes with 2x100 Gigabit-per-second (Gbps) and 20x10Gbps line cards options. The platform has real output queuing where all the input ports' packets are queued with quality of service applied prior to the exit port. The router also supports software-defined networking (SDN) that enables external control of traffic routing.
The company has its own clean room where it makes its optical interface. Compass-EOS has also developed its own ASICs and the router software for the r10004.
Somekh says developing the optical interface has been challenging, requiring years of development working with the Fraunhofer Institute and Tel-Aviv University. One challenge was developing a glue to fix the VCSELs on top of the silicon.
The start-up has raised US $120M with investors such as Cisco Systems, Deutsche Telekom and Comcast as well as several venture capitalist firms.
icPhotonics technology
Compass-EOS refers to its optical interface IC as silicon photonics but a more accurate description is integrated silicon-optics; silicon itself is not used as a medium for light. But its use of embedded optics to the chip has created a disruptive system.
The optical-interconnect addresses two chip design challenges: signal integrity for long transmission lengths and chip input/output (I/O).
With high-speed interfaces, achieving signal integrity across a high-speed line card and between boards is challenging. Routers use a midplane and switch fabric to connect the the router cards within a platform and parallel optics to connect chassis.
Compass-EOS has taken board-mounted optics one step further and integrated VCSELs and photodetectors to the packaged chip. This simplifies the platform by connecting cards using a mesh architecture, and allows scaling by linking systems.
The chip window shows the VCSELs and photodetectors Source: Compass-EOS
The design also addresses chip I/O issues. "The I/O density is about 30x higher than traditional solutions and the gap will grow in future," says Somekh.
Directly attaching the optical interconnect to the CMOS chip overcomes limitations imposed by ball grid array and printed circuit board (PCB) technologies.
Typically data is routed from the host PCB to an ASIC via a ball grid array matrix which has a ball pitch of 0.8mm. Shrinking this further is non-trivial given PCB signal integrity issues. Moreover, each electrical serdes (serialiser/ deserialiser) for data I/O uses at least eight bumps (transmit, receive, signal and ground) occupying a cell of 3.2×1.6 mm. For a 10Gbps device the resulting duplex data density is 2Gbps/mm2, increasing to 5Gbps/mm2 if a 25Gbps device is used, according to Compass-EOS.
The start-up says its optical-interconnect achieves a chip I/O of 61Gbps/mm2. "This will increase to 243Gbps/mm2 once we move to 32Gbps."
The resulting design uses 10 percent of the total CMOS area for I/O. "This is a more efficient chip design," says Somekh. "Most of the silicon is used for logic tasks."
The serdes on chip still need to interface to hundreds of 8Gbps channels. And moving to 32Gbps will present a greater challenge. In comparison, silicon photonics promises to simplify the coupling of optics and electronics.
Another design challenge is that the VCSELs are co-packaged with a large chip consuming 30-50W and generating heat. The design needs to make sure that the operating temperature of the VCSELs is not affected by the heat from the chip.
This is another promised advantage of silicon photonics where the operating temperature of the optics and silicon are matched.
Analysts' perspective
Gazettabyte asked two analysts - IDC's Vernon Turner and ACG Research's Eve Griliches - about the significance of Compass-EOS's announcement. The analysts were also asked for their views on the router's modularity, the total cost of ownership claims, the support for SDN and real output queueing, and whether the platform will gain market share from the IP core router incumbents.
IDC
Vernon Turner, senior vice president & general manager enterprise computing, network, telecom, storage, consumer and infrastructure.
One of the hardest places to innovate in the ICT (information and communications technology) world is at or around the speed of light. Anytime you can make things run faster, the last hurdle tends to be the speed by which things travel over an optical network.
Therefore, to see something that changes the form factor of a network router and innovates at the interconnect speed, it may be able to disrupt a significant part of the network industry.
"Separating the interconnect with the physical building block is huge. It means that you scale the pieces that you need, when and where you want them; this is not just a repackaging announcement"
Building the capacity of a router as needed is great for service providers and large enterprises since you deploy capacity only as you need it. Second, by using a photonics interconnect, the speed and distance over which two devices can sit is enhanced greatly, changing the way one builds network infrastructures.
Separating the interconnect with the physical building block is huge. It means that you scale the pieces that you need, when and where you want them; this is not just a repackaging announcement.
Regarding the total-cost-of-ownership claims, if these are valid, they are of a magnitude that does fit into a 'disruptive innovation' class where it will deliver network services to an underserved market and create new network services markets.
SDN is the latest buzzword [regarding the router's support for SDN]. But it is the last part of the virtualised data centre as the compute and I/O have already been figured out. SDN is not new, but the need to separate the data plane from the control plane for the service provider industry means that they can begin to create network services through virtualisation without impacting the network performance, something that already happens in server and storage performance.
Existing core router vendors use their own ASIC designs as the last-stop differentiation, so to do this [as Compass-EOS has done] on merchant silicon could have wide implications on router commoditisation, or at least at a faster rate than current trends.
ACG Research
Eve Griliches, vice president of optical networking
As to the significance of the announcement, it is not huge in the scheme of things, but it does bring the optical component use of replacing a backplane to market earlier than what has been quoted to ACG Research.
"Virtual output queueing is a smart way to do quality of service"
In theory, the router should be a smaller footprint which results in better total cost of ownership due to the optical modules. The advantage with this optical patch-panel approach is that it allows a much higher bandwidth to cross the backplane which is now an optical interconnect. That means you don't have to do as much flow control, or drop as many packets, or keep the utilization of the router so low. You can bring up the utilisation rate from let's say 15 percent to maybe 25 percent or higher. All that results in lower total cost of ownership in theory.
SDN in a bit nebulous. Virtual output queueing is a smart way to do quality of service, but there are key software features like how many BGP (border gateway protocol) peers are supported, multicast capability, as well as signaling for MPLS (multiprotocol label switching), do they support RSVP-TE (resource reservation protocol - traffic engineering) or LDP (label distribution protocol)? Or both? Building a real router still takes years of work.
Faster interconnects are the way to go across routing and optical platforms, period. This [Compass-EOS platform] can help. Do I see this optical piece fitting nicely into an already existing router? Yes. I think if that doesn't happen, they will have a bit of an uphill battle nudging the incumbents.
On the other hand, if full router functionality is not needed at some junctures, as we've seen with the LSR (label switch router) technology, then they may have a place in the network. But operators don't like to play around with their routed network too much, so it may be greenfield application that are mostly available to them [Compass-EOS] initially.