Drut tackles disaggregation at a data centre scale
- Drut’s DynamicXcelerator supports up to 4,096 accelerators using optical switching and co-packaged optics. Four such clusters enable the scaling to reach 16,384 accelerators.
- The system costs less and is cheaper to run, has lower latency, and better uses the processors and memory.
- The system is an open design supporting CPUs and GPUs from different vendors.
- DynamicXcelerator will ship in the second half of 2024.
Drut Technologies has detailed a system that links up to 4,096 accelerator chips. And further scaling, to 16,384 GPUs, is possible by combining four such systems in ‘availability zones’.
The US start-up previously detailed how its design can disaggregate servers, matching the processors, accelerators, and memory to the computing task at hand. Unveiled last year, the product comprises management software, an optical switch, and an interface card that implements the PCI Express (PCIe) protocol over optics.
The product disaggregates the servers but leaves intact the tiered Ethernet switches used for networking servers across a data centre.
Now the system start-up is expanding its portfolio with a product that replaces the Ethernet switches with optical ones. “You can compose [compute] nodes and drive them using our software,” says Bill Koss, CEO of Drut.
Only Google has demonstrated the know-how to make such a large-scale flexible computing architecture using optical switching.
Company background
Drut was founded in 2018 and has raised several funding rounds since 2021.
Jitender Miglani, founder and president of Drut, previously worked at MEMS-based optical switch maker, Calient Technologies.
Drut’s goal was to build on its optical switching expertise and add the components needed to make a flexible, disaggregated computing architecture. “The aim was building the ecosystem around optical switches,” says Miglani.
The company spent its first two years porting the PCIe protocol onto an FPGA for a prototype interface card. Drut showcased its prototype product alongside a third-party optical switch as part of a SuperMicro server rack at the Supercomputing show in late 2022.
Drut has spent 2023 developing its next-generation architecture to support clusters of up to 4,096 endpoints. These can be accelerators like graphics processing units (GPUs), FPGAs, data processing units (DPUs), or storage using the NVM Express (nonvolatile memory express).
The architecture, dubbed DynamicXcelerator, supports PCIe over optics to link processors (CPUs and GPUs) and RDMA (Remote Direct Memory Access) over optics for data communications between the GPUs and between the CPUs.
The result is the DynamicXcelerator system, a large-scale reconfigurable computing for intensive AI model training and high-performance computing workloads.
DynamicXcelerator
The core of the DynamicXcelerator architecture is a photonic fabric based on optical switches. This explains why Drut uses PCIe and RDMA protocols over optics.
Optical switches brings size and flexibility and by relaying optical signals, their ports are data-rate independent.
Another benefit of optical switching is power savings. Drut says an optical switch consumes 150W whereas an equivalent-sized packet switch consumes 1,700W. On average, an Infiniband or Ethernet packet switch draws 750W when used with passive cables. Using active cables, the switch’s maximum power rises to 1,700W. “[In contrast], a 32-64-128-144 port all-optical switch draws 65-150W,” says Koss.
Drut also uses two hardware platforms. One is the PCIe Resource Unit, dubbed the PRU-2000, which hosts eight accelerator chips such as GPUs. Unlike Nvidia’s DGX platform, which uses Nvidia GPUs such as the Hopper, or Google, which uses its TPU5 tensor processor unit (TPU), Drut’s PRU-2000 is an open architecture and can use GPUs from Nvidia, AMD, Intel, and others. The second class of platform is the compute node or server, which hosts the CPUs.
DynamicXcelerator’s third principal component are the FIC 2500 interface cards.
The iFIC 2500 card is similar to Drut’s current product’s iFIC 1000, which features an FPGA and four QSFP28s. However, the iFIC 2500 supports the PCIe 5.0 generation bus and the Compute Express Link (CXL) protocols. The two other FIC cards are the tFIC 2500 and rFIC 2500.
“The iFIC and tFIC are the same card, but different software images,” says Koss. “The iFIC fits into a compute node or server while the tFIC fits into our Photonic Resource Unit (PRU) unit, which holds GPUs, FPGAs, DPUs, NVMe, and the like.”
The rFIC provides RDMA over photonics for GPU-to-GPU memory sharing. The rFIC card for CPU-to-CPU memory transfers is due later in 2024.
Miglani explains that PCIe is used to connect the GPUs and CPUs, but for GPU-to-GPU communication, RDMA is used since even PCIe over photonics has limitations.
Certain applications will use hundreds and even thousands of accelerators, so a PCIe lane count is one limitation, distance is another; a 5ns delay is added for each metre of fibre. “There is a window where the PCIe specification starts to fall off,” says Miglani.
The final component is DynamicXcelerator’s software. There are two software systems: the Drut fabric manager (DFM), which controls the system’s hardware configuration and traffic flows, and the Drut software platform (DSP) that interfaces applications onto the architecture.
Co-packaged optics
Drut knew it would need to upgrade the iFIC 1000 card. DynamicXcelerator uses PCIe 5.0, each lane being 32 gigabit-per-second (Gbps). Since 16 lanes are used, that equates to 512 gigabits of bandwidth.
“That’s a lot of bandwidth, way more that you can crank out with four 100-gigabit pluggables,” says Koss, who revealed co-packaged optics will replace pluggable modules for the iFIC 2500 and tFIC 2500 cards.
The card for the iFIC and tFIC will use two co-packaged optical engines, each 8×100 gigabits. The total bandwidth of 1.6 terabits – 16×100-gigabit channels – is a fourfold increase over the iFIC 1000.
System workings
The system’s networking can be viewed as a combination of circuit switching and packet switching.
The photonic fabric, implemented as a 3D torus (see diagram), supports circuit switching. Using a 3D torus, three hops at most are needed to link any two of the system’s endpoints.
One characteristic of machine learning training, such as large language models, is that traffic patterns are predictable. This suits an architecture that can set the resources and the connectivity for a task’s duration.
Packet switching is not performed using Infiniband. Nor is a traditional spine-leaf Ethernet switch architecture used. The DynamicXcelerator does uses Ethernet but in the form of a small, distributed switching layer supported in each interface card’s FPGA.
The smallest-sized DynamicXcelerator would use two racks of stacked PRU-2000s (see diagram). Further racks would be added to expand the system.
“The idea is that you can take a very large construct of things and create virtual PODs,” says Koss. “All of a sudden, you have flexible and fluid resources.”
Koss says a system can scale to 16,384 units by combining four clusters, each of 4,096 accelerators. “Each one can be designated as an ‘availability zone’, with users able to call resources in the different zones,” he says.
Customers might use such a configuration to segment users, run different AI models, or for security reasons. “It [a 16,384 unit system] would be huge and most likely something that only a service provider would do or maybe a government agency,” says Koss.
Capital and operation savings
Drut claims the architecture costs 30 per cent less than conventional systems, while operational cost-savings are 40 per cent.
The numbers need explaining, says Koss, given the many factors and choices possible.
The bill of materials of a 16, 32, 64 or 128-GPU design has a 10-30 per cent saving solely from the interconnect.
“The bigger the fabric, the better we scale in price as solutions using tiered leaf-spine-core packet switches involving Ethernet-Infiniband-PCIe are all built around the serdes of the switch chip in the box,” says Koss. “We have a direct-connect fabric with a very high radix, which allows us to build the fabric without stacked tiers like legacy point-to-point networks.”
There are also the power savings, as mentioned. Less power means less heat and hence less cooling.
“We can also change the physical wires in the network,” says Koss, something that can’t be done with leaf-spine-core networks, unless data centre staff change the cabling.
“By grouping resources around a workload, utilisation and performance are much better,” says Koss. “Apps run faster, infrastructure is grouped around workloads, giving users the power to do more with less.”
The system’s evolution is another consideration. A user can upgrade resources because of server disaggregation and the ability to add and remove resources from active machines.
“Imagine that you bought the DynamicXcelerator in 2024. Maybe it was a small sized, four-to-six rack system of GPUs, NVMe, etc,” says Koss. If, in mid-2026, Nvidia releases a new GPU, the user can take several PRU-2000s offline and replace the existing GPUs with the new ones.
“Also if you are an Nvidia shop but want to use the new Mi300 from AMD, no problem,” says Koss. “You can mix GPU vendors with the DynamicXcelerator.” This is different from today’s experience, where what is built is wasteful, expensive, complex, and certainly not climate-conscious, says Koss.
Plans for 2024
Drut has 31 employees, 27 of which are engineers. “We are going on a hiring binge and likely will at least double the company in 2024,” says Koss. “We are hiring in engineering, sales, marketing, and operations.”
Proof-of-concept DynamicXcelerator hardware will be available in the first half of 2024, with general availability then following.
The APC’s blueprint for silicon photonics
The Advanced Photonics Coalition (APC) wants to smooth the path for silicon photonics to become a high-volume manufacturing technology.
The organisation is talking to companies to tackle issues whose solutions will benefit the photonics technology.
The Advanced Photonics Coalition wants to act as an industry catalyst to prove technologies and reduce the risk associated with their development, says Jeffery Maki, Distinguished Engineer at Juniper Networks and a member of the Advanced Photonics Coalition’s board.
Origins
The Advanced Photonics Coalition was unveiled at the Photonic-Enabled Cloud Computing (PECC) Industry Summit jointly held with Optica last October.
The Coalition was formerly known as the Coalition for On-Board Optics (COBO), an industry initiative led by Microsoft.
Microsoft wanted a standard for on-board optics, until then it was a proprietary technology. At the time, on-board optics was seen as an important stepping stone between pluggable optical modules and their ultimate successor, co-packaged optics.
After years of work developing specifications and products, Microsoft chose not to adopt on-board optics in its data centres. Although COBO added other work activities, such as co-packaged optics, the organisation lost momentum and members.
Maki stresses that COBO always intended to tackle other work besides its on-board optics starting point.
Now, this is the Advanced Photonics Coalition’s goal: to have a broad remit to create working groups to address a range of issues.
Tackling technologies
Many standards organisations publish specifications but leave the implementation technologies to their member companies. In contrast, the Advanced Photonics Coalition is taking a technology focus. It wants to remove hurdles associated with silicon photonics to ease its adoption.
“Today, we see the artificial intelligence and machine learning opportunities growing, both in software and hardware,” says Maki. “We see a need in the coming years for more hardware and innovative solutions, especially in power, latency, and interconnects.”
Work Groups
In the past, systems vendors like Cisco or Juniper drove industry initiatives, and other companies fell in line. More recently, it was the hyperscalers that took on the role.
There is less of that now, says Maki: “We have a lot of companies with technologies and good ideas, but there is not a strong leadership.”
The Advanced Photonics Coalition wants to fill that void and address companies’ common concerns in critical areas. “Key customers will then see the value of, and be able to access, that standard or technology that’s then fostered,” says Maki.
The Advanced Photonics Coalition has yet to announce new working groups but it expects to do so in 2024.
One area of interest is silicon photonics foundries and their process design kits (PDKs). Each foundry has a PDK, made up of tools, models, and documentation, to help engineers with the design and manufacture of photonic integrated devices.
“A starting point might be support for more than one foundry in a multi-foundry PDK,” says Maki. “Perhaps a menu item to select the desired foundry where more than one foundry has been verified to support.”
Silicon photonics has long been promoted as a high-volume manufacturing technology for the optical industry. “But it is not if it has been siloed into separate efforts such that there is not that common volume,” says Maki.
Such a PDK effort would identify gaps that each foundry would need to fill. “The point is to provide for more than one foundry to be able to produce the item,” he says.
A company is also talking to the Advanced Photonics Coalition about co-packaged optics. The company has developed an advanced co-packaged optics solution, but it is proprietary.
“Even with a proprietary offering, one can make changes to improve market acceptance,” says Maki. The aim is to identify the areas of greatest contention and remedy them first, for example, the external laser source. “Opening that up to other suppliers through standards adoption, existing or new, is one possibility,” he says.
The Advanced Photonics Coalition is also exploring optical interconnecting definitions with companies. “How we do fibre-attached to silicon photonics, there’s a desire that there is standardisation to open up the market more,” says Maki. “That’s more surgical but still valuable.”
And there are discussions about a working group to address co-packaged optics for the radio access network (RAN). Ericsson is one company interested in co-packaged optics for the RAN. Another working group being discussed could tackle optical backplanes.
Maki says there are opportunities here to benefit the industry.
“Companies should understand that nothing is slowing them down or blocking them from doing something other than their ingenuity or their own time,” he says.
Status
COBO had 50 members earlier in 2023. Now, the membership listed on the website has dropped to 39 and the number could further dip; companies that joined for COBO may still decide to leave.
At the time of writing, an new as yet unannounced member has joined the Advanced Photonics Coalition, taking the membership to 40.
“Some of those companies that left, we think they will return once we get the working groups formed,” says Maki, who remains confident that the organisation will play an important industry role.
“Every time I have a conversation with a company about the status of the market and the needs that they see for the coming years, there’s good alignment amongst multiple companies,” he says.
There is an opportunity for an organisation to focus on the implementation aspects and the various technology platforms and bring more harmony to them, something other standards organisations don’t do, says Maki.
Books of 2023 - Final part
Gazettabyte has been asking industry figures to pick their reads of the year. In the final part, contributions are from Larry Dennison, Tim Doiron, Catherine White and Neil McRae.
Larry Dennison, Network Research Group, Nvidia
At this point in my life, book reading is to unwind and is mostly fiction. I get nearly everything else from reading selected technical papers, the daily news and Real Clear Politics. There is just so much cognitive dissonance in the news and editorials that I retreat into fantasy for some down-time.
The best books for me this year are the Beware of Chicken series. This is a light-hearted, martial arts/ cultivation world. Most of the world believes that ‘one strives for the heavens alone’. The main protagonist believes that ‘everything is connected’ and that relationships and doing the right thing are most important. This creates a central set of likeable characters who prevail and grow when challenges arise.
The other series is The Wandering Inn, a truly massive work with a multitude of likable and unlikable characters. Very rich world building, the main character is Erin who was transported from Earth and becomes an inn keeper. Erin sees the good in nearly everyone, including goblins, which results in her finding ways of dispelling prejudice. It isn’t always happy but there is always a sense of noble conduct.
Tim Doiron, Vice President, Solution Marketing, Infinera
In recent years, my reading has leaned toward technology, leadership, marketing, and history. However, with a son who recently completed his master’s degree in psychology, I found myself in 2023 developing an interest in topics related to human behavior and how people are wired.
In parallel with my newfound interest in psychology, I was asked to give a presentation at one of our recent leadership events. In that presentation, I referenced four books that had an impact on me and my thinking in 2023.
Three of these books were written in the past few years and the fourth is an older one that I revisited to prepare for my presentation. I’ll explain.
The first book is Think Again: The Power of Knowing What You Don’t Know, by Adam Grant, an organisational psychologist and well-known author. In Think Again, Grant identifies four styles commonly used to approach problems: preacher, prosecutor, politician, and scientist. While each of these approaches might be useful under certain situations, Grant argues that we should spend more time thinking like a scientist. We need to remain curious, challenging our own positions and assumptions and inviting others around us to do the same.
The world is changing fast, and positions that were accurate yesterday may not hold for today or tomorrow. For most of us in the technology industry, thinking like a scientist might come naturally, but we may not always apply it when making tradeoffs or debating strategies with colleagues. The second book is Dare to Lead: Bare Work. Tough Conversations. Whole Hearts. by Brenee Brown.
Brown is a research professor and storyteller in areas of shame, empathy, courage, and vulnerability. Brown has worked with all types of companies and organisations. To be courageous, you must be vulnerable. And vulnerability involves fear, uncertainty, and risk. If you find yourself thinking about effective leadership, this is a great book.
If we are going to think like a scientist and be courageous leaders, how do we solidify and anchor change in our organizations and our companies? That’s where John P. Kotter’s Leading Change comes in. I read this book 20 years ago and revisited it in preparation for my leadership presentation. We need to anchor change in the company culture if it’s going to stick. While this book isn’t new, the eight steps Kotter outlines for helping transform any organisation remain relevant.
Finally, as a marketeer I am always thinking about effective communications. Earlier this year, one of my colleagues at Infinera shared Smart Brevity: The Power of Saying More with Less by Jim VandeHei, Mike Allen and Roy Schwartz.
In our digital and social-media infused life, the ability to deliver relevant, concise, and impactful information has never been more important. This book provided some useful tips and scenarios and was a fast read. Maybe I don’t need to write all those white papers after all. Nice!
Catherine White, Researcher, Optical and Quantum technology, BT.
One book I read is Wasteland: The Dirty Truth About What We Throw Away, Where It Goes, and Why It Matters by Oliver Franklin-Wallis.
Technology has contributed to creating more waste than providing good solutions to solving the harm waste creates. There is also much work to be done to reclaim valuable materials.
At BT, there are programmes to reclaim and recycle materials from technical waste, among other initiatives for sustainability. For example, BT Group looks to circular networks in sustainability drive.
Wasteland is well written and brings home – in great detail – what we all basically know and must not ignore. It is also a fascinating, and sometimes horrifying journey waste takes once we say goodbye to it.
Another book I read is Novacene: The Coming Age of Hyperintelligence by James Lovelock. I am a great admirer of Lovelock and his early work on ecology. He was a brilliant, multi-talented engineer, and his Gaia theory, though it has a New Age association to some people, was based on computer simulations.
Lovelock lived for over a century, and his final book was published a few years ago, providing a startling vision of the future in which the predominant new intelligent life forms of the galaxy will be artificial, and the first of them (at least on this planet) created by us.
I am not sure he is right. I hope he is not because the thought is unsettling (though he has been proved to be prescient about many things). But his ideas are thought-provoking, even as a strawman to criticise, and it is the final work of a great individual.
During a break in Devon, I picked up a secondhand copy of a book of short stories The Rest of the Robots by the classic sci-fi author, Isaac Asimov.
It’s not his best robot book but I found an interesting story within this book in which the robotic proof-reader makes changes to the meaning of the text it is correcting, to match hard coded AI ethical rules that subsume other rules, with unintended effects.
Asimov had remarkable perception of the future but reading his work makes it clear he did not go far enough in predicting the sophistication with which AI would be able to reason. However, he was right about the unpredictability, and that is the key message for me. We finally need a robot psychologist like Asimov’s Susan Calman!
Neil McRae, Chief Network Strategist, Juniper Networks
The first book I read earlier this year was I May Be Wrong: And Other Wisdoms from Life as a Forest Monk, by Björn Natthiko Lindeblad.
I was recommended to read this by a friend. He recommended the book to help me with a big change in my life that I was going through, having left the company I worked for 12 years and sensing it was going to be more difficult than I might like to admit.
The book is the story of the author, a monk in Thailand. What I liked about this book is how closely the author seemed to mirror my thinking but from a totally different vantage point and wildly different life choices. He illustrates the struggle of being a monk and the realities of life, but it also teaches that the simplest things will make a difference in the world.
I found this approach inspiring, and the ending, well, I’m not going to give it away, but in a world where mental health is increasingly important, this energised me and got me moving on my next journey much quicker.
Through the Glass Ceiling to the Stars: The Story of the First American Woman to Command a Space Mission by Eileen Collins and Jonathan Ward is the amazing life story of Eileen Collins, the first female Space Shuttle Command and Pilot.
I have been fortunate to meet Eileen on many occasions, and the book surprised me in the way that Eileen had to deal with some brutal highs and lows, with immense mental strength during difficult times for her and for NASA and the Space Shuttle programme, and then the pressure of being the public face of the return to flight programme.
She is known for being the first female space shuttle pilot and commander, but Eileen was also the first woman to fly the F-15 fighter jet.
The book tells me that if you are determined enough and hungry enough, the sky is not the limit.
Broadcoms taps AI to improve switch chip traffic analysis
Broadcom’s Trident 5-X12 networking chip is the company’s first to add an artificial intelligence (AI) inferencing engine.
Data centre operators can use their network traffic to train the chip’s neural network. The Trident 5’s inference engine, dubbed the Networking General-purpose Neural-network Traffic-analyzer or NetGNT, is loaded with the resulting trained model to classify traffic and detect security threats.
“It is the first time we have put a neural network focused on traffic analysis into a chip,” says Robin Grindley, principal product line manager with Broadcom’s Core Switching Group.
Adding an inference engine shows how AI can complement traditional computation, in this case, packet processing.
Trident family
Trident is one of Broadcom’s main three lines of networking and switch chips, the Jericho and Tomahawk being the other two.
Service providers favour the Jericho family for high-end IP routing applications. The Ethernet switch router chip’s features include a programmable pipeline and off-chip store for large traffic buffering and look-up tables.
The latest Jericho 3, the 28.8 terabits-per-sec (Tbps) Jericho 3, was announced in September. Broadcom launched the first family device, the Jericho3-AI, earlier this year; a chip tailored for AI networking requirements.
In contrast, Broadcom’s Tomahawk Ethernet network switch family addresses the data centre operators’ needs. The Tomahawk has a relatively simple fixed packet-processing pipeline to deliver the highest switching capacity. The Tomahawk 5 has a capacity of 51.2 terabits and includes 512, 100-gigabit PAM4 serialiser-deserializer (serdes).
“The big hyperscalers want maximum bandwidth and maximum radix [switches],” says Grindley. “The hyperscalers have a pretty simple fabric network and do everything else themselves.”
The third family, the Trident Ethernet switch chips, is popular for enterprise applications. Like the Jericho, the Trident has a programmable pipeline to address enterprise networking tasks such as Virtual Extensible LAN (VXLAN), tunnelling protocols, and segment routing (SRv6).
The speeds and timelines of the various Tomahawk and Trident chips are shown in the chart.
Trident 5-X12
The Trident 5-X12 is implemented using a 5nm CMOS process and has a capacity of 16 terabits. The chip’s input-output includes 160, 100-gigabit PAM4 serdes. These are the serdes that Broadcom introduced with the Tomahawk 5.
The first chip of each new generation of Trident usually has the highest capacity and is followed by lower-capacity devices tailored to particular markets.
Trident 5 is aimed at top-of-rack switch applications. Typically, 24 or 48 ports of the top-of-rack switch are used for downlinks to connect to servers, while 4 or 8 are used for higher-capacity uplinks (see diagram).
The Trident 5 can support 48 ports of 200 gigabits for the downlinks and eight 800 gigabit for the uplinks. To support 800-gigabit interfaces, the chip uses eight 100-gigabit serdes and an one-chip 800-gigabit media access controller (MAC). Other top-of-rack switch configurations are shown in the diagram.
Currently, 400-gigabit network interface cards are used for demanding applications such as machine learning. Trident5 is also ready to transition to 800-gigabit network interface cards.
Another Tomahawk feature the Trident 5 has adopted is cognitive routing, a collection of congestion management techniques for demanding machine-learning workloads.
One of the techniques is global load balancing. Previous Trident devices supported dynamic load balancing, where the hardware could see the congested port and adapt in real-time. However, such a technique gives no insight into what happens further along the flow path. “If I knew that, downstream, somebody else was congested, then I could make a smarter decision,” says Grindley. Global load balancing does just this. It sends notification to the routing chips upstream that there is congestion so they can all work together.
Another cognitive routing feature is drop congestion notification. Here, packets dropped due to congestion are captured such that what is sent is only their header data and where the packet was dropped. This mechanism improves flow completion times compared to normal packet loss, which is costly for machine-learning workloads.
Trident 5, like its predecessor, Trident 4, has a heterogeneous pipeline of tile types. The tiles contain static random-access memory (SRAM), ternary content-addressable memory (TCAM) or arithmetic logic units. The tiles allow multiple look-ups or actions in parallel at each stage in the pipeline.
Broadcom has a compiler that maps high-level packet-processing functions to its pipeline in the NPL programming language. The latency through the device stays constant, however the packet processing is changed, says Grindley.
Trident 5’s NetGNT inference engine is a new pipeline resource for higher-level traffic patterns. “NexGNT looks at things not at a packet-by-packet level, but across time and the overall packet flow through the network,” says Grindley.
The NetGNT
Until now system architects and network operation centre staff have defined a set of static rules written in software to uncover and treat suspicious packet flows. “A pre-coded set of rules is limited in its ability to catch higher-level traffic patterns,” says Grindley.
When Broadcom started the Trident 5 design, its engineers thought a neural network approach could be used. “We knew it would be useful if you had something that looked at a higher level, and we knew neural networks could do this kind of task,” says Grindley.
The neural network sits alongside the existing traffic analysis logic. Information such as packet headers, or data already monitored and generated by the pipeline, can be fed to the neural network to assess the traffic patterns.
“It sits there and looks for high-level patterns such as the start of a denial of service attack” says Grindley.
Training
The neural network is trained using supervised learning. A human expert must create the required training data and train the model using supervised learning. The result is a set of weights loaded onto the Trident 5’s neural network.
When the neural network is triggered, i.e. when it identifies a pattern of interest, the Trident 5 must decide what it should do. The chip can drop the packets or change the quality of service (QoS). The device can also drop a packet while creating a mirror packet containing headers and metadata. This can then be sent to a central analyser at the network operations centre to perform higher-level management algorithms.
Performance
The Trident 5 chip is now sampling. Broadcom says there is no performance data as end customers are still to train and run live traffic through the Trident 5’s inference engine.
“What it can do for them depends on getting good data and then running the training,” says Grindley. “Nobody has done this yet.”
Will the inference engine be used in other Broadcom networking chips?
“It depends on the market,” says Grindley. “We can replicate it, just like taking IP from the Tomahawk where appropriate.”
Books of 2023 - Part 3
Gazettabyte has been asking industry figures to pick their reads of the year. In Part 3, Noam Mizrahi, Katharine Schmidtke, Steve Suarez, and Vladimir Kozlov share their readings of the year.
Noam Mizrahi, EVP, corporate CTO at Marvell.
Start with Why: How Great Leaders Inspire Everyone to Take Action, by Simon Sinek is a book about the obvious. It is so obvious, in fact, that it is very hard to do. We all want our message to get through so that people understand, see things through our eyes and share our vision.
When we start a journey, we know very well why we do it. This is also when we inspire and motivate the most, ourselves and others. But, as we develop our company, products and careers, intuitively, routine makes us focus on what we do and how we do it, and in some (or many) cases, we forget why we do it.
Once we forget the why, it is harder for us to experience a sense of accomplishment and, in most cases, will make it harder for us to inspire others to follow our vision.
Focus on the why as a means for inspiration and motivation. I find this simple advice something I always try to remember and in everything I do.
This book did not necessarily tell me what to do or how to do it, but I sure know why it was vital for me to read it.
Katharine Schmidtke, Ph.D., Eribel Systems LLC
Integrated Photonics for Data Communications Applications, Edited by Madeleine Glick, Ling Liao, and Katharine Schmidtke (2023), is the book I definitely read most in 2023!
This book, the inaugural volume in a series on integrated photonics, is a testament to the collaborative expertise inspired by Prof. Kimerling at MIT. It is the culmination of a three-year collaboration between co-editors Madeleine, Ling and me. We are incredibly grateful to the over ninety authors, each a leader in the field, whose technical expertise shines through and makes the content enriching and inspiring.
Diving into the world of advanced photonic devices and integrated photonic circuits, the book explores key concepts, design principles, performance metrics, and manufacturing processes. It goes beyond the theoretical, offering a comprehensive view of the practical aspects crucial for understanding and advancing this field.
One of the book’s strengths is its examination of the current trends and commercial requirements in data communication for data centres and high-performance computing. The inclusion of contributions from end users sharing key performance indicators adds a valuable real-world perspective.
At its core, the book dissects the fundamental building blocks of integrated photonics, unravelling the complexities of lasers, modulators, photodetectors, and passive devices. It’s a holistic journey through the individual elements that collectively form the intricate web of photonic integrated circuits.
Over the summer, I was back in England clearing out old bookshelves and discovered the series of spy novels by the British writer, John le Carré. I picked up The Little Drummer Girl, published in 1983. The story follows the manipulations of Martin Kurtz, an Israeli spymaster who intends to kill Khalil – a Palestinian terrorist who is bombing Jewish-related targets in Europe, particularly Germany – and Charlie, an English actress and double agent working on behalf of the Israelis.
It’s a thrilling and complex plot with many unexpected twists and turns, but this story has no heroes. Everyone loses something, including Charlie, who loses her mind. Reading it forty years after its writing, I experienced déjà vu during the events which started on October 7th, 2023.
I’ve had the book Narrative and Numbers: The Value of Stories in Business, by Aswath Damodaran on my reading list since its publication in 2017, and it certainly lived up to the anticipation.
The author delves into the intricacies of valuing companies, offering a profound analysis beyond the numbers. What sets this book apart is its exploration of the transformative power of storytelling in the business world.
As engineers, we often underestimate the impact of a well-crafted narrative. The author argues that a logical and rational story, when presented effectively, can breathe life into facts and figures.
The book emphasizes the importance of storytelling in making data understandable and unforgettable. The art of storytelling is revealed to be a compelling force that captivates audiences, making it challenging to dismiss even seemingly improbable valuations.
What struck me was the insight into how companies, seemingly without substantial revenue, can achieve remarkably high valuations.
By reading this book, you gain a deeper understanding of the alchemy that occurs when a compelling story intertwines with the cold, hard metrics of business. It’s a valuable read, shedding light on the often-underestimated influence of narrative in shaping perceptions and valuations.
Von der Nutzlosigkeit Erwachsen zu Werden, by Georg Heinzen and Uwe Koch (1994) can be translated as ‘Growing up from Uselessness’ or, because it’s a double entendre, ‘About the Uselessness of Growing up’.
The book is a farce about a tragic victim of the German education crisis of the 1970s and the job market of the 1990s. At thirty, still unemployed and living at home, this hopeless character discovers that having graduated high school, his education is helpful for everything but not needed for anything.
I wasn’t educated in Germany, but this mood was contagious throughout the rest of Europe, and I, too, struggled to get my first job during the recession of the 1990s.
Reading it now, with two teenagers preparing to launch themselves into the workforce in a few years, I’m sure they feel the same way. This might seem to be a depressing topic for me to dwell on, but the book is a fun read filled with ironic humour and many relevant topics for today.
Steve Suarez, Founder & CEO of HorizonX
This year marked a significant milestone for me. I took the courageous step of pursuing my lifelong aspiration of entrepreneurship. My goal is to empower organisations to innovate effectively and at scale. In a world where innovation is consistent and a requisite, there is a demand for skilled professionals who can excel in this realm across all industries and geographies.
To equip myself, I recognised the necessity to acquire new competencies, particularly in sales and the fundamentals of operating a thriving consulting business. To this end, I started listening to The Consulting Bible: How to Launch and Grow a Seven-Figure Consulting Business, by Alan Weiss, 2nd Edition in audiobook format, which I listened to during my commutes to London.
The insights have been invaluable. It has sharpened my focus on what drives business success and has influenced my approach to consulting. The lessons learned have been instrumental in shaping my entrepreneurial journey, allowing me to concentrate on strategies that make an impact.
I am eager to share more about how these learnings have transformed my business practices and the innovative solutions I can offer clients.
Vlad Kozlov, CEO and founder of LightCounting Market Research
An Immense World: How Animal Senses Reveal the Hidden Realms Around Us, by Ed Yong, reads like poetry, but it offers the depth of a Ph.D thesis, or several, with its range of topics.
You may already know that bats navigate the world using echolocation. Still, it works and it is fascinating, an incredible level of complexity chiselled by evolution over millennia, one mutation at a time. Even a recovering communist may wonder if evolution can do such a feat.
The book is dense and you have to take it in slowly. I’m almost finished now, and the most incredible chapter so far was on electric fields. Not about hundreds of volts that stingrays use but weak electric fields that many fish use to navigate murky waters. Unbelievable.
I am saving the next chapter on magnetic fields for the holidays.
A must read for anyone interested in high-tech innovation. Yes, this is a cutting edge technology. It is also a journey into a parallel world, or worlds, of creatures around us. What drives them remains a mystery, but all of them are caused by something in their lives. And it is more than just hunger.
Books of 2023 - Part 2
Gazettabyte asks industry figures to pick their reads of the year. In Part 2, Alan Liu, Yves LeMaitre, and, in this case, the editor of Gazettabyte list their recommended reads.
Alan Liu, CEO & Co-Founder at Quintessent Inc.
One book that left a deep impression on me is Man’s Search for Meaning by Viktor Frankl, a recounting and reflection by the author of his time as a prisoner in various concentration camps during WWII.
I listened to the audiobook mostly during commutes to work at the beginning of the year. Whatever challenges awaited me for the day, no matter how big, they seemed less daunting when reframed against the book’s stories.
The extreme deprivation and suffering described also gave me a deeper appreciation for the basic creature comforts of modern life that we enjoy (such as food, shelter, and coffee), which are easy to take for granted due to their constancy.
Yves LeMaitre, CEO of AstroBeam
Let me start with my favourite spy novel writer, John Le Carre. Pick any of his books. I just read his first small novel from 1961: Call for the Dead.
I recommend starting with his first major success, The Spy Who Came In from the Cold and if you like it, work your way to more recent books relevant to today’s tumultuous world: The Little Drummer Girl and A Most Wanted Man. Hopefully, it will bring you with an alternative viewpoint on some of today’s geopolitical hotspots
As the world continues to accept more diversity, if you want to glimpse Native American culture, try the easy path of the Tony Hillerman mystery books.
Then follow up with a trip to the Navajo and Hopi reservations in the Southwest. I promise it will change completely your views of the US history and Indian land ownership and occupation.
My favourite is A Thief of Time: A Leaphorn and Chee Novel but you can safely pick any of his books.
If you want to have the best Native guides in the Southwest, call my friend, Louis Williams, at Ancient Wayves River and Hiking Adventures: Guided Tours. He will make you discover the world of Diné and the incredible mystery of the lost Anasazi people.
Last summer, we had the best rafting trip on the San Juan River with his team, with incredible hikes in hidden canyons discovering ruins and artefacts left behind by the Ancient People.
Roy Rubenstein, Editor, Gazettabyte
One reading topic of continual interest is Israel. I have also listened to more podcasts this year and am a big fan of long-read articles.
I’m reading Isabel Kershner’s book: The Land of Hope and Fear: Israel’s Battle for Its Inner Soul. Kershner is the New York Times’s veteran correspondent in Israel. There is no shortage of books by journalists impacted by covering Israel. This is a timely primer for anyone wanting to understand the complexities of Israel.
Kai Bird is known for co-authoring the book on Robert Oppenheimer that was the basis of this year’s blockbuster film. But years ago he wrote a biography about CIA intelligence officer, Robert Aimes. Aimes was an outstanding character who served in the Middle East and died in the truck bomb assault on the US embassy in Beirut in 1983. Aimes got the Americans to talk to the PLO, ultimately leading to the Oslo Peace Accords.
Simon Baron-Cohen’s book, The Science of Evil: On Empathy and the Origins of Cruelty is another revisited book. The author is a psychologist and a leading authority on autism. Early in the book, he explains that he has an issue with the word ‘Evil’. In it, he explores why certain people cannot read, or don’t care, how others feel. He discusses the brain and structures such as the empathy circuit function. Empathy is absent when the circuit doesn’t work. However, the effects can vary significantly: people with autism differ from psychopaths. Why the circuit may malfunction is complex. It involves genetics, social, and environmental issues. The book, published in 2011, gives a different view on how to think about and treat cruelty.
In 2014, Prof Baron-Cohen co-signed a letter to The Times (of London) addressed to the leaders in Israel and Gaza that ends with the word empathy: “So, we say to the leaders of Israel and Hamas, please sit down, talk without table thumping, listen to each other and start a new politics based on the principles of respect, dignity, and empathy.”
One of my best reads is the book Two Roads Home: Hitler, Stalin, and the Miraculous Survival of My Family, by Daniel Finkelstein. It combines a period of upheaval in Europe and the Soviet Union with the survival of the author’s parents – who eventually meet and settle in Hendon, North London.
The book describes the history happening around two individuals who spent the rest of their lives bringing up their children in a loving home. The tale is remarkable and moving, including an early chapter where the author pays tribute to his father.
I met Finkelstein’s parents in the early 1990s but knew nothing of their story. I was also at Daniel’s sister’s wedding and remember being incredibly moved by the father’s speech.
Jonathan Raban is an author I lost track of only for him to resurface in the obituary columns, sadly. I realise he had moved to the US two decades ago.
His last book, Father and Son: A Memoir, is just out: about his recovery from a stroke coupled with the story of his parents and their love letters while separated during WWII.
Raban is a beautiful writer. “A nurse had assisted me into the wheelchair, and I was dozing there when Julia (his daughter) arrived to visit. The oddity of the situation made us both shy. We were deferential newcomers to the conventions of the hospital, like tourists with lowered voices tiptoeing around a foreign cathedral.
Lastly, The Atlantic and The New Yorker magazines published some great articles on AI this year:
- Talk to Me: Can artificial intelligence allow us to speak to another species?
- Geoffrey Hinton: It’s far too late to stop AI
- Does Sam Altman Know What he is Creating?
- How Jensen Huang’s Nvidia is powering the AI revolution
- The Inside Story of Microsoft’s Partnership with OpenAI
Books in 2023
Gazettabyte asks industry figures to pick their reads of the year. William Koss, Dean Bubley and Scott Wilkinson kick off this year’s recommended reads.
William R Koss, CEO at Drut Technologies
My 2023 reading list is less than normal as the year has been full of technical reading and presentation materials for work. I enjoy history books as well as business history that tell the rise and fall of some company, industry or person.
In Progress
Target Tokyo: The Story of the Sorge Spy Ring by Gordon W. Prange: I picked this book out of Amazon’s recommendation list. Gordon Prange being the author of At Dawn We Slept and Tora, Tora, Tora. Currently plowing through this book that was unfinished at the time of his death.
The Crusades: The War for the Holy Land by Thomas Asbridge. My knowledge of the Crusades was thin and I was looking for a book that provided a grand overview. So far it has not disappointed, but I have had to familiarize myself with many new names.
Completed Reads
Band of Brothers: E Company, 506th Regiment, 101st Airborne, from Normandy to Hitler’s Eagle’s Nest by Stephen E. Ambrose. A second read for me as I watched the series on Netflix over the summer and the thought occurred to read the book and compare and contrast the series to the book. Ambrose is a wonderful writer.
Going Infinite: The Rise and Fall of a New Tycoon by Michael Lewis. I was raising venture capital during the crypto craze from the same firms SBF raised capital and I admit that reading this book is part schadenfreude.
Circle of Treason: CIA Traitor Aldrich Ames and the Men He Betrayed by Sandra Grimes and Jeanne Vertefeuille.A second read for me. Something triggered the thought of Aldrich Ames and I read the book in two days.
The Wager: A Tale of Shipwreck, Mutiny and Murder by David Grann. A very fun read and puts into perspective the speed of news and information that we enjoy today. People thought along the time scale of years in the 1700s
This Kind of War: The Classic Korean War History by T.R. Fehrenbach. My father was in the Korean War and I have read many a book on the subject. It was a new read for me.
Duel in the Sun: Alberto Salazar, Dick Beardsley, and America’s Greatest Marathon by John Brant. My hobby is road cycling, but I have a colleague who has run the Boston Marathon a few times. The Boston Marathon route is within walking distance of my house and my colleague recommended this book as the best book written on marathon racing. I finished it on a couple of airplane rides.
Unscripted: The Epic Battle for a Media Empire and the Redstone Family Legacy by James B. Stewart and Rachel Abrams. A complete disappointment. The book was recommended by a former colleague and I just did not find all the personal details that interesting. I think I was hoping for a better read along the lines of the series Succession which had just ended and that was the reason for my reading.
Castles of Steel: Britain, Germany, and the Winning of the Great War at Sea by Robert K. Massie. Robert Massie is a master historian. One of the greats of our time. I have read Dreadnaught and Castles of Steel a few times. This book is master level history telling. Magnificent in all regards. Sections of the book can be read as short books. The story of Von Spee’s journey from the Pacific to Atlantic could be a single book. I am about to start his book Nicholas and Alexandra about the fall of the Romanov Dynasty.
Dean Bubley, technology industry analyst & futurist at Disruptive Analysis
A recent stand-out for me is Material World: The Six Raw Materials That Shape Modern Civilization by Ed Conway.
I found the book fascinating. It helped me gain a new angle on a lot of the issues faced in the economy and society overall, as well as specific bits of the tech sector.
It tells the stories of the production, processing, transport and use of some of the core minerals we use throughout society and technology. The book covers:
- sand/silicon used for concrete and also semiconductors and optical fibre
- lithium for batteries
- copper for cables, generators and motors
- oil & gas and why they’re still necessary at least for creating products rather than combustion (such as carbon anodes in batteries)
- salt(s) for multiple purposes
- iron & steel
One of the things I often realise is that it is easy to get wrapped up in technology including telecoms. We talk about virtualisation, AI, cloud, orchestration and software all the time.
There’s also a lot of physics. I often talk about radio spectrum and wireless propagation, including 5G and WiFi indoors and through walls. But I don’t pay much attention to the chemistry and materials involved.
This book poses some hard questions, such as where we get enough lithium (and also cobalt and other metals) for decarbonisation, or enough copper for new generators and grid capacity.
My takeout is that the next 20-30 years involve a tightrope walk, buffeted by the winds of physical materials, economics, geopolitics and hidden dependencies. It’s all very well saying ‘just stop doing X’, but sometimes (at least some) of X is essential in order to continue making Y or doing Z.
We also must be careful not just about “supplier diversity” for complex systems like radio access network equipment, or even the components and chips, but all the way down to the raw materials, which may be mined or refined in only a few places around the world.
Worth a read or a listen. I’m an audiobook devotee & this is narrated well enough to listen at 3x speed.
Scott Wilkinson, lead analyst, networking components, Cignal AI
There have been several books this year that I recommended to friends and colleagues. The Cartel by Don Winslow provides unique insights (for fiction) into the crisis at the southern US border.
My son, a Biomechanical Engineering Master’s student at Virginia Tech, and I both read Andy Weir’s Project Hail Mary. It’s like candy to engineers and I enjoyed discussing it with him as he made his way through the chapters.
I recently finished Rod Chernow’s massive biography, Grant, which was fascinating on every page, especially to those who were erroneously taught that he was a mediocre general who won the Civil War due only to attrition and not due to his strategic genius.
But the one book that I recommend the most to my engineering colleagues and history fans is The Great Bridge: The Epic Story of the Building of the Brooklyn Bridge by David McCullough. I’ve read several of McCullough’s histories, but never got around to reading this, his first. The Great Bridge tells the story of the building of the Brooklyn Bridge. It was an engineering feat that is hard to comprehend today.
Anyone driving or taking a subway train across the East River nay have difficulty imagining a time when Brooklyn and Manhattan were separate cities. The only way to get from one to the other was by ferry, and the residents in Brooklyn were worried that any more permanent connection might bring NYC corruption across the river. Washington Roebling took over the project when his father unexpectedly died early in the planning stages. With only his mind and his pencil, he designed every aspect of the bridge from the caissons sunk deep into the river to the cables spanning the towers. Plagued by an unknown disease he contracted after repeatedly descending into the pressurized caissons (what we now know as the bends), Roebling – and his very underappreciated wife, Emily – nevertheless managed a feat that boggles the mind, especially for engineers who let computers do the heavy lifting today.
The book describes challenges ranging from river currents to corruption to political interference, and parallels to modern times are not hard to make. Yet, almost 100 years later, when the bridge was inspected, the only recommendation was to add a coat of paint. The engineering is breathtaking, but the ability of the Chief Engineer to accomplish it with the tools of his time and with all of the roadblocks thrown up is awe-inspiring.
On a recent visit to New York to visit my daughter during her internship at the AMNH, I tried to convince the family to all travel down to the Brooklyn Bridge, just to look at it again in person. I was overruled, but that’s ok. It’ll still be there the next time, and for a long time after.
Ribbon offers for trial its 1.2T wavelength 9408 platform
Add Your Into Content Here
Ribbon Communications has started working with operators to trial its latest Apollo 9408 optical transport platform that supports 1.2 terabits per second (Tbps) optical wavelengths.
The company’s modular platform can also send 800 gigabit-per-second (Gbps) wavelengths over 1,000km and 400Gbps wavelengths over ultra-long-haul networks.
“We have conducted trials, including one with a Tier 1 European provider,” says Jonathan Homa, senior director of solutions marketing at Ribbon. “You can get 1.2 terabits within major cities, 800 gigabits covering major states or regions, and 400 gigabits for about as long as you want to go.”
“The Apollo 9408 is Ribbon’s first disaggregated transponder unit or compact modular box using the CIM 8 for up to 1.2Tbps of wavelength speed,” says Jimmy Yu, vice president at market research firm Dell’Oro Group.
Yu believes the product has shipped to a customer this quarter and is likely the first commercial shipment of a 1.2Tbps wavelength system for network deployment.
Acacia’s CIM 8 pluggable coherent modem
The Apollo 9408 uses Acacia’s pluggable Coherent Interconnect Module (CIM 8) coherent modem. The CIM 8 uses Acacia’s 5nm CMOS Jannu digital signal processor (DSP) and its silicon photonics-based coherent optics operating at a symbol rate of up to 140 gigabaud.
“The advantage of this smaller transistor geometry is not only the higher density per die but also lower power and faster processing speed,” says Yu. “All the things needed to help service providers achieve cost and power efficiencies.” This is why the market looks forward to the next generation of coherent DSPs, says Yu.
Acacia started shipping the CIM 8 at the year’s start, and Ribbon says the module’s availability enables the company to leapfrog existing 7nm CMOS-based coherent optical transport solutions.
Before 1.2 Tbps-capable wavelengths, the highest speed was 800Gbps, delivered by Ciena, Huawei, and Infinera, says Yu.
“Ciena was first to market and captured the lion’s share of shipment volumes,” says Yu. “We peg Ciena’s market share of 800 Gbps-capable wavelengths at approximately 70 per cent of the cumulative shipments through 2Q 2023. That is a huge share, benefiting from being first to market.”
Compact modular platform
The compact modular platform format was developed to meet the large-scale data centre operators’ computing needs. The platform is used for data centre interconnect applications while the large communications service providers are become interested in the platform form factor.
Compact modular platforms are 600mm deep and use front-to-back airflow for cooling. In contrast, standard telecom equipment is 300mm deep and uses a left-to-right airflow. The compact modular format thus suits data centres with alternate hot and cold aisles of equipment. The platforms face each other, so the air in a cold aisle is blown through each platform, exiting in the adjacent hot aisles. The efficient cooling scheme enables the equipment to be run hotter.
“With the compact modular platform’s front and back airflow, we can run the CIM 8 to 1.2 terabits,” says Homa. “In our standard [telecom] platform [the Apollo 9600 series], we’re using the same CIM 8 pluggable, but from a power dissipation point of view, we can only run it to 800 gigabits.”
The 9408 supports different channel plans depending on how the platform is used. For a cost-optimised transmission, a 400Gbps wavelength fits in a 75GHz channel, and a performance-optimised 800Gbps or 1.2Tbps wavelength fits in a 150GHz channel.
“With continuous baud rate control from 68-140Gbaud, the CIM 8 can accommodate any channel width such as 112.5GHz with networks that have flexible grid ROADMs [reconfigurable optical add/drop multiplexers],” says Homa. “It also uses probabilistic constellation shaping to maximise the line rate for that channel width.”
Configurations
The Apollo 9408 is a two rack unit (2RU) platform. For high-performance optical transport, it holds four MPJ1200_2 sleds. The sleds slot into the compact modular platform, with each sled hosting two CIM 8 modules. The power consumption of the double CIM 8 sled is 270W or less than 0.12W/gigabit. The total transport capacity is thus 9.6 terabits.
Ribbon plans to double the CIM 8s within the 2RU capacity platform to offer 19.2 terabits of capacity.
Alternatively, the 2RU rack can hold up to four MPQ_8 sleds hosting eight 400-gigabit coherent optical modules for a total capacity of 12.8 terabits. Ribbon uses 64 gigabaud 400-gigabit QSFP-DDs that use a transmit power of 0dBm and are OpenROADM MSA compliant.
“The MPQ_8 is also designed to accept a new generation of 124Gbaud 800Gbps QSFP-DD pluggables currently in development and expected to be available in early 2025,” says Homa.
Ribbon also offers its standard telecom Apollo 9600 series platforms, from the smallest 2RU 9603 to the 5RU 9608 to the largest 15RU 9624 chassis. The Apollo 9600 modular platforms can use two CIM 8s in the TM800_2 double-slot card for performance-optimised transmission to 800 gigabits, or two CFP2-DCO modules in the TM400_2 single-card slot card for cost-optimised transmission to 400 gigabits.
Industry timing
Optical system vendors that don’t develop their own coherent DSP chips or modems, such as Ribbon, have several supply options. The leading merchant DSP suppliers include Acacia, NEL and Marvell. There are also competitor optical transport providers that source their coherent modem solutions. Ribbon discussed with several coherent modem suppliers but chose Acacia’s CIM 8 for the 9408. Ribbon has worked with Acacia for a decade.
The CIM 8’s 5nm Jannu DSP leapfrogs the 90-100GBd 7nm CMOS generation of coherent DSPs now deployed. This year, 5nm CMOS coherent DSPs have been announced by Nokia and Infinera. Merchant suppliers NEL and Marvell have also detailed their latest coherent DSPs. All these devices operate at symbol rates in the region of 130-150GBd.
Acacia also supplies the CIM 8 to other optical transport vendors such as Cisco, Acacia’s parent company, ZTE, and Adtran. Cisco has announced its Network Convergence System (NCS) 1014 compact modular platform that includes a 2.4Tbps transponder Line using the CIM 8. In March, Adtran reported sending an 800-gigabit signal over 2,200km using the CIM 8 as part of a networking trial. The route included 14 route-and-select flexible-grid ROADMs.
“It will be interesting to see the market dynamics unfold over the next year. There will be more system suppliers of 1.2 Tbps-capable wavelengths,” says Dell’Oro’s Yu. “Many system vendors will use the CIM 8, and some will use NEL’s ExaSpeed GAIA DSP. Some will also develop in-house DSPs such as Huawei and Nokia.”
Every dense wavelength division multiplexing (DWDM) system vendor will have a 1.2 Tbps-capable line card available for sale before the end of 2024, except for Ciena, says Yu: “This is because Ciena will come out with a 1.6 Tbps-capable DSP on a 3nm process node in 2024, one to two years ahead of any other vendors.”
Earlier this year, Ciena announced its WaveLogic 6, the first coherent DSP that operates at 200GBd. Ciena says it will offer its optical transport systems using its 3nm CMOS coherent DSP in the second half of 2024.
Homa believes that the next jump will be 240-plus GBd coherent DSPs, likely implemented using an even smaller 2nm CMOS process node.
The OIF’s 1600ZR 1.6-terabit coherent pluggable module standard will use a 240GBd symbol rate DSP.
ECOC 2023 industry reflections - Final Part
Gazettabyte has been asking industry figures to reflect on the recent ECOC show in Glasgow. The final instalment emphasises coherent technology with contributions from Adtran, Cignal AI, Infinera, Ciena, and Acacia.
Jörg-Peter Elbers, head of advanced technology at Adtran
The ECOC 2023 conference and show was a great event. The exhibition floor was busy and offered ample networking opportunities. In turn, the conference and the Market Focus sessions provided information on the latest technologies, products, and developments.
One hot topic was coherent 800ZR modems. Several vendors demonstrated coherent 800ZR modules and related components. Importantly, these modules also boast new and improved 400 gigabit-per-second (Gbps) modes. The 120 gigabaud (GBd) symbol rate enables 400-gigabit dual-polarisation quadrature phase shift keying (DP-QPSK) transmission over demanding links and long-haul routes. In turn, the advent of 5nm CMOS digital signal processor (DSP) technology enables lower power DP-16QAM than 400ZR modules.
There is broad agreement that the next step in coherent transmission is a 240GBd symbol rate, paving the way to single-wavelength 1.6 terabit-per-second (Tbps) optical transport.
Meanwhile, the use of coherent optical technology closer to the network edge continues. Several players announced plans to follow Adtran and Coherent and jump on the low-power 100 gigabit-per-second ZR (100ZR) ‘coherent lite’ bandwagon. Whether passive optical networking (PON) systems will adopt coherent technology after 50G-PON sparked lively debate but no definitive conclusions.
The OIF 400ZR+ demonstration showed interoperability between a dozen optical module vendors over metro-regional distances. It also highlighted the crucial role of an intelligent optical line system such as Adtran’s FSP3000 OLS in automating operation and optimising transmission performance.
The post-deadline papers detailed fibre capacity records by combining multiple spectral bands and multiple fibre cores. The line-system discussions on the show floor focused on the practical implications of supporting C-, L-, extended, and combined band solutions for customers and markets.
From workshops to the regular sessions, the application of artificial intelligence (AI) was another prominent theme, with network automation a focus area. Examples show not only how discriminative AI can detect anomalies or analyse failures but also how generative AI can improve the interpretation of textual information and simplify human-machine and intent interfaces. For network engineers, ‘Copilot’-like AI assistance is close.
After ECOC is also before ECOC, so please mark in your calendars September 22-26, 2024. ECOC will celebrate its 50th anniversary next year and will take place in Frankfurt, Germany. As one of the General Chairs of the ECOC 2024 event, and on behalf of the entire organising committee, I look forward to welcoming you!
Andrew Schmitt, founder and directing analyst, Cignal AI
ECOC is a great show, it’s like OFC (the annual optical communications and networking event held in the US) but refined to only the critical elements. Here are my key takeaways.
The most impressive demonstration was 800ZR test boards and modules from Marvell and its partners Coherent and Lumentum. Within eight weeks of the first silicon, Marvell has demos up and running in-house and at its partners. The company has at least a 6-month lead in the 800ZR market, making intelligent tradeoffs to achieve this.
Lumentum showed an 8-QAM mode of operation that allows 800 gigabit transmission within a 100GHz channel spacing, which should be interesting. After the massive success of 400ZR, it’s natural to extrapolate the same success for 800ZR, but the use cases for this technology are substantially different. We also heard updates and broader support for 100ZR.
Linear drive pluggable optics (LPO) was a hot topic, although it was our impression that, while optimism ruled public discussion, scepticism was widely expressed in private. There was more agreement than disagreement with our recent report (see the Active Insight: The Linear Drive Market Opportunity). No one is more confident about LPO than the companies who view this as another opportunity to bid for business at hyperscale operators they don’t currently have.
The 200 gigabit per lane silicon/ physical media device (PMD)/ optics development continues, and it is on track to enable 1.6-terabit optics by 2024. Marvell had a more advanced and mature demo of what they showed behind closed doors at OFC. The advancements here are the real threat to adopting LPO, and people need to realise that LPO is competing with the power specs of 200 gigabits per lane, not 100 gigabits per lane solutions.
Also impressive was the comprehensive engineering effort by Eoptolink to show products that covered 100 gigabit and 200 gigabit per lane solutions, both retimed and linear. The company’s actions show that if you have the engineering resources and capital, rather than pick the winning technology, do everything and let the market decide. Also impressive is the CEO, who understood the demos and the seasoned application engineers. Kudos to keeping engaged with the products!
System vendors had a more significant presence at the show, particularly Ciena and Infinera. It’s unsurprising to see more system vendors since they are increasing investments in pluggables, particularly coherent pluggables.
We had many discussions about our forecasts for IPoDWDM deployment growth. This disruption is something that component vendors are excited about, and hardware OEMs view it as an opportunity to adjust how they deliver value to operators (see the Active Insight: Assessing the Impact of IP-over DWDM).
Lastly, the OIF coordinated 400ZR+ and OpenROADM interoperability testing despite the organisation not being directly involved in those industry agreements. The OIF is a fantastic organisation that gets valuable things done that its members need.
Paul Momtahan, director, solutions marketing, Infinera
ECOC 2023 provided an excellent opportunity to catch the latest trends regarding transponder innovation, coherent pluggables and optical line systems. A bonus was getting to the show without needing a passport.
Transponder innovation topics included coherent digital signal processor (DSP) evolution, novel modulators, and the maximum possible baud rate. DSP sessions included the possibility of offloading DSP functions into the photonic domain to reduce power consumption and latency.
There were also multiple presentations on constellation shaping, including enhanced nonlinear performance, reduced power consumption for probabilistic constellation shaping, and potential uses for geometric shaping.
Novel modulators with very high baud rates, including thin-film lithium niobate, barium titanate, plasmonic, and silicon-organic hybrid, were covered. The need for such modulators is the limited bandwidth potential of silicon photonics modulators, though each face challenges such as integration with silicon photonics and manufacturability.
From the baud rate session, the consensus was that 400GBd symbol rates are probable, up to 500GBd might be possible, but higher rates are unlikely. The critical challenges are the radio frequency (RF) interconnects and the digital-to-analogue and analogue-to-digital converters. However, several presenters wondered whether a multi-wavelength transponder might be more sensible for symbol rates above 200 to 250GBd.
Coherent pluggables were another topic, especially at 800 gigabit. However, one controversial topic was the longevity of coherent pluggables in routers (IPoDWDM). Several presenters argued the current period would pass once router port speeds and coherent port speeds no longer align.
As the coherent optical engines approach the Shannon limit, innovation is shifting towards optical line systems and fibres as alternative way to scale capacity.
Several presentations covered ROADM evolution to 64 degrees and even 128 degrees. A contrasting view is that ROADMs’ days are numbered to be replaced by fibre switches and full spectrum transponders, at least in core networks.
Additional options for scaling capacity included increasing the spectrum of existing bands with super-C and super-L. Lighting different bands, such as the S-band (in addition to C+L bands), is seen as the best candidate, with commercial solutions three to five years away.
Overall, it was a great event, and I look forward to seeing how things evolve by the time of next year’s ECOC show in Frankfurt. (For more, click here)
Helen Xenos, senior director, portfolio marketing, Ciena
This was my third year attending ECOC, and the show never disappoints. I always leave this event excited and energised about what we’ve accomplished as an industry.
Every year seems to bring new applications and considerations for coherent optical technology. This year, ECOC showcased the ever-growing multi-vendor ecosystem for 400-gigabit coherent pluggable transceivers, considerations in the evolution to 800-gigabit pluggables, evolution to coherent PON, quantum-secure coherent networking, and the evolution to 200 gigabaud and beyond. When will coherent technology make it into the data centre? A question still open for debate.
Ciena’s optical engineer wizards were on hand to share specifics about our recently announced 3nm CMOS-based WaveLogic 6 technology, which includes the industry’s first performance-optimised 1.6 teraburs-per-second (Tbps) optics as well as 800-gigabit pluggables.
It was exciting for me to introduce customers, suppliers and research graduates to their first view of 3nm chip performance results and show how these enable the next generation of products. And, of course, Ciena was thrilled that WaveLogic 6 was awarded the Most Innovative Coherent Module Product at the event.
Tom Williams, director of technical marketing at Acacia
From my perspective, while there weren’t as many major product announcements as OFC, several trends and technologies continued to progress, including OIF interoperability, 800ZR/ZR+, linear pluggable optics (LPO) and terabit optics.
The OIF interop demonstration was once again a highlight of the show. The booth was at the entrance to the exhibition and seemed to be packed with people each time I passed by.
OIF has expanded the scope of these demonstrations with each show, and this year was the largest ever. In addition to having the participation of 12 module vendors (with 34 modules), the focus was on the ZR+ operation. What was successfully demonstrated was a single-span 400ZR network and a multi-span network.
As co-chair of the OpenZR+ MSA, I was excited by the great collaboration with OIF. These efforts help to drive the industry forward. Karl Gass is not only the most creatively dressed person at every trade show; he is exceptional at coordinating these activities.
It is clear that linear drive pluggable optics (LPO) works in some situations, but views differ about how widespread its adoption will be and how standardisation should be addressed. I lived through the analogue coherent optics (ACO) experience. ACO was essentially a linear interface for a coherent module where the digital processing happened outside the module. For ACO, it was a DSP on the host board and for LPO it is the switch ASIC. The parameters that need to be specified are similar. There is a precedent for this kind of effort. Hopefully, lessons learned there will be helpful for those driving LPO. I am interested to see how this discussion progresses in the industry as some of the challenges are discussed, such as its current limited interoperability and support for 200 gigabits per lane.
There have been announcements from several companies about performance-optimised coherent optics in what we call Class 3 (symbol rates around 140 gigabaud), which support up to 1.2 terabits on a wavelength. Our CIM 8 module has been used in multiple field trials, demonstrating the performance benefits of these solutions.
Our CIM 8 (Coherent Interconnect Module 8) achieves this performance in a pluggable form factor. The CIM 8 uses the same 3D siliconisation technology we introduced for our 400-gigabit pluggables and enables operators to scale their network capacity in a cost- and power-efficient way.
ECOC 2023 industry reflections - Part 3
Gazettabyte is asking industry figures for their thoughts after attending the recent ECOC show in Glasgow. In particular, what developments and trends they noted, what they learned and what, if anything, surprised them. Here are responses from Coherent, Ciena, Marvell, Pilot Photonics, and Broadcom.
Julie Eng, CTO of Coherent
It had been several years since I’d been to ECOC. Because of my background in the industry, with the majority of my career in data communications, I was pleasantly surprised to see that ECOC had transitioned from primarily telecommunications, and largely academic, into more industry participation, a much bigger exhibition, and a focus on datacom and telecom. There were many exciting talks and demos, but I don’t think there were too many surprises.
In datacom, the focus, not surprisingly, was on architectures and implementations to support artificial intelligence (AI). The dramatic growth of AI, the massive computing time, and the network interconnect required to train models are driving innovation in fibre optic transceivers and components.
There was significant discussion about using Ethernet for AI compared to protocols such as InfiniBand and NVLink. For us as a transceiver vendor, the distinction doesn’t have a significant impact as there is little if any, difference in the transceivers we make for Ethernet compared to the transceivers we make for InfiniBand/NVLink. However, the impact on the switch chip market and the broader industry are significant, and it will be interesting to see how this evolves.
Linear pluggable optics (LPO) was a hot topic, as it was at OFC 2023, and multiple companies, including Coherent, demonstrated 100 gigabit-per-lane LPO. The implementation has pros and cons, and we may find ourselves in a split ecosystem, with some customers preferring LPO and others preferring traditional pluggable optics with DSP inside the module. The discussion is now moving to the feasibility of 200 gigabit-per-lane LPO.
Discussion and demonstrations of co-packaged optics also continued, with switch vendors starting to show Ethernet switches with co-packaged optics. Interestingly, the success of LPO may push out the implementation of co-packaged optics, as LPO realizes some of the advantages of co-packaged optics with a much less dramatic architectural change.
One telecom trend was the transition to 800-gigabit digital coherent optical modules, as customers and suppliers plan for and demonstrate the capability to make this next step. There was also significant interest in and discussion about 100G ZR. We demonstrated a new version with 0dBm high optical output power at ECOC 2023 while other companies showed components to support it. This is interesting for cable providers and potentially for data centre interconnect and mobile fronthaul and backhaul.
I was very proud that our 200 gigabit-per-lane InP-based DFB-MZ laser won the 2023 ECOC Exhibition Industry Award for Most Innovative Product in the category of Innovative Photonics Component.
ECOC was a vibrant conference and exhibition, and I was pleased to attend and participate again.
Loudon Blair, senior director, corporate strategy, Ciena
ECOC 2023 in Glasgow gave me an excellent perspective on the future of optical technology. In the exhibition, integrated photonic solutions, high-speed coherent pluggable optical modules, and an array of testing and interoperability solutions were on display.
I was especially impressed by how high-bandwidth optics is being considered beyond traditional networking. Evolving use cases include optical cabling, the radio access network (RAN), broadband access, data centre fabrics, and quantum solutions. The role of optical connectivity is expanding.
In the conference, questions and conversations revolved around how we solve challenges created by the expanding use cases. How do we accommodate continued exponential traffic growth on our fibre infrastructure? Coherent optics supports 1.6Tbps today. How many more generations of coherent can we build before we move on to a different paradigm? How do we maximize density and continue to minimize cost and power? How do we solve the power consumption problem? How do we address the evolving needs of data centre fabrics in support of AI and machine learning? What is the role of optical switching in future architectures? How can we enhance the optical layer to secure our information traversing the network?
As I revisited my home city and stood on the banks of the river Clyde – at a location once the shipbuilding centre of the world – I remembered visiting my grandfather’s workshop where he built ships’ compasses and clocks out of brass.
It struck me how much the area had changed from my childhood and how modern satellite communications had disrupted the nautical instrumentation industry. In the same place where my grandfather serviced ships’ compasses, the optical industry leaders were now gathering to discuss how advances in optical technology will transform how we communicate.
It is a good time to be in the optical business, and based on the pace of progress witnessed at ECOC, I look forward to visiting San Diego next March for OFC 2024.
Dr Loi Nguyen, executive vice president and general manager of the cloud optics business group, Marvell
What was the biggest story at ECOC? That the story never changes! After 40 years, we’re still collectively trying to meet the insatiable demand for bandwidth while minimizing power, space, heat, and cost. The difference is that the stakes get higher each year.
The public debut of 800G ZR/ZR+ pluggable optics and a merchant coherent DSP marked a key milestone at ECOC 2023. For the first time, small-form-factor coherent optics delivers performance at a fraction of the cost, power, and space compared to traditional transponders. Now, cloud and service providers can deploy a single coherent optics in their metro, regional, and backbone networks without needing a separate transport box. 800 ZR/ZR+ can save billions of dollars for large-scale deployment over the programme’s life.
Another big topic at the show was 800G linear drive pluggable optics (LPO). The multi-vendor live demo at the OIF booth highlighted some of the progress being made. Many hurdles, however, remain. Open standards still need to be developed, which may prove difficult due to the challenges of standardizing analogue interfaces among multiple vendors. Many questions remain about whether LPO can be scaled beyond limited vendor selection and bookend use cases.
Frank Smyth, CTO and founder of Pilot Photonics
ECOC 2023’s location in Glasgow brought me back to the place of my first photonics conference, LEOS 2002, which I attended as a postgrad from Dublin City University. It was great to have the show close to home again, and the proximity to Dublin allowed us to bring most of the Pilot team.
Two things caught my eye. One was 100G ZR. We noted several companies working on their 100G ZR implementations beyond Coherent and Adtran (formerly Adva) who announced the product as a joint development over a year ago.
100G ZR has attracted much interest for scaling and aggregation in the edge network. Its 5W power dissipation is disruptive, and we believe it could find use in other network segments, potentially driving significant volume. Our interest in 100G ZR is in supplying the light source, and we had a working demo of our low linewidth tunable laser and mechanical samples of our nano-iTLA at the booth.
Another topic was carrier and spatial division multiplexing. Brian Smith from Lumentum gave a Market Focus talk on carrier and spatial division multiplexing (CSDM), which Lumentum believes will define the sixth generation of optical networking.
Highlighting the approaching technological limitation on baud rate scaling, the ‘carrier’ part of CSDM refers to interfaces built from multiple closely-spaced wavelengths. We know that several system vendors have products with interfaces based on two wavelengths, but it was interesting to see this from a component/ module vendor.
We argue that comb lasers come into their own when you go beyond two to four or eight wavelengths and offer significant benefits over independent lasers. So CSDM aligns well with Pilot’s vision and roadmap, and our integrated comb laser assembly (iCLA) will add value to this sixth-generation optical networking.
Speaking of comb lasers, I attended an enjoyable workshop on comb lasers on the Sunday before the meetings got too hectic. The title was ‘Frequency Combs for Optical Communications – Hype or Hope’. It was a lively session featuring a technology push team and a market pull team presenting views from academia and industry.
Eric Bernier offered an important observation from HiSilicon. He pointed to a technology gap between what the market needs and what most comb lasers provide regarding power per wavelength, number of wavelengths, and data rate per lane. Pilot Photonics agrees and spotted the same gap several years ago. Our iCLA bridges it, providing a straightforward upgrade path to scaling to multi-wavelength transceivers but with the added benefits that comb lasers bring over independent lasers.
The workshop closed with an audience participation survey in which attendees were asked: Will frequency combs play a major role in short-reach communications? And will they play a major role in long-reach communications?
Unsurprisingly, given an audience interested in comb lasers, the majority’s response to both questions was yes. However, what surprised me was that the short-reach application had a much larger majority on the yes side: 78% to 22%. For long-reach applications the majority was slim: 54% to 46%.
Having looked at this problem for many years, I believe the technology gap mentioned is easier to bridge and delivers greater benefits for long-reach applications than for short-reach, at least in the near term.
Natarajan Ramachandran, director of product marketing, physical layer products division, Broadcom
Retimed pluggables have repeatedly shown resiliency due to their standards-based approach, offering reliable solutions, manufacturing scale, and balancing metrics around latency, cost and power.
At ECOC this year, multiple module vendors demonstrated 800G DR4 and 1.6T DR8 solutions with 200 gigabit-per-lane optics. As the IEEE works towards ratifying the specs around 200 gigabit per lane, one thing was clear at ECOC: the ecosystem – comprising DSP vendors, driver and transimpedence amplifier (TIA) vendors, and VCSEL/EML/silicon photonics vendors – is ready and can deliver.
Several vendors had module demonstrations using 200 gigabit-per-lane DSPs. What also was apparent at ECOC was that the application space and use cases, be it within traditional data centre networks, AI and machine learning clusters and telcom, continue to grow. Multiple technologies will find the space to co-exist.