Building an AI supercomputer using silicon photonics
- Luminous Computing is betting its future on silicon photonics as an enabler for an artificial intelligence (AI) supercomputer
Silicon photonics is now mature enough to be used to design complete systems.
So says Michael Hochberg (pictured), who has been behind four start-ups including Luxtera and Elenion whose products used the technology. Hochberg has also co-authored a book along with Lukas Chrostowski on silicon photonics design.
In the first phase of silicon photonics, from 2000 to 2010, people wondered whether they could even do a design using the technology.
“Almost everything that was being done had to fit into an existing socket that could be served by some other material system,” says Hochberg.
A decade later it was more the case that sockets couldn’t be served without using silicon photonics. “Silicon photonics had dominated every one of the transceiver verticals that matter: intra data centre, data centre interconnect, metro and long haul,” he says.
Now people have started betting their systems using silicon photonics, says Hochberg, citing the examples as lidar, quantum optics, co-packaged optics and biosensing.
Several months ago Hochberg joined as president of Luminous Computing, a start-up that recently came out of stealth mode after raising $105 million in Series A funding.
Luminous is betting its future on silicon photonics as an enabler for an artificial intelligence (AI) supercomputer that it believes will significantly outperform existing platforms.
Machine learning
The vision of AI is to take tasks that were the exclusively the domain of the human mind and automate them at scale, says Hochberg.
Just in the last decade, the AI community has advanced from doing things using machine learning (ML) that are trivial for humans to tasks that only the most talented experts can achieve.
“We have reached the point where machine learning capabilities are superhuman in many respects,” says Hochberg. “Where they produce results quantifiably better than humans can.”
But achieving such machine learning progress has required huge amounts of data and hardware.
“The training runs for the state-of-the-art recommendation engines and natural language models take tens to hundreds of thousands of GPUs (graphics processing units) and they run from months to years,” says Hochberg.
Moreover, the computational demands associated with machine learning training aren’t just doubling every 18 months, like with Moore’s law, but every 3-4 months. “And for memory demands, it is even faster,” he says.
What that means is that the upper limit for doing such training runs are complete data centres.
Luminous Computing wants to develop AI hardware that scales quickly and simply. And a key element of that will be to use silicon photonics to interconnect the hardware.
“One of the central challenges scaling up big clusters is that you have one kind of bus between your CPU and memory, another between your CPU and GPU, another between the GPUs in a box and yet another – Infiniband – between the boxes,” says Hochberg.
These layers of connectivity run at different speeds and latencies that complicate programming for scale. Such systems also result in expensive hardware like GPUs being under-utilised.
“What we are doing is throwing massive optical interconnect at this problem and we are building the system around this optical interconnect,” says Hochberg.
Using sufficient interconnect will enable the computation to scale and will simplify the software. “It is going to be simple to use our system because if you need anything in memory, you just go and get it because there is bandwidth to spare.”
Supercomputing approach
Luminous is not ready to reveal its supercomputer architecture. But the company says it is vertically integrated and is designing the complete system including the processing and interconnect.
When the company started in 2018, it planned to use a photonic processor as the basis of the compute but the class of problems it could solve were deemed insufficiently impactful.
The company then switched to developing a set of ASICs designed around the capabilities of the optics. And it is the optics that rearchitects how data moves within the supercomputer.
“That is the place where you get order-of-magnitude advantages,” says Hochberg.
The architecture will tackle a variety of AI tasks typically undertaken by hyperscalars. “If we can enable them to run models that are bigger than what can be run today while using much smaller programming teams, that has enormous economic impact,” he says.
Hochberg also points out that many organisations want to use machine learning for lots of markets: “They would love to have the ability to train on very large data sets but they don’t have a big distributed systems engineering team to figure out how to scale things up onto big-scale GPUs; that is a market that we want to help.”
The possible customers of Luminous’s system are so keen to access such technology that they are helping Luminous. “That is something I didn’t experience in the optical transceiver world,” quips Hochberg.
The supercomputer will be modular, says Luminous, but its smallest module will have much greater processing capability than, say, a platform hosting 8 or 16 GPUs.
Silicon photonics
Luminous is confident in using silicon photonics to realise its system even though the design will advance how the technology has been used till now.
“You are always making a bet in this space that you can do something that is more complex than anything anyone else is doing because you are going to ship your product a couple of years hence,” says Hochberg
Luminous is has confidence because of the experience of its design team, the design tools it has developed and its understanding of advanced manufacturing processes.
“We have people that know how to stand up complex things,” says Hochberg.
Status
Luminous’s staff is currently around 100, a doubling in the last year. And it is set to double again by year-end.
The company is busy doing modelling work as to how the machine learning algorithms will run on its system. “Not just today’s models but also tomorrow’s models,” says Hochberg.
Meanwhile, there is a huge amount of work to be done to deliver the first hardware by 2024.
“We have a bunch of big complex chips we have to build, we have software that has to live on top of it, and it all has to come together and work,” concludes Hochberg.
Books in 2015 - Part 2
Yuriy Babenko, senior network architect, Deutsche Telekom
The books I particularly enjoyed in 2015 dealt with creativity, strategy, and social and organisational development.
People working in IT are often right-brained people; we try to make our decisions rationally, verifying hypotheses and build scenarios and strategies. An alternative that challenges this status quo and looks at issues from a different perspective is Thinkertoys by Michael Michalko.
Thinkertoys develops creativity using helpful tools and techniques that show problems in a different light that can help a person stumble unexpectedly on a better solution.
Some of the methods are well known such as mind-mapping and "what if" techniques but there is a bunch of intriguing new approaches. One of my favourites this year, dubbed Clever Trevor, is that specialisation limits our options, whereas many breakthrough ideas come from non-experts in a particular field. It is thus essential to talk to people outside your field and bounce ideas with them. It leads to the surprising realisation that many problems are common across fields.
The book offers a range of practical exercises, so grab them and apply.
I found From Third World to First: The Singapore Story - 1965-2000 by by Lee Kuan Yew, the founder of modern Singapore, inspiring.
Over 700 pages, Mr. Lee describes the country’s journey to ‘create a First World oasis in a Third World region". He never tired to learn, benchmark and optimise. The book offers perspectives on how to stay confident no matter what happens, focus and execute the set strategy; the importance of reputation and established ties, and fact-based reasoning and argumentation.
Lessons can be drawn here for either organisational development or business development in general. You need to know your strengths, focus on them, not rush and become world class in them. To me, there is a direct link to a resource-based approach, or strategic capability analysis here.
The massive Strategy: A History by Lawrence Freeman promises to be the reference book on strategy, strategic history and strategic thinking.
Starting with the origins of strategy including sources such as The Bible, the Greeks and Sun Tzu, the author covers systematically, and with a distinct English touch, the development of strategic thinking. There are no mathematics or decision matrices here, but one is offered comprehensive coverage of relevant authors, thinkers and methods in a historical context.
Thus, for instance, Chapter 30 (yes, there are a lot of chapters) offers an account of the main thinkers of strategic management of the 20th century including Peter Drucker, Kenneth Andrews, Igor Ansoff and Henry Mintzberg.
The book offers a reference for any strategy-related questions, in both personal or business life, with at least 100 pages of annotated, detailed footnotes. I will keep this book alive on my table for months to come.
The last book to highlight is Continuous Delivery by Jez Humble and David Farley.
The book is a complete resource for software delivery in a continuous fashion. Describing the whole lifecycle from initial development, prototyping, testing and finally releasing and operations, the book is a helpful reference in understanding how companies as diverse as Facebook, Google, Netflix, Tesla or Etsy develop and deliver software.
With roots in the Toyota Production System, continuous delivery emphasises empowerment of small teams, the creation of feedback processes, continuous practise, the highest level of automation and repeatability.
Perhaps the most important recommendation is that for a product to be successful, ‘the team succeeds or fails’. Given the levels of ever-rising complexity and specialisation, the recommendation should be taken seriously.
Roy Rubenstein, Gazettabyte
I asked an academic friend to suggest a textbook that he recommends to his students on a subject of interest. Students don’t really read textbooks anymore, he said, they get most of their information from the Internet.
How can this be? Textbooks are the go-to resource to uncover a new topic. But then I was at university before the age of the Internet. His comment also made me wonder if I could do better finding information online.
Two textbooks I got in 2015 concerned silicon photonics. The first, entitled Handbook of Silicon Photonics provides a comprehensive survey of the subject from noted academics involved in this emerging technology. At 800-pages-plus, the volume packs a huge amount of detail. My one complaint with such compilation books is that they tend to promote the work and viewpoints of the contributors. That said, the editors Laurent Vivien and Lorenzo Pavesi have done a good job and while the chapters are specialist, effort is made to retain the reader.
The second silicon photonics book I’d recommend, especially from someone interested in circuit design, is Silicon Photonics Design: From Devices to Systems by Lukas Chrostowski and Michael Hochberg. The book looks at the design and modelling of the key silicon photonics building blocks and assumes the reader is familiar with Matlab and EDA tools. More emphasis is given to the building blocks than systems but the book is important for two reasons: it is neither a textbook nor a compendium of the latest research, and is written for engineers to get them designing. [1]
I also got round to reading a reflective essay by Robert W. Lucky included in a special 100th anniversary edition of the Proceedings of the IEEE magazine, published in 2012. Lucky started his career as an electrical engineer at Bell Labs in 1961. In his piece he talks about the idea of exponential progress and cites Moore’s law. “When I look back on my frame of reference in 1962, I realise that I had no concept of the inevitability of constant change,” he says.
1962 was fertile with potential. Can we say the same about technology today? Lucky doesn’t think so but accepts that maybe such fertility is only evident in retrospect: “We took the low-hanging fruit. I have no idea what is growing further up the tree.”
A common theme of some of the books I read in the last year is storytelling.
I read journalist Barry Newman’s book News to Me: Finding and Writing Colorful Feature Stories that gives advice on writing. Newman has been writing colour pieces for the Wall Street Journal for over four decades: “I’m a machine operator. I bang keys to make words.”
I also recommend Storytelling with Data: A Data Visualization Guide for Business Professionals by Cole Nussbaumer Knaflic about how best to present one’s data.
I discovered Abigail Thomas’s memoirs A Three Dog Life: A Memoir and What Comes Next and How to Like It. She writes beautifully and a chapter of hers may only be a paragraph. Storytelling need not be long.
Three other books I hugely enjoyed were Atul Gawande's Being Mortal: Medicine and What Matters in the End, Roger Cohen’s The Girl from Human Street: A Jewish Family Odyssey and the late Oliver Sacks’ autobiography On the Move: A Life. Sacks was a compulsive writer and made sure he was never far away from a notebook and pen, even when going swimming. A great habit to embrace.
Lastly, if I had to choose one book - a profound work and a book of our age - it is One of Us: The Story of Anders Breivik and the Massacre in Norway by Asne Seierstad.
For Books in 2015 - Part 1, click here
Further Information
[1] There is an online course that includes silicon photonics design, fabrication and data analysis and which uses the book. For details, click here