Making optical networking feel like cycling downhill
BT’s chief architect, Neil McRae, is a fervent believer in the internet, a technology built on the continual progress of optical networking. He discussed both topics during his invited talk at the recent OFC 2021 virtual conference and exhibition.
Neil McRae’s advocacy of the internet as an educational tool for individuals from disadvantaged backgrounds stems from his childhood experiences.
“When I was a kid, I lived in a deprived area and the only thing that I could do was go to the library,” says McRae, chief architect and managing director for architecture and technology strategy at BT.
His first thought on discovering the internet was just how much there was to read.
“If I’m honest, everything I’ve learnt in technology has been pretty much self-taught,” says McRae.
This is why he so values the internet. It has given him a career where he has travelled widely and worked with talented and creative people.
“Anyone who is out there in the world can do the same thing,” he says. “I strongly believe that the internet brings opportunities to people who are willing to spend the time to learn.”
Optical networking
McRae surveyed the last 20 years of optical networking in his OFC talk. He chose the period since it was only at the end of the last century that the internet started to have a global impact.
“The investment in networking [during this period] has been orders of magnitude bigger than prior years,” says McRae. “There has also been a lot of deregulation across the world, more telecoms companies, more vendors and ultimately more people getting connected.”
In 2000, networks used the SONET/SDH protocol and fixed wavelengths. “We have brought in many new technologies – coherent, coloured optics, programable lasers and silicon photonics – and they have been responsible for pretty significant changes.”
McRae likens optical network to gears on a bike. “It powers the rest of what we do in the network and without those advances, we wouldn’t be the digitally connected society we are today,” says McRae. “If I think about the pandemic of the last year, can you imagine what the pandemic would have been like if it had happened in the year 2000?”
McRae says he spends a fifth of his time on optical networking. This is more than previously due to the relentless growth in network bandwidth.
“Ultimately, if you get optical wrong, it feels like you are in the wrong gear cycling uphill,” says McRae. “If you get it right, you are in the right gear, you are going as fast as you can go and it feels like a downhill ride.”
And it’s not just bandwidth but also from a cost, capability and customer experience perspective. “We recognise the value that it brings to all the other layers right up to the application,” he says.
Research
BT Labs has an optical networking programme that is run by Professor Andrew Lord. The programme’s remit is to help BT address existing and future issues.
“There is a longer-term research aspect to what Andrew and his team do, but there are some here-and-now issues that they support me on like the hollow-core fibre work and some of the 400-gigabit [coherent] platforms we have been reviewing recently,” he says.
He cites as examples the work the programme did for BT’s next-generation optical platform that was designed for growth and which indeed has grown massively in the last decade. “We have launched optical services as a product because of the platform,” says McRae.
The programme has also helped Openreach, BT Group’s copper and fibre plant subsidiary, with its fibre-to-the-premise (FTTP) deployments that use such technologies as GPON and XGS-PON.
Reliable, dynamic, secure networks
McRae admits he is always nervous about predicting the future. But he is confident 400 gigabits will be a significant optical development over the next decade.
This includes inside the data centre, driven by servers, and in the network including long haul.
“The challenge will be around getting the volume and interoperability as quickly as we possibly can,” says McRae.
The other big opportunity is the increased integration of IP and optical using a control plane aligned to both.
“The biggest networking technology out there is IP,” says McRae. “And that will not change in the coming decade.”
The Layer-3 capabilities include working around issues but it is bad at managing bandwidth. Optical is the opposite: great at managing bandwidth but less dynamic for working around problems. Merging the two promises significant benefits.
This idea, advocated as IP-over-DWDM, has long been spoken of but has not been deployed widely. The advent of 400-gigabit coherent implemented using client-side modules means that the line-side interface density can equal that of the host. And other developments such as software-defined networking and artificial intelligence also help.
Software-defined networking will make a big difference because it will enable the move to automation and that will enable new technologies such as artificial networking (AI) and machine-learning to be introduced.
McRae talks of a control plane capable of deciding which interface to send packets down and also determine what paths to create across the optical infrastructure.
“We have seen some of that but we have not seen enough,” says McRae. AI and machine-learning technologies will provide networks with almost autonomous control over which paths to use and enable for the various traffic types the network sees.
McRae stresses that it is getting harder to get the maximum out of the network: “If we maintain human intervention, the network will never see its full potential because of complexity, demands and scale.”
He predicts that once the human component is taken out of the network, some of the silos between the different layers will be removed. Indeed, he believes networks built by AI and aided by automation will look very different to today’s networks.
Another technology McRae highlights is hollow-core fibre which BT Labs has been researching.
“Increasingly, we are starting to reach some limits although many folks have said that before, but hollow-core fibre gives us some interesting and exciting opportunities around latency and the total use of a fibre,” says McRae.
There are still challenges to be overcome such as manufacturing the fibre at scale but he sees a path in many parts of the network where hollow-core fibre could be valuable to BT.
Quantum key distribution (QKD) and the importance of network security is another area starting to gain momentum.
“We have gone from a world where people were scared to send an email rather than a fax to one where the network is controlling mission-critical use cases,” says McRae. “The more secure and reliable we make those networks, the more it will help us in our everyday lives.”
McRae believes this is the decade where the underlying optical network capability coupled with QKD security will take effect.
Making a difference
McRae has run several events involving children with autism although during the pandemic this has not happened. He uses gaming as a way to demonstrate how electronics works – switching things on and off – and then he introduces the concept of computer programming.
“I find that kids with autism get it really quickly” he says. BT runs such events two or three times a year.
McRae also works with children who are learning to program but find it difficult. “Again, it is something self-taught for me,” he says although he quips that the challenge he has is that he teaches them bad programming habits.
“I’m keen to find the next generation of fantastic engineers; covid has shown us that we need them more than ever,” he says.
T-API taps into the transport layer
The Optical Internetworking Forum (OIF) in collaboration with the Open Networking Foundation (ONF) and the Metro Ethernet Forum (MEF) have tested the second-generation transport application programming interface (T-API 2.0).
SK Telecom's Park Jin-hyo
T-API 2.0 is a standardised interface, released in late 2017 by the ONF, that enables the dynamic allocation of transport resources using software-defined networking (SDN) technology.
The interface has been created so that when a service provider, or one of its customers, requests a service, the required resources including the underlying transport are configured promptly.
The OIF-led interoperability demonstration tested T-API 2.0 in dynamic use cases involving equipment from several systems vendors. Four service providers - CenturyLink, Telefonica, China Telecom and SK Telecom - provided their networking labs, located in three continents, for the testing.
Packets and transport
SDN technology is generally associated with the packet layer but there is also a need for transport links, from fibre and wavelength-division multiplexing technology at Layer 0 through to Layer 2 Ethernet.
Transport SDN differs from packet-based SDN in several ways. Transport SDN sets up dedicated pipes whereas a path is only established when packets flow for packet SDN. “When you order a 100-gigabit connection in the transport network, you get 100 gigabits,” says Jonathan Sadler, the OIF’s vice president and Networking Interoperability Working Group chair. “You are not sharing it with anyone else.”
Another difference is that at the packet layer with its manipulation of packet headers is a digital domain whereas the photonic layer is analogue. “A lot of the details of how a signal interacts with a fibre, with the wavelength-selective switches, and with the different componentry that is used at Layer 0, are important in order to characterise whether the signal makes it through the network,” says Sadler.
T-API 1.0 is a configure and step-away deployment, T-API 2.0 is where the dynamic reactions to things happening in the network become possible
Prior to SDN, control functions resided on a platform as part of a network’s distributed control plane. Each vendor had their own interface between the control and the optical domain embedded within their platforms. T-API has been created to expose and standardise that interface such that applications can request transport resources independent of the underlying vendor equipment.
NBI refers to a northbound interface while SBI stands for a southbound interface. Source: OIF.
To fulfil a connection across an operator’s network involves a hierarchy of SDN controllers. An application’s request is first handled by a multi-domain SDN controller that decomposes the request for the various domain controllers associated with the vendor-specific platforms. T-API 2.0’s role is to link the multi-domain controller to the application layer’s orchestrator and also connect the individual domain controllers to the multi-domain SDN controller (see diagram above). T-API is an example of a northbound interface.
The same T-API 2.0 interface is used at both SDN controller levels, what differs is the information each handles. Sadler compares the upper T-API 2.0 interface to a high-level map whereas the individual TAPI 2.0 domain interfaces can be seen as maps with detailed ‘local’ data. “Both [interfaces] work on topology information and both direct the setting-up of connections,” says Sadler. “But the way they are doing it is with different abstractions of the information.”
T-API 2.0
The ONF developed the first T-API interface as part of its Common Information Model (CIM) work. The interface was tested in 2016 as part of a previous interoperability demonstration involving the OIF and the ONF.
One important shortfall revealed during the 2016 demonstrations, and which has slowed its deployment, is that the T-API 1.0 interface didn't fully define how to notify an upper controller of events in the lower domains. For example, if a link is congested, or worst, lost, it couldn’t inform the upper controller to re-route traffic. This has been put right with T-API 2.0.
“T-API 1.0 is a configure and step-away deployment, T-API 2.0 is where the dynamic reactions to things happening in the network become possible,” says Sadler.
When it comes to the orchestrator tying into the transport network, we do believe T-API will be one of the main approaches for these APIs
Interoperability demonstration
In addition to the four service providers, six systems vendors took part in the recent interoperability demonstration: ADVA Optical Networking, Coriant, Infinera, NEC/ Netcracker, Nokia and SM Optics.
The recent tests focussed on the performance of the TAPI-2.0 interface under dynamic network conditions. Another change since the 2016 tests was the involvement of the MEF. The MEF has adopted and extended T-API as part of its Network Resource Modeling (NRM) and Network Resource Provisioning (NRP) projects, elements of the MEF’s Lifecycle Service Orchestration (LSO) architecture. The LSO allows for service provisioning using T-API extensions that support the MEF’s Carrier Ethernet services.
Three aspects of the T-API 2.0 interface were tested as part of the use cases: connectivity, topology and notification.
Setting up a service requires both connectivity and topology. Topology refers to how a service is represented in terms of the node edge points and the links. Notification refers to the northbound aspect of the interface, pushing information upwards to the orchestrator at the application layer. This allows the orchestrator in a multi-domain network to re-route connectivity services across domains.
The four use cases tested included multi-layer network connections whereby topology information is retrieved from a multi-domain network with services provisioned across domains.
T-API 2.0 was also used to show the successful re-routing of traffic when network situations change such as a fault, congestion, or to accommodate maintenance work. Re-routing can be performed across the same layer such as the IP, Ethernet or optical layer, or, more optimally, across two or more layers. Such a capability promises operators the ability to automate re-routing using SDN technology.
The two other use cases tested during the recent demonstration were the orchestrator performing network restoration across two or more domains, and the linking of data centres’ network functions virtualisation infrastructure (NFVI). Such NFVI interconnect is a complex use case involving SDN controllers using T-API to create a set of wide area networks connecting the NFV sites. The use case set up is shown in the diagram below.
Source: OIF
SK Telecom, one of the operators that participated in the interoperability demonstration, welcomes the advent of T-API 2.0 and says how such APIs will allow operators to enable services more promptly.
“It has been difficult to provide services such as bandwidth-on-demand and networking services for enterprise customers enabled using a portal,” says Park Jin-hyo, executive vice president of the ICT R&D Centre at SK Telecom. “These services will be provided within minutes, according to the needs, using the graphical user interface of SK Telecom’s network-as-service platform.”
SK Telecom stresses the importance of open APIs in general as part of its network transformation plans. As well as implementing a 5G Standalone (SA) Core, SK Telecom aims to provide NFV and SDN-based services across its network infrastructure including optical transport, IP, data centres, wired access as well as networks for enterprise customers.
“Our final goal is to open the network itself to enterprise customers via an open API,” says Park. “Our mission is to create 5G-enabled network-slicing-based business models and services for vertical markets.”
Takeways
The OIF says the use cases have shown that T-API 2.0 enables real-time orchestration and that the main shortcomings identified with the first T-API interface have been addressed with T-API 2.0.
The OIF recognises that while T-API may not be the sole approach available for the industry - the IETF has a separate activity - the successful tests and the broad involvement of organisations such as the ONF and MEF make a strong case for T-API 2.0 as the approach for operators as they seek to automate their networks.
“When it comes to the orchestrator tying into the transport network, we do believe T-API will be one of the main approaches for these APIs,“ says Sadler.
SK Telecom said participating in the interop demonstrations enabled it to test and verify, at a global level, APIs that the operators and equipment manufacturers have been working on. And from a business perspective, the demonstration work confirmed to SK Telecom the potential of the ‘global network-as-a-service’ concept.
Editor note: Added input from SK Telecom on September 1st.
ONF advances its vision for the network edge
The Open Networking Foundation’s (ONF) goal to create software-driven architectures for the network edge has advanced with the announcement of its first reference designs.
In March, eight leading service providers within the ONF - AT&T, Comcast, China Unicom, Deutsche Telekom, Google, NTT Group, Telefonica and Turk Telekom - published their strategic plan whereby they would take a hands-on approach to the design of their networks after becoming frustrated with what they perceived as foot-dragging by the systems vendors.
Timon SloaneThree months on, the service providers have initial drafts of the the first four reference designs: a broadband access architecture, a spine-leaf switch for network functions virtualisation (NFV), a more general networking fabric that uses the P4 packet forwarding programming language, and the open disaggregated transport network (ODTN).
The ONF also announced four system vendors - Adtran, Dell EMC, Edgecore Networks, and Juniper Networks - have joined to work with the operators on the reference design programmes.
“We are disaggregating the supply chain as well as disaggregating the technology,” says Timon Sloane, the ONF’s vice president of marketing and ecosystem. “It used to be that you’d buy a complete solution from one vendor. Now operators want to buy individual pieces and put them together, or pay somebody to do it for them.”
We are disaggregating the supply chain as well as disaggregating the technology
CORD and Exemplars
The ONF is known for various open-source initiatives such as its ONOS software-defined networking (SDN) controller and CORD. CORD is the ONF’s cloud optimised remote data centre work, also known as the central office re-architected as a data centre. That said, the ONF points out that CORD can be used in places other than the central office.
“CORD is a hardware architecture but it is really about software,” says Sloane. “It is a landscape of all our different software projects.”
However, the ONF received feedback last year that service providers were putting the CORD elements together slightly differently. “Vendors were using that as an excuse to say that CORD was too complicated and that there was no critical mass: ‘We don’t know how every operator is going to do this and so we are not going to do anything’,” says Sloane.
It led to the ONF’s service providers agreeing to define the assemblies of common components for various network platforms so that vendors would know what the operators want and intend to deploy. The result is the reference designs.
The reference designs offer operators some flexibility in terms of the components they can use. The components may be from the ONF but need not be; they can also be open-source or a vendor’s own solution.
Source: ONF
The ONF has also announced the exemplar platforms aligned with the reference designs (see diagram). An exemplar platform is an assembly of open-source components that builds an example platform based on a reference design. “The exemplar platforms are the open source projects that pull all the pieces together,” says Sloane. “They are easy to download, trial and deploy.”
The ONF admits that it is much more experienced with open source projects and exemplar platforms that it is with reference designs. The operators are adopting an iterative process involving all three - open source components, exemplar designs and reference designs - before settling on the solutions that will lead to deployments.
Two of the ONF exemplar platforms announced are new: the SDN-enabled broadband access (SEBA) and the universal programmable automated network (UPAN).
Reference designs
The SEBA reference design is a broadband variant of the ONF’s CORD work and addresses residential and backhauling applications. The design uses Kubernetes, the cloud-native orchestration system that automates the deployment, scaling and management of container-based applications, while the use of the OpenStack platform is optional. “OpenStack is only used if you want to support a virtual machine-based virtual network function,” says Sloane.
Source: ONF
SEBA uses VOLTHA, the open-source virtual passive optical networking (PON) optical line terminal (OLT) developed by AT&T and contributed to the ONF, and provides interfaces to both legacy operational support systems (OSS) and the Linux Foundation’s Open Networking Automation Platform (ONAP).
SEBA also features FCAPS and mediation. FCAPS is an established telecom capability for network management that can identify faults while the mediation presents information from FCAPS in a way the OSS understands.
“In its slimmest implementation, SEBA doesn’t need CORD switches, just a pair of aggregation switches,” says Sloane. The architecture can place sophisticated forwarding rules onto the optical line terminal and the aggregation switches such that servers and OpenStack are not required. “That has tremendous performance and scale implications,” says Sloane. “No other NFV architecture does this kind of thing.”
The second reference design - the NFV Fabric - ties together two ONF projects - Trellis and ONOS - to create a spine-leaf data centre fabric for edge services and applications.
The two remaining reference designs are UPAN and ODTN.
UPAN can be viewed as an extension of the NFV fabric that adds the P4 data plane programming language. P4 brings programmability to the data plane while the SDN controller enables developers to specify particular forwarding behaviour. “The controller can pull in P4 programs and do intelligent things with them,” says Sloane. “This is a new world where you can write custom apps that will push intelligence into the switch.”
Meanwhile, the ODTN reference design is used to add optical capabilities including reconfigurable optical add-drop multiplexers (ROADMs) and wide-area-network support.
There are also what the ONF calls two trailblazer projects - Mobile CORD (M-CORD) and CORD - that are not ready to become reference designs as they depend on 5G developments that are still taking place.
CORD represents the ONF’s unifying project that brings all the various elements together to address multi-access edge cloud. Also included as part of CORD is an edge cloud services platform. “This is the ultimate vision: what is the app store for edge applications?” says Sloane. “If you write a latency-sensitive application for eyeglasses, for example, how does that get deployed across multiple operators and multiple geographies?”
The ONF says it has already achieved a ‘critical mass’ of vendors to work on the development of the reference designs three months after announcing its strategic plan. The supply chain for each of the reference designs is shown in the table.
Source: ONF
“We boldly stated that we were going to reconstitute the supply chain as part of this work and bring in partners more aligned to embrace enthusiastically open source and help this ecosystem form and thrive,” says Sloane. “It is a whole new approach and to be able to rally the ecosystem in a short timeframe is notable.”
Our expectation is that at least two of these reference designs will go through this transition this year. This is not a multi-year process.
Next steps
It is the partner operators that are involved in the development of the reference designs. For example, the partners working on ODTN are China Unicom, Comcast and NTT. Once the reference designs are ready, they will be released to ONF members and then publicly.
However, the ONF has yet to give timescales as to when that will happen. “Our expectation is that at least two of these reference designs will go through this transition this year,” says Sloane. “This is not a multi-year process.”
What the cable operators are planning for NFV and SDN
Cable operators are working on adding wireless to their fixed access networks using NFV and SDN technologies.
Don Clarke“Cable operators are now every bit as informed about NFV and SDN as the telcos are, but they are not out there talking too much about it,” says Don Clarke, principal architect for network technologies at CableLabs, the R&D organisation serving the cable operators.
Clarke is well placed to comment. While at BT, he initiated the industry collaboration on NFV and edited the original white paper which introduced the NFV concept and outlined the operators’ vision for NFV.
NFV plans
The cable operators are planning developments by exploiting the Central Office Re-architected as a Datacenter (CORD) initiative being pursued by the wider telecom community. Comcast is one cable operator that has already joined the Open Networking Lab’s (ON.Lab) CORD initiative. The aim is to add a data centre capability to the cable operators’ access network onto which wireless will be added.
CableLabs is investigating adding high-bandwidth wireless to the cable network using small cells, and the role 5G will play. The cable operators use DOCSIS as their broadband access network technology and it is ideally suited for small cells once these become mainstream, says Clarke: “How you overlay wireless on top of that network is probably where there is going to be some significant opportunities in the next few years.”
One project CableLabs is working on is helping cable operators provision services more efficiently. At present, operators deliver services over several networks: DOCSIS, EPON and in some cases, wireless. CableLabs has been working for a couple of years on simplifying the provisioning process so that the system is agnostic to the underlying networks. “The easiest way to do that is to abstract and virtualize the lower-level functionality; we call that virtual provisioning,” says Clarke.
CableLabs recently published its Virtual Provisioning Interfaces Technical Report on this topic and is developing data models and information models for the various access technologies so that they can be abstracted. The result will be more efficient provisioning of services irrespective of the underlying access technology, says Clarke.
How you overlay wireless on top of that network is probably where there is going to be some significant opportunities in the next few years
SNAPS
CableLabs is also looking at how to virtualise functionality cable operators may deploy near the edge of their networks.
“As the cable network evolves to do different things and adds more capabilities, CableLabs is looking at the technology platform that would do that,” says Clarke.
To this aim, CableLabs has created the SDN-NFV Application development Platform and Stack - SNAPS - which it has contributed to the Open Platform for NFV (OPNFV) group, part of the open source management organisation, The Linux Foundation.
SNAPS is a reference platform to be located near the network edge, and possibly at the cable head-end where cable operators deliver video over their networks. The reference platform makes use of the cloud-based operating system, OpenStack, and other open source components such as OpenDaylight, and is being used to instantiate virtual network functions (VNFs) in a real-time dynamic way. “The classic NFV vision,” says Clarke.
CableLabs' Randy Levensalor says one challenge facing cable operators is that, like telcos, they have separate cloud infrastructures for their services and that impacts their bottom line.
Cable operators are now every bit as informed about NFV and SDN as the telcos are, but they are not out there talking too much about it
“You have one [cloud infrastructure] for business services, one for video delivery and one for IT, and you are operationally less efficient when you have those different stacks,” says Levensalor, lead software architect at CableLabs. “With SNAPS, you bring together all the capabilities that are needed in a reference configuration that can be replicated.”
This platform can support local compute with low latency. "We are not able to say much but there is a longer-term vision for that capability that we’ll develop new applications around,” says Clarke.
Challenges and opportunities
The challenges facing cable operators concerning NFV and SDN are the same as those facing the telcos, such as how to orchestrate and manage virtual networks and do it in a way that avoids vendor lock-in.
“The whole industry wants an open ecosystem where we can buy virtual network functions from one vendor and connect them to virtual network functions and other components from different vendors to create an end-to-end platform with the best capabilities at any given time,” says Clarke.
He also believes that cable operators can move more quickly than telcos because of how they collaborate via CableLabs, their research hub. However, the cable operators' progress is inevitably linked to that of the telcos given they want to use the same SDN and NFV technologies to achieve economies of scale. “So we can’t diverge in the areas that need to be common, but we can move more quickly in areas where the cable network has an inherent advantage, for example in the access network,” says Clarke.
ETSI embraces AI to address rising network complexity
The growing complexity of networks is forcing telecom operators and systems vendors to turn to machine intelligence for help. It has led the European Telecommunications Standards Institute, ETSI, to set up an industry specification group to define how artificial intelligence (AI) can be applied to networking.
“With the advent of network functions virtualisation and software-defined networking, we can see the eventuality that network management is going to get very much more complicated,” says Ray Forbes, convenor of the ETSI Industry Specification Group, Experimental Network Intelligence (ISG-ENI).
Source: ETSI
The AI will not just help with network management, he says, but also with the introduction of services and the more efficient use of network resources.
Visibility of events at many locations in the network will be needed with the deployment of network functions virtualisation (NFV), says Forbes. In current networks, a large switch may serve hundreds of thousands of users but with NFV, virtual network functions will be at many locations. The ETSI group will look at how AI can be used to manage and control this distributed deployment of virtual network functions, says Forbes.
The group’s work has started by inviting interested parties to bring and discuss use cases from which a set of requirements will be generated. In parallel, the group is looking at AI techniques.
The aim is to use computing to derive data from across the network. The data will be analysed, and by having 'context awareness', the machine intelligence will compute various scenarios before presenting the most promising ones for consideration by the network management team. “The process is collecting data, analysing it, testing out various scenarios and then advising people on what would happen in the better scenarios,” says Forbes.
With the advent of NFV and SDN, we can see the eventuality that network management is going to get very much more complicated
ETSI's goal is to make it easier for operators to deploy services quickly, reroute around networking faults, and make better use of networking resources. “In very large cities like Shanghai and Tokyo, where there are populations of 25 million, there is a need for this,” says Forbes. “In London, with about 12 million people, there is still a need but not quite so quickly.”
Operators and system vendors have some understanding of AI but there is a learning curve in bringing more and more AI experts on board, says Forbes: "Hence, we are trying to involve various universities in the research project."
Project schedule
The ISG-ENI's initial document work will be followed by defining the architecture and specifying the parameters needed to measure the network and the 'intelligence' of the scenarios.
“ETSI has a two-year project with the possibility of an extension,” says Forbes, with AI deployed in networks as early as 2019.
Forbes says open-source software to add AI to networks could be available as soon as 2018. Such open-source software will be developed by operators and systems vendors rather than ETSI.
Infinera inches closer to cognitive networking
The second and final part as to how optical networking is becoming smarter
Infinera says it has made it easier for operators to deploy optical links to accommodate traffic growth.
The system vendor says its latest capability, known as Instant Network, also paves the way for autonomous networks that will predict traffic trends and enable capacity as required.
The latest announcement builds on Infinera’s existing Instant Bandwidth feature, introduced in 2012, that uses its photonic integrated circuit (PIC) technology.
Instant Bandwidth exploits the fact that all five 100-gigabit wavelengths of a line card hosting Infinera’s 500-gigabit PIC are lit even though an operator may only need a subset of the 100-gigabit wavelengths. Using Instant Bandwidth, extra capacity can be added to a link - until all five wavelengths are used - in a matter of hours.
The technology allows 100-gigabit wavelengths to be activated in minutes, says Geoff Bennett, director, solutions and technology at Infinera (pictured). It takes several hours due to the processing time for the operator to raise a purchasing order for the new capacity and get it signed off.
Instant Bandwidth has been enhanced since its introduction. Infinera has introduced its latest generation 2.4 terabit PIC which is also sliceable. With a sliceable PIC, individual wavelengths can be sent to different locations using reconfigurable optical add-drop multiplexer (ROADM) technology within the network.
Another feature added is time-based Instant Bandwidth. This allows an operator to add extra capacity without first raising a purchase order. Paying for the extra capacity is dealt with at a later date. This feature has already benefited operators that have experienced a fibre cut and have used Instant Bandwidth to reroute traffic.
Infinera says over 70 of its customers use Instant Bandwidth. These include half of its long-haul customers, its top three submarine network customers and over 60 percent of its data centre interconnect players that use its Cloud Xpress and XTS products. Some of its data centre interconnect customers request boxes with all the licences already activated, says Bennett.
The internet content providers are banging the drum for cognitive networking
Instant Network
Now, with the Instant Network announcement, Infinera has added a licence pool and moveable licences. The result is that an operator can add capacity in minutes rather than hours by using its pool of prepaid licenses.
Equally, if an operator wants to reroute a 100-gigabit or 200-gigabit wavelength to another destination, it can transfer the same licence from the original end-point to the new one.
“They [operators] can activate capacity when the revenue-generating service asks for it,” says Bennett.
Another element of Instant Network still to be introduced is the Automated Capacity Engineering that is part of Infinera’s Xceed software.
Source: Infinera
“Automated Capacity Engineering will be an application that runs on Xceed,” says Bennett. The Automated Capacity Engineering is an application running on the OpenDaylight open source software-defined networking (SDN) controller that takes advantage of plug-ins that Infinera has added to the Xceed platform such as multi-layer path computation and traffic monitoring.
Using this feature, the SDN orchestrator can request a 100 Gigabit Ethernet private line, for example. If there is insufficient capacity, the Automated Capacity Engineering app will calculate the most cost-effective path and install the necessary licences at the required locations, says Bennett.
“We think this is leading the way to cognitive networking,” he says. “We have the software foundation and the hardware foundation for this.”
Networks that think
With a cognitive network, data from the network is monitored and fed to a machine learning algorithm to predict when capacity will be exhausted. New capacity can then be added in a timely accordingly.
Bennett says internet content providers, the likes of Google, Microsoft and Facebook, will all deploy such technology in their networks.
Being consumers of huge amounts of bandwidth, they will be the first adopters. Wholesale operators which also serve the internet content providers will likely follow. Traditional telecom operators with their more limited traffic growth will be the last to adopt such technology.
But cognitive networking is not yet ready. “The machine learning algorithms are still basic,” says Bennett. “But the biggest thing that is missing is the acceptance [of such technology] by network operations staff.”
However, this is not an issue with the internet content providers. “They are banging the drum for cognitive networking,” says Bennett.
Part 1: Ciena's Liquid Spectrum, click here
The white box concept gets embraced at the optical layer
Brandon Collings
White boxes have arisen to satisfy the data centre operators’ need for simple building-block functions in large number that they can direct themselves.
“They [data centre operators] started using very simple white boxes - rather simple functionality, much simpler than the large router companies were providing - which they controlled themselves using software-defined networking orchestrators,” says Brandon Collings, CTO of Lumentum.
Such platforms have since evolved to deliver high-performance switching, controlled by third-party SDN orchestrators, and optimised for the simple needs of the data centre, he says. Now this trend is moving to the optical layer where the same flexibility of function is desired. Operators would like to better pair the functionality that they are going to buy with the exact functionality they need for their network, says Collings.
“There is no plan to build networks with different architectures to what is built today,” he says. “It is really about how do we disaggregate conventional platforms to something more flexible to deploy, to control, and which you can integrate with control planes that also manage higher layers of the network, like OTN and the packet layer.”
White box products
Lumentum has a background in integrating optical functions such as reconfigurable optical add/drop multiplexers (ROADMs) and amplifiers onto line cards, known as its TrueFlex products. “That same general element is now the element being demanded by these white box strategies, so we are putting them in pizza boxes,” says Collings.
At OFC, Lumentum announced several white box designs for linking data centres and for metro applications. Such designs are for large-scale data centre operators that use data centre interconnect platforms. But several such operators also have more complex, metro-like optical networking requirements. Traditional telcos such as AT&T are also interested in pursuing the approach.
The first Lumentum white box products include terminal and line amplifiers, a dense WDM multiplexer/ demultiplexer and a ROADM. These hardware boxes come with open interfaces so that they can be controlled by an SDN orchestrator and are being made available to interested parties.
OpenFlow, which is used for electrical switches in the data centre, could be use with such optical white boxes. Other more likely software are the Restconf and Netconf protocols. “They are just protocols that are being defined to interface the orchestrator with a collection of white boxes,” says Collings.
Lumentum’s mux-demux is defined as a white box even though it is passive and has no power or monitoring requirements. That is because the mux-demux is a distinct element that is not part of a platform.
AT&T is exploring the concept of a disaggregated ROADM. Collings says a disaggregated ROADM has two defining characteristics. One is that the hardware isn’t required to come with a full network control management system. “You can buy it and operate it without buying that same vendor’s control system,” he says. The second characteristic is that the ROADM is physically disaggregated - it comes in a pizza box rather than a custom, proprietary chassis.
There remains a large amount of value between encompassing optical hardware in a pizza box to delivering an operating network
Lumentum: a systems vendor?
The optical layer white box ecosystem continues to develop, says Collings, with many players having different approaches and different levels of ‘aggressiveness’ in pursuing the concept. There is also the issue of the orchestrators and who will provide them. Such a network control system could be written by the hyper-scale data centre operators or be developed by the classical network equipment manufacturers, says Collings.
Collings says selling pizza boxes does not make Lumentum a systems vendor. “There is a lot of value-add that has to happen between us delivering a piece of hardware with simple open northbound control interfaces and a complete deployed, qualified, engineered system.”
Control software is needed as is network engineering; value that traditional system vendors have been adding. “That is not our expertise; we are not trying to step into that space,” says Collings. There remains a large amount of value between encompassing optical hardware in a pizza box to delivering an operating network, he says.
This value and how it is going to be provided is also at the core of an ongoing industry debate. “Is it the network provider or the people that are very good at it: the network equipment makers, and how that plays out.”
Lumentum’s white box ROADM was part of an Open Networking Lab proof-of-concept demonstration at OFC.
OFC 2016: a sample of the technical paper highlights
Here is a small sample of the technical paper highlights being presented at the conference.
Doubling core network capacity
Microsoft has conducted a study measuring the performance of its North American core backbone network to determine how the use of bandwidth-variable transceivers (BVTs) could boost capacity.
The highest capacity modulation scheme suited for each link from the choice of polarisation-multiplexed, quadrature phase-shift keying (PM-QPSK), polarisation-multiplexed, 8 quadrature amplitude modulation (PM-8QAM) and PM-16QAM can then be used.
By measuring the signal (Q-factor) for all its PM-QPSK based 100 gigabit links, Microsoft's study found that network capacity could be increased by 70 percent using BVTs. Equally, having the ability to increase capacity in 25-gigabit increments would increase capacity by a further 16 percent while a finer resolution of 1-gigabit would add an extra 13 percent.
Microsoft says such tuning of links need not be done in real time but rather when a link is commissioned or undergoing maintenance.
[paper M2J.2]
Architecting a new metro
How can operators redesign their metro network to enable rapid service innovation? This is the subject of a joint paper from AT&T, the Open Networking Lab and Stanford University. The work is part of a programme dubbed CORD to redesign the central office as a data centre using commodity hardware and open software to enable the rapid scaling of services. In particular, OpenFlow-enabled white boxes, the Open Network Operating System (ONOS) - a software-defined networking (SDN) operating system, and network control and management applications are used.
As part of CORD, three legacy telecom devices - optical line termination (OLT), customer premises equipment (CPE), and broadband network gateways (BNG) - have been virtualised and implemented on servers.
The paper details how a single SDN control plane based on ONOS is used to create a converged packet-optical metro network and how its support for bandwidth on-demand and automatic restoration at the optical level is used for enterprise connectivity and video distribution services.
The paper also discusses how the metro architecture supports 'disaggregated' reconfigurable optical add/ drop multiplexers (ROADMs). By disaggregating a chassis-based ROADM into commodity components, an operator can size its infrastructure as required and grow it with demand, the paper says.
[paper Th1A.7]
400 gigabit single-carrier transmission
Nokia Bell Labs reports work on 400 gigabit-per-second (Gbps) single-carrier optical transmission over submarine distances. The attraction of adopting 400 gigabit single-carrier transmission as that it is the most efficient way to reduce the cost-per-bit of optical transmission systems.
The Bell Labs' paper reviews state-of-the-art 400 gigabit single-channel transmissions over 6,000km and greater distances, and discusses the tradeoffs between such variables as symbol rate, modulation and forward error correction (FEC) schemes.
400Gbps single-carrier submarine transmission is likely in the next few years
PM-16QAM is proposed as a promising modulation scheme to achieve beyond 6,000km distances and a spectral efficiency exceeding 5b/s/Hz. But this requires a symbol rate of greater than 60 gigabaud to accommodate the 20 percent overhead FEC. Pulse-shaping at the transmitter is also used.
Exploring the receiver performance with the varying symbol rate/ FEC overhead, Bell Labs reports that the best tradeoff between coding gain and implementation penalties is 64 gigabaud and 27.3% overhead. It concludes that single-carrier 400Gbps submarine transmission is likely in the next few years.
[paper Th1B.4]
Silicon modulator for CFP2-ACOs
Cisco has developed a compact flip-chip assembly that combines a silicon photonics modulator and a silicon germanium BiCMOS Mach-Zehnder modulator driver. Such an assembly forms the basis for low-cost advanced coherent optical transceivers such as the CFP2-ACO.
Cisco has demonstrated the assembly operating at 128.7Gbps using PM-QPSK and 257.3Gbps using PM-16QAM. Cisco believes this is the first demonstration of transmission at 257.3Gbps using PM-16QAM over 1,200km of standard single-mode fibre using a silicon photonics-based device.
The modulator has also been demonstrated operating at 321.4Gbps using PM-16QAM transmission and a 20 percent FEC overhead, the highest bit rate yet achieved using a silicon-photonics based transmitter, claims Cisco.
Cisco is already using CFP2-ACO modules as part of its NCS 1002 data centre interconnect platform that implement PM-16QAM and deliver 250 gigabit due to the use of a higher baud rate than the 32 gigabaud used for existing 100-gigabit coherent systems.
[paper Th1F.2]
Flexible Ethernet to exploit line-side efficiencies
Given how the optical network network is starting to use adaptive-rate interfaces, a paper from Google asks how the client side can benefit from such line-side flexibility.
The paper points out that traditional DWDM transport equipment typically multiplexes lower-rate client ports but that this doesn't apply to network operators that manage their own data centres. Here, traffic is exclusively packet-based from IP routers and typically matches the line rate. This is why data centre interconnect platforms have become popular as they require limited grooming capability.
Google highlights how Flexible Ethernet (FlexE), for which the Optical Internetworking Forum has just defined an Implementation Agreement for, combined with data centre interconnect equipment is an extremely effective combination.
FlexE supports Ethernet MAC rates independent of the Ethernet physical layer rate being used. Google shows examples of how using FlexE, sub client rates can match the line-side rate as well as how multiple client ports can support a higher speed router logical port.
The paper concludes that combining FlexE with data centre interconnect results in a low cost, low power, compact design that will enable Internet content providers to scale their networks.
[paper W4G.4]
ECOC 2015: Reflections
Valery Tolstikhin, head of a design consultancy, Intengent
ECOC was a big show and included a number of satellite events, such as the 6th European Forum on Photonic Integration, the 3rd Optical Interconnect in Data Center Symposium and Market Focus, all of which I attended. So, lots of information to digest.
My focus was mainly on data centre optical interconnects and photonic integration.
Data centre interconnects
What became evident at ECOC is that 50 Gig modulation and the PAM-4 modulation format will be the basis of the next generation (after 100 Gig) data centre interconnect. This is in contrast to the current 100 Gig non-return-to-zero (NRZ) modulation using 25 Gig lanes.
This paves the way towards 200 Gig (4 x PAM-4 lanes at 25 Gig) and 400 Gig (4 x PAM-4 lanes at 50 Gig) as a continuation of quads of 4 x NRZ lanes at 25 Gig, the state-of-the-art data centre interconnect still to take off in terms of practical deployment.
The transition from 100 Gig to 400 Gig seems to be happening much faster than from 40Gig to 100 Gig. And 40 Gig serial finally seems to have gone; who needs 40 Gig when 50 Gig is available?
Another observation is that despite the common agreement that future new deployments should use single-mode fibre rather than multi-mode fibre, given the latter’s severe reach limitation that worsens with modulation speed, the multi-mode fibre camp does not give up easily.
That is because of the tons of multi-mode fibre interconnects already deployed, and the low cost of gallium arsenide 850 nm VCSELs these links use. However, the spectral efficiency of such interconnects is low, resulting in high multi-mode fibre count and the associated cost. This is a strong argument against such fibre.
Now, a short-wave WDM (SWDM) initiative is emerging as a partial solution to this problem, led by Finisar. Both OM3 and OM4 multi-mode fibre can be used, extending link spans to 100m at 25 Gig speeds.
Single mode fibre 4 x 25 Gig QSFP28 pluggables with a reach of up to 2 km, which a year ago were announced with some fanfare, seems to have become more of a commodity.
The SWDM Alliance was announced just before ECOC 2015, with major players like Finisar and Corning on board, suggesting this is a serious effort not to be ignored by the single mode fibre camp.
Lastly, single mode fibre 4 x 25 Gig QSFP28 pluggables with a reach of up to 2 km, which a year ago were announced with some fanfare, seems to have become more of a commodity. Two major varieties – PSM and WDM – are claimed and, probably shipping, by a growing number of vendors.
Since these are pluggables with fixed specs, the only difference from the customer viewpoint is price. That suggests a price war is looming, as happens in all massive markets. Since the current price still are an order of magnitude or more above the target $1/Gig set by Facebook and the like, there is still a long way to go, but the trend is clear.
This reminds me of that I’ve experienced in the PON market: a massive market addressed by a standardised product that can be assembled, at a certain time, using off-the-shelf components. Such a market creates intense competition where low-cost labour eventually wins over technology innovation.
Photonic integration
Two trends regarding photonic integration for telecom and datacom became clear at ECOC 2015.
One positive development is an emerging fabless ecosystem for photonic integrated circuits (PICs), or at least an understanding of a need for such. These activities are driven by silicon photonics which is based on the fabless model since its major idea is to leverage existing silicon manufacturing infrastructure. For example, Luxtera, the most visible silicon component vendor, is a fabless company.
There are also signs of the fabless ecosystem building up in the area of III-V photonics, primarily indium-phosphide based. The European JePPIX programme is one example. Here you see companies providing foundry and design house services emerging, while the programme itself supports access to PIC prototyping through multi-project wafer (MPW) runs for a limited fee. That’s how the ASIC business began 30 to 40 years ago.
A link to OEM customers is still a weak point, but I see this being fixed in the near future. Of course, Intengent, my design house company, does just that: links OEM customers and the foundries for customised photonic chip and PIC development.
As soon as PICs give a system advantage, which Infinera’s chips do, they become a system solution enabler, not merely ordinary components made a different way
The second, less positive development, is that photonic integration continues to struggle to find applications and markets where it will become a winner. Apart from devices like the 100 Gig coherent receiver, where phase control requirements are difficult to meet using discretes, there are few examples where photonic integration provides an edge.
Even a 4 x 25 Gig assembly using discrete components for today’s 100 Gig client side and data centre interconnect has been demonstrated by several vendors. It then becomes a matter of economies of scale and cheap labour, leaving little space for photonic integration to play. This is what happened in the PON market despite photonic integrated products being developed by my previous company, OneChip Photonics.
On a flip side, the example of Infinera shows where the power of photonic integration is: its ability to create more complicated PICs as needed without changing the technology.
One terabit receiver and transmitter chips developed by Infinera are examples of complex photonic circuits, simply undoable by means of an optical sub-assembly. As soon as PICs give a system advantage, which Infinera’s chips do, they become a system solution enabler, not merely ordinary components made a different way.
However, most of the photonic integration players - silicon photonics and indium phosphide alike - still try to do the same as what an optical sub-assembly can do, but more cheaply. This does not seem to be a winning strategy.
And a comment on silicon photonics. At ECOC 2015, I was pleased to see that, finally, there is a consensus that silicon photonics needs to aim at applications with a certain level of complexity if it is to provide any advantage to the customer.
Silicon photonics must look for more complex things, maybe 400 Gig or beyond, but the market is not there yet
For simpler circuits, there is little advantage using photonic integration, least of all silicon photonics-based ones. Where people disagree is what this threshold level of complexity is. Some suggest that 100 Gig optics for data centres is the starting point but I’m unsure. There are discrete optical sub-assemblies already on the market that will become only cheaper and cheaper. Silicon photonics must look for more complex things, maybe 400 Gig or beyond, but the market is not there yet.
One show highlight was the clear roadmap to 400 Gig and beyond, based on a very high modulation speed (50 Gig) and the PAM-4 modulation format, as discussed. These were supported at previous events, but never before have I seen the trend so clearly and universally accepted.
What surprised me, in a positive way, is that people have started to understand that silicon photonics does not automatically solve their problems, just because it has the word silicon in its name. Rather, it creates new challenges, cost efficiency being an important one. The conditions for cost efficient silicon photonics are yet to be found, but it is refreshing that only a few now believe that the silicon photonics can be superior by virtue of just being ‘silicon’.
I wouldn’t highlight one thing that I learned at the show. Basically, ECOC is an excellent opportunity to check on the course of technology development and people’s thoughts about it. And it is often better seen and felt on the exhibition floor than attending the conference’s technical sessions.
For the coming year, I will continue to track data centre interconnect optics, in all its flavours, and photonic integration, especially through a prism of the emerging fabless ecosystem.
Vishnu Shukla, distinguished member technical staff in Verizon’s network planning group.
There were more contributions related to software-defined networking (SDN) and multi-layer transport at ECOC. There were no new technology breakthroughs as much as many incremental evolutions to high-speed optical networking technologies like modulation, digital signal processors and filtering.
I intend to track technologies and test results related to transport layer virtualisation and similar efforts for 400 Gig-and-beyond transport.
Vladimir Kozlov, CEO and founder of LightCounting
I had not attended ECOC since 2000. It is a good event, a scaled down version of OFC but just as productive. What surprised me is how small this industry is even 15 years after the bubble. Everything is bigger in the US, including cars, homes and tradeshows. Looking at our industry on the European scale helps to grasp how small it really is.
What is the next market opportunity for optics? The data centre market is pretty clear now, but what next?
Listening to the plenary talk of Sir David Paine, it struck me how infinite technology is. It is so easy to get overexcited with the possibilities, but very few of the technological advances lead to commercial success.
The market is very selective and it takes a lot of determination to get things done. How do start-ups handle this risk? Do people get delusional with their ideas and impact on the world? I suspect that some degree of delusion is necessary to deal with the risks.
As for issues to track in the coming year, what is the next market opportunity for optics? The data centre market is pretty clear now, but what next?
Nuage uses SDN to aid enterprise connectivity needs
"Across the WAN and out to the branch, the context is increasingly complicated, with the need to deliver legacy and cloud applications to users - and sometimes customers - that are increasingly mobile, spanning several networks," says Brad Casemore, research director, data centre networks at IDC. These networks can include MPLS, Metro Ethernet, broadband and 3G and 4G wireless.
The data centre is a great microcosm of the network - Houman Modarres
At present, remote offices use custom equipment that require a visit from an engineer. In contrast, VNS uses SDN technology to deliver enterprise services to a generic box, or software that runs on the enterprise's server. The goal is to speed up the time it takes an enterprise to set up or change their business services at a remote site, while also simplifying the service provider's operations.
What has been done
Nuage designed its SDN-enabled connectivity products from the start for use in the data centre and beyond. "The data centre is a great microcosm of the network," says Modarres. "But we designed it in such a way that the end points could be flexible, within and across data centres but also anywhere."
Nuage uses open protocols like OpenFlow to enable the control plane to talk to any device, while its software agents that run on a server can work with any hypervisor. The control plane-based policies are downloaded to the end points via its SDN controller.
Using VNS, services can be installed without a visit from a specialist engineer. A user powers up the generic hardware or server and connects it to the network whereby policies are downloaded. The user enters a sent code that enables their privileges as defined by the enterprise's policies.
"Just as in the data centre, there is a real need for greater agility through automation, programmability, and orchestration," says IDC's Casemore. "One could even contend that for many enterprises, the pain is more acutely felt on the WAN, especially as they grapple with how to adapt to cloud and mobility."
Extending the connectivity end points beyond the data centre has required Nuage to bolster security and authentication procedures. Modarres points out that data centers and service provider central offices are secured environments; a remote office that could be a worker's home is not.
"You need to do authentication differently and IPsec connections are needed for security, but what if you unplug it? What if it is stolen?" he says. "If someone goes to the bank and steals a router, are they a bank branch now?"
To address this, once a remote office device is unplugged for a set time - typically several minutes - its configuration is reset. Equally, when a router is deliberated unplugged, for example during an office move, if notification is given, the user receives a new authentication code on the move's completion and the policies are restored.
Nuage's virtualised services platform comprise three elements: the virtualised services directory (VSD), virtualised services controller (VSC) - the SDN controller - and the virtual routing and switching module (VR&S).
"The only thing we are changing is the bottom layer, the network end point, which used to be in the data centre as the VR&S, and is now broken out of the data centre, as in the network services gateway, to be anywhere," says Modarres. "The network services gateway has physical and virtual form factors based on standard open compute."
Nuage is finding that businesses are benefitting from an SDN approach in surprising ways.
The company cites banks as an example that are forced by regulation to ensure that there are no security holes at their remote locations. One bank with 400 branches periodically sends individuals to each to check the configuration to ensure no human errors in its set-up could lead to a security flaw. With 400 branches, this procedure takes months and is costly.
With SDN and its policy-level view of all locations - what each site and what each group can do - there are predefined policy templates. There may be 10, 20 or 30 templates but they are finite, says Modarres: "At the push of a button, an organisation can check the templates, daily if needed".
This is not why a bank will adopt SDN, says Modarres, but the compliance department will be extremely encouraging for the technology to be used, especially when it saves the department millions of dollars in ensuring regulatory compliance.
Nuage Networks says it has 15 customer wins and 60 ongoing trials globally for its products. Customers that have been identified include healthcare provider UPMC, financial services provider BBVA, cloud provider Numergy, hosting provider OVH, infrastructure providers IDC Frontier and Evonet, and telecom providers TELUS and NTT Communications.