Deutsche Telekom explains its IP-over-DWDM thinking

Telecom operators are always seeking better ways to run their networks. In particular, operators regularly scrutinise how best to couple the IP layer with their optical networking infrastructure.

The advent of 400-gigabit coherent modules that plug directly into an IP router is one development that has caught their eye.

Placing dense wavelength division multiplexing (DWDM) interfaces directly onto an IP router allows the removal of a separate transponder box and its interfacing.

IP-over-DWDM is not a new concept. However, until now, operators have had to add a coherent line card, taking up valuable router chassis space.

Werner Weiershausen

Now, with the advent of compact 400-gigabit coherent pluggables developed for the hyperscalers to link their data centres, telecom operators have realised that such pluggables also serve their needs.

BT will start rolling out IP-over-DWDM in its network this year, while Deutsche Telekom has analysed the merits of IP-over-DWDM.

"The adoption of IP-over-DWDM is the subject of our techno-economical studies," says Werner Weiershausen, senior architect for the transport network at Deutsche Telekom.

Network architecture

Deutsche Telekom's domestic network architecture comprises 12 large nodes where IP and OTN backbones align with the underlying optical networking infrastructure. These large nodes - points of presence - can be over 1,000km apart.

Like many operators, Deutsche Telekom has experienced IP annual traffic growth of 35 per cent. The need to carry more traffic without increasing costs has led the operators to adopt coherent technology, with the symbol rate rising with each new generation of optical transport technology. 

A higher channel bit rate sends more data over an optical wavelength. The challenge, says Weiershausen, is maintaining the long-distance reaches with each channel rate hike.

Deutsche Telekom's in-house team forecasts that IP traffic growth will slow down to a 20 per cent annual growth rate and even 16 per cent in future.

Weiershausen says this is still to be proven but that if annual traffic growth does slow down to 16-20 per cent, bandwidth growth issues will remain; it is just that they can be addressed over a longer timeframe.

Bandwidth and reach are long-haul networking issues. Deutsche Telekom's metro networks, which are horse-shoe-shaped, have limited spans overall.

"For metro, our main concern is to have the lowest cost-per-bit because we are fibre- and spectrum-rich, and even a single DWDM fibre pair per metro horseshoe ring offer enough bandwidth headroom," says Weiershausen. "So it's easy; we have no capacity problem like the backbone. Also there, we are fibre-rich but can avoid the costly activation of multiple parallel fibre trunks."

IP-over-DWDM

IP-over-DWDM is increasingly associated with adding pluggable optics onto an IP core router.

"This is what people call IP-over-DWDM, or what Cisco calls it hop-by-hop approach," says Dr Sascha Vorbeck, head of strategy and architecture IP-core & transport networks at Deutsche Telekom.

Dr Sascha Vorbeck

Cisco's routed optical networking - its term for the hop-by-hop approach - uses the optical layer for point-to-point connections between IP routers. As a result, traffic switching and routing occur at the IP layer rather than the optical layer, where optical traffic bypass is performed using reconfigurable optical add/drop multiplexers (ROADMs).

Routed optical networking also addresses the challenge of the rising symbol rate of coherent technology, which must maintain the longest reaches when passing through multiple ROADM stages.

Deutsche Telekom says it will not change its 12-node backbone network to accommodate additional routing stages.

"We will not change our infrastructure fundamentally because this is costly," says Weiershausen. "We try to address this bandwidth growth with technology and not with the infrastructure change."

Deutsche Telekom's total cost-of-ownership analysis highlights that optical bypass remains attractive compared to a hop-by-hop approach for specific routes.

However, the operator has concluded that the best approach is to have both: some hop-by-hop where it suits its network in terms of distances but also using optical bypass for longer links using either ROADM or static bypass technology.

"A mixture is the optimum from our total cost of ownership calculation," says Weiershausen. "There was no clear winner."

Strategy 

Deutsche Telecom favours coherent interfaces on its routers for its network backbone because it wants to simplify its network. In addition, the operator wants to rid its network of existing DWDM transponders and their short reach - 'grey' - interfaces linking the IP router to the DWDM transponder box.

"They use extra power and are an extra capex [capital expenditure] cost," says Weiershausen. "They are also an additional source of failures when you have in-line several network elements. That said, heat dissipation of long-reach coherent optical DWDM interfaces limited the available IP router interfaces that could have been activated in the past.

For example, a decade ago, Deutsche Telecom tried to use IP-over-DWDM for its backbone network but had to step back to use an external DWDM transponder box due to heat dissipation problems.

The situation may have changed with modern router and optical interface generations, but this is under further study by Deutsche Telecom and is an essential prerequisite for its evolution roadmap.

Deutsche Telecom is still using traditional DWDM equipment between the interconnection of IP routers with grey interfaces. Deutsche Telecom undertook an evaluation in 2020 and calculated a traditional DWDM network versus a hop-by-hop approach. Then, the hop-by-hop method was 20 per cent more expensive. Deutsche Telecom plans to redo the calculations to see if anything has changed.

The operator has yet to decide whether to adopt ZR+ coherent pluggable optics and a hop-by-hop approach or use more advanced larger coherent modules in its routers. "This is not decided yet and depends on pricing evolution," says Weiershausen.

With the volumes expected for pluggable coherent optics, the expectation is they will have a notable price advantage compared to traditional high-performance coherent interfaces.

But Deutsche Telekom is still determining, believing that conventional coherent interfaces may also come down markedly in price.

SDN controller

Another issue for consideration with IP-over-DWDM is the software-defined networking (SDN) controller.

IP router vendors offer their SDN controllers, but there also is a need for working with third-party SDN controllers.

For example, Deutsche Telekom is a member of the OpenROADM multi-source agreement and has pushed for IP-over-DWDM to be a significant application of the MSA.

But there are disaggregation issues regarding how a router's coherent optical interfaces are controlled. For example, are the optical interfaces overseen and orchestrated by the OpenROADM SDN controller and its application programming interface (API) or is the SDN controller of each IP router vendor responsible for steering the interfaces?

Deutsche Telekom says that a compromise has been reached for the OpenROADM MSA whereby the IP router vendors' SDN controllers oversee the optics but that for the solution to work, information is exchanged with the OpenROADM's SDN controller.

"That way, the path computation engine (PCE) of the optical network layer, including the ROADMs, can calculate the right path to network the traffic. "Without information from the IP router, it would be blind; it would not work," says Weiershausen.

Automation

Weiershausen says it is not straightforward to say which approach - IP-over-DWDM or a boundary between the IP and optical layers - is easier to automate.

"Principally, it is the same in terms of the information model; it is just that there are different connectivity and other functionalities [with the two approaches]," says Weiershausen.

But one advantage of a clear demarcation between the layers is the decoupling of the lifecycles of the different equipment.

Fibre has the longest lifecycle, followed by the optical line system, with IP routers having the shortest of the three, with new generation equipment launched every few years.

Decoupling and demarcation is therefore a good strategy here, notes Weiershausen.