Cyan's stackable optical rack for data centre interconnent
"The drivers for these [data centre] guys every day of the week is lowest cost-per-gigabit"
Joe Cumello
The amount of traffic moved between data centres can be huge. According to ACG Research, certain cloud-based applications shared between data centres can require between 40 to 500 terabits of capacity. This could be to link adjacent data centre buildings to appear as one large logical one, or connect data centres across a metro, 20 km to 200 km apart. For data centres separated across greater distances, traditional long-haul links are typically sufficient.
Cyan says it developed the N-series platform following conversations conducted with internet content providers over the last two years. "We realised that the white box movement would make its way into the data centre interconnect space," says Cumello.
White box servers and white box switches, manufactured by original design manufacturers (ODMs), are already being used in the data centre due to their lower cost. Cyan is using a similar approach for its N-Series, using commercial-off-the-shelf hardware and open software.
"The drivers for these [data centre] guys every day of the week is lowest cost-per-gigabit," says Cumello.
N-Series platform
Cyan's N-Series N11 is a 1-rack-unit (1RU) box that has a total capacity of 800 Gigabit-per-second (Gbps). The 1RU shelf comprises two units, each using two client-side 100Gbps QSFP28s and a line-side interface that supports 100 Gbps coherent transmission using PM-QPSK, or 200 Gbps coherent using PM-16QAM. The transmission capacity can be traded with reach: using 100 Gbps, optical transmission up to 2,000 km is possible, while capacity can be doubled using 200 Gbps lightpaths for links up to 600 km. Cyan is using Clariphy's CL20010 coherent transceiver/ framer chip. Stacking 42 of the 1RUs within a chassis results in an overall capacity - client side and line side - of 33.6 terabit.
There is a whole ecosystem of companies competing to drive better capacity and scale
The N-Series N11 uses a custom line-side design but Cyan says that by adopting commercial-off-the-shelf design, it will benefit from the pluggable line-side optical module roadmap. The roadmap includes 200 Gbps and 400 Gbps coherent MSA modules, pluggable CFP2 and CFP4 analogue coherent optics, and the CFP2 digital coherent optics that also integrates the DSP-ASIC.
"There is a whole ecosystem of companies competing to drive better capacity and scale," says Cumello. "By using commercial-off-the-shelf technology, we are going to get to better scale, better density, better energy efficiency and better capacity."
To support these various options, Cyan has designed the chassis to support 1RU shelves with several front plate options including a single full-width unit, two half-width ones as used for the N11, or four quarter-width units.
Open software
For software, the N-series platform uses a Linux networking operating system. Using Linux enables third-party applications to run on the N-series, and enables IT staff to use open source tools they already know. "The data centre guys use Linux and know how to run servers and switches so we have provided that kind of software through Cyan's Linux," says Cumello. Cyan has also developed its own networking applications for configuration management, protocol handling and statistics management that run on the Linux operating system.
The open software architecture of the N-Series. Also shown are the two units that make up a rack. Source: Cyan.
"We have essentially disaggregated the software from the hardware," says Cumello. Should a data centre operator chooses a future, cheaper white box interconnect product, he says, Cyan's applications and Linux networking operating system will still run on that platform.
The N-series will be available for customer trials in the second quarter and will be available commercially from the third quarter of 2015.
Oclaro demonstrates flexible rate coherent pluggable module
- The CFP2 coherent optical module operates at 100 and 200 Gig
- Samples are already with customers, with general availability in the first half of 2015
- Oclaro to also make more CFP2 100GBASE-LR4 products
The CFP2 is not just used in metro/ regional networks but also in long-haul applications
Robert Blum
The advent of a pluggable CFP2, capable of multi-rate long-distance optical transmission, has moved a step closer with a demonstration by Oclaro. The optical transmission specialist showed a CFP2 transmitting data at 200 Gigabits-per-second.
The coherent analogue module demonstration, where the DSP-ASIC resides alongside rather than within the CFP2, took place at ECOC 2014 held in September at Cannes. Oclaro showcased the CFP2 to potential customers in March, at OFC 2014, but then the line side module supported 100 Gig only.
"What has been somewhat surprising to us is that the CFP2 is not just used in metro/ regional networks but also in long-haul applications," says Robert Blum, director of strategic marketing at Oclaro. "We are also seeing quite significant interest in data centre interconnect, where you want to get 400 Gig between sites using two CFP2s and two DSPs." Oclaro says that the typical distances are from 200km to 1,000km.
The CFP2 achieves 200 Gig using polarisation multiplexing, 16-quadrature amplitude modulation (PM-16-QAM) while working alongside ClariPhy's merchant DSP-ASIC. ClariPhy announced at ECOC that it is now shipping its 200 Gig LightSpeed-II CL20010 coherent system-on-chip, implemented using a 28nm CMOS process.
"One of the beauties of an analogue CFP2 is that it works with a variety of DSPs," says Blum. Other merchant coherent DSPs are becoming available, while leading long-haul optical equipment vendors have their own custom coherent DSPs.
Oclaro's CFP2, even when operating at 200 Gig, falls within the 12W module's power rating. "One of the things you need to have for 200 Gig is a linear modulator driver, and such drivers consume slightly more power [200mW] than limiting modulator drivers [used for 100 Gig only]," says Blum.
Oclaro will offer two CFP2 line-side variants, one with linear drivers and one using limiting ones. The limiting driver CFP2 will be used for 100 Gig only whereas the linear driver CFP2 supports 100 Gig PM-QPSK and 200 Gig PM-16-QAM schemes. "Some customers prefer the simplicity of a limiting interface; for the linear interface you have to do more calibration or set-up," says Blum. "Linear also allows you to do pre-emphasis of the signal path, from the DSP all the way to the modulator." Pre-emphasis is used to compensate for signal path impairments.
By consuming under 12W, up to eight line-side CFP2 interfaces can fit on a line card, says Blum, who also stresses the CFP2 has a 0dBm output power at 200 Gig. Achieving such an output power level means the 200 Gig signal is on a par with 100 Gig wavelengths. "When you launch a 200 Gig signal, you want to make sure that there is not a big difference between signals," says Blum.
To achieve the higher output power, the micro integrable tunable laser assembly (micro-iTLA) includes a semiconductor optical amplifier (SOA) with the laser, while SOAs are also added to the Mach–Zehnder modulator chip. "That allows us to compensate for some of the [optical] losses," says Blum.
Customers received first CFP2 samples in May, with the module currently at the design validation stage. Oclaro expects volume shipments to begin in the first half of 2015.
100 Gig and the data centre
Oclaro also announced at ECOC that it has expanded manufacturing capacity for its CFP2-based 100GBASE-LR4 10km-reach module.
One reason for the flurry of activity around 100 Gig mid-reach interfaces that span 500m-2km in the data centre is that the 100GBASE-LR4 module is relatively expensive. Oclaro itself has said it will support the PSM-4, CWDM4 and CLR4 Alliance mid-reach 100 Gig interfaces. So why is Oclaro expanding manufacturing of its CFP2-based 100GBASE-LR4?
It is about being pragmatic and finding the most cost-effective solution for a given problem
"There is no clear good solution to get 100 Gig over 500m or 2km right now," says Blum. "CFP2 is here, it is a mature technology and we have made improvements both in performance and cost."
Oclaro has improved its EML design such that the laser needs less cooling, reducing overall power dissipation. The accompanying electronic functions such as clock data recovery have also been redesigned using one IC instead of two such that the CFP2 -LR4's overall power consumption is below 8W.
Demand has been so significant, says Blum, that the company has been unable to meet customer demand. Oclaro expects that towards year-end, it will have increased its CFP2 100GBASE-LR4 manufacturing capacity by 50 percent compared to six months earlier.
"It is about being pragmatic and finding the most cost-effective solution for a given problem," says Blum. "There are other [module] variants that are of interest [to us], such as the CWDM4 MSA that offers a cost-effective way to get to 2km."
2012: The year of 100 Gigabit transponders
“The world is moving to coherent, there is no question about that”
Per Hansen, Oclaro
The 100Gbps module expands the company's coherent offerings. Oclaro is already shipping a 40Gbps coherent module. “The world is moving to coherent, there is no question about that,” says Per Hansen, vice president of product marketing, optical networks solutions at Oclaro.
Why is this significant?
Having a selection of 100Gbps long-haul optical modules will aid the uptake of high-capacity links in the network core. Opnext announced in September its OTM-100 100Gbps coherent optical module, in production from April 2012. And at least one other module maker has worked with ADVA Optical Networking to make its 100Gbps module, a non-coherent design.
The 100Gbps coherent optical modules will enable system vendors without their own technology to enter the marketplace. It also presents those system vendors with their own 100Gbps technology - the likes of Alcatel-Lucent, Ciena, Cisco and Huawei - with a dilemma: do they continue to evolve their products or embrace optical modules?
“These system vendors have developed [100Gbps] in-house to have a strategic differentiator," says Hansen. "But with lower volumes you have a higher cost.” The advent of 100Gbps modules diminishes the strategic advantage of in-house technology while enabling system vendors to benefit from cheaper, more broadly available modules, he says.
What has been done
Oclaro is still developing the MI 8000XM module and has yet to reveal the reach performance of the module: “We want to do many more tests before we share,” says Hansen. The module will meet the Optical Internetworking Forum's (OIF) 100Gbps module maximum power consumption limit of 80W, he says.
The OIF 100 Gigabit module architecture
The NEL DSP chip is the same device that Opnext is using for its 100Gbps module. “A partnership agreement and sourcing arrangement with NEL allows us to come to market with what we think is a very good product at the right time,” says Hansen.
The DSP uses soft-decision forward error correction. Opnext has said this adds 2-3dB to the optical performance to achieve a reach of 1500-1600km before regeneration.
In 2010 Oclaro announced it had invested US $7.5 million in Clariphy Communications as part of the chip company's development of its 100Gbps coherent receiver chip, the CL10010. As part of the agreement, Oclaro will get a degree of exclusivity as a module supplier (at least one other module maker will also benefit).
ClariPhy has said that while it will not be first to market with a 100Gbps ASIC, the CL10010 will be a 28nm CMOS second-generation chip design. To be able to enter the market with a 100Gbps module next year, Oclaro adopted NEL's design which exists now.
Next
Hansen says that the MI 8000XM, which uses a lithium niobate modulator, is designed to achieve maximum reach and optical performance. But future 100Gbps modules will be developed that may use other modulator technologies and be optimised in terms of power or size.
Hansen is also in no doubt that the next speed hike after 100Gbps will be 400Gbps. Like 100Gbps, there will be some early-adopter operators that embrace the technology one or two years before the consensus.
Such a development is still several years away, however, since an industry standard for 400Gbps must be developed which is only expected in 2014 only.
How ClariPhy aims to win over the system vendors
“We can build 200 million logic gate designs”
Reza Norouzian, ClariPhy
ClariPhy is in the camp that believes that the 100 Gigabit-per-second (Gbps) market is developing faster than people first thought. “What that means is that instead of it [100Gbps] being deployed in large volumes in 2015, it might be 2014,” says Reza Norouzian, vice president of worldwide sales and business development at ClariPhy.
Yet the fabless chip company is also glad it offers a 40Gbps coherent IC as this market continues to ramp while 100Gbps matures and overcomes hurdles common to new technology: The 100Gbps industry has yet to develop a cost-effective solution or a stable component supply that will scale with demand.
Another challenge facing the industry is reducing the power consumption of 100Gbps systems, says Norouzian. The need to remove the heat from a 100Gbp design - the ASIC and other components - is limiting the equipment port density achievable. “If you require three slots to do 100 Gig - whereas before you could use these slots to do 20 or 30, 10 Gig lines - you are not achieving the density and economies of scale hoped for,” says Norouzian.
40G and 100G coherent ASICs
ClariPhy has chosen a 40nm CMOS process to implement its 40Gbps coherent chip, the CL4010. But it has since decided to adopt 28nm CMOS for its 100Gbps design – the CL10010 - to integrate features such as soft-decision forward error correction (see New Electronics' article on SD-FEC) and reduce the chip’s power dissipation.
The CL4010 integrates analogue-to-digital and digital-to-analogue converters, a digital signal processor (DSP) and a multiplexer/ demultiplexer on-chip. “Normally the mux is a separate chip and we have integrated that,” says Norouzian.
The first CL4010 samples were delivered to select customers three months ago and the company expects volume production to start by the end of September. The CL4010 also interoperates with Cortina Systems’ optical transport network (OTN) processor family of devices, says the company.
The start-up claims there is strong demand for the CL4010. “When we ask them [operators]: ‘With all the hoopla about 100 Gig, why are you buying all this 40 Gig?’, the answer is that it is a pragmatic solution and one they can ship today,” says Norouzian.
ClariPhy expects 40Gbps volumes to continue to ramp for the next three or four years, partly because of the current high power consumption of 100Gbps. The company says several system vendors are using the CL4010 in addition to optical module customers.
The 28nm 100Gbps CL10010 is a 100 million gate ASIC. ClariPhy acknowledges it will not be first to market with an 100Gbps ASIC but that by using the latest CMOS process it will be well position once volume deployments start from 2014.
ClariPhy is already producing a quad-10Gbps chip implementing the maximum likelihood sequence estimation (MLSE) algorithm used for dispersion compensation in enterprise applications. The device covers links up to 80km (10GBASE-ZR) but the main focus is for 10GBASE-LRM (220m+) applications. “Line cards that used to have four times 10Gbps lanes now are moving to 24 and will use six of these chips,” says Norouzian. The device sits on the card and interfaces with SFP+ or Quad-SFP optical modules.
“The CL10010 is the platform to demonstrate all that we can do but some customers [with IP] will get their own derivatives”
System vendor design wins
The 100Gbps transmission ASIC market may be in its infancy but the market is already highly competitive with clear supply lines to the system vendors.
Several leading system vendors have decided to develop their own ASICs. Alcatel-Lucent, Ciena, Cisco Systems (with the acquisition of CoreOptics), Huawei and Infinera all have in-house 100Gbps ASIC designs.
System vendors have justified the high development cost of the ASIC to get a time-to-market advantage rather than wait for 100Gbps optical modules to become available. Norouzian also says such internally-developed 100Gbps line card designs deliver a higher 100Gbps port density when compared to a module-based card.
Alternatively, system vendors can wait for 100Gbps optical modules to become available from the likes of an Oclaro or an Opnext. Such modules may include merchant silicon from the likes of a ClariPhy or may be internally developed, as with Opnext.
System vendors may also buy 100Gbps merchant silicon directly for their own 100Gbps line card designs. Several merchant chip vendors are targeting the coherent marketplace in addition to ClariPhy. These include such players as MultiPhy and PMC-Sierra while other firms are known to be developing silicon.
Given such merchant IC competition and the fact that leading system vendors have in-house designs, is the 100Gbps opportunity already limited for ClariPhy?
Norouzian's response is that the company, unlike its competitors, has already supplied 40Gbps coherent chips, proving the company’s mixed signal and DSP expertise. The CL10010 chip is also the first publicly announced 28nm design, he says: “Our standard product will leapfrog first generation and maybe even second generation [100Gbps] system vendor designs.”
The equipment makers' management will have to decide whether to fund the development of their own second-generation ASICs or consider using ClariPhy’s 28nm design.
ClariPhy acknowledges that leading system vendors have their own core 100Gbps intellectual property (IP) and so offers vendors a design service to develop their own custom systems-on-chip. For example a system vendor could use ClariPhy's design but replace the DSP core with the system vendor’s own hardware block and software.
Source: ClariPhy Communications
Norouzian says system vendors making 100Gbps ASICs develop their own intellectual property (IP) blocks and algorithms and use companies like IBM or Fujitsu to make the design. ClariPhy offers a similar service while also being able to offer its own 100Gbps IP as required. “The CL10010 is the platform to demonstrate all that we can do,” says Norouzian. “But some customers [with IP] will get their own derivatives.”
The firm has already made such custom coherent devices using customers’ IP but will not say whether these were 40 or 100Gbps designs.
Market view
ClariPhy claims operator interest in 40Gbps coherent is not so much because of its superior reach but its flexibility when deployed in networks alongside existing 10Gbps wavelengths. “You don't have to worry about [dispersion] compensation along routes,” says Norouzian, adding that coherent technology simplifies deployments in the metro as well as regional links.
And while ClariPhy’s focus is on coherent systems, the company agrees with other 100Gbps chip specialists such as MultiPhy for the need for 100Gbps direct-detect solutions for distances beyond 40km. “It is very likely that we will do something like that if the market demand was there,” says Norouzian. But for now ClariPhy views mid-range 100Gbps applications as a niche opportunity.
Funding
ClariPhy raised US $14 million in June. The biggest investor in this latest round was Nokia Siemens Networks (NSN).
An NSN spokesperson says working with ClariPhy will help the system vendor develop technology beyond 100Gbps. “It also gives us a clear competitive edge in the optical network markets, because ClariPhy’s coherent IC and technology portfolio will enable us to offer differentiated and scalable products,” says the spokesperson.
The funding follows a previous round of $24 million in May 2010 where the investors included Oclaro. ClariPhy has a long working relationship with the optical components company that started with Bookham, which formed Oclaro after it merged with Avanex.
“At 100Gbps, Oclaro get some amount of exclusivity as a module supplier but there is another module supplier that also gets access to this solution,” says Norouzian. This second module supplier has worked with ClariPhy in developing the design.
ClariPhy will also supply the CL10010 to the system vendors. “There are no limitations for us to work with OEMs,” he says.
The latest investment will be used to fund the company's R&D effort in 100, 200 and 400Gbps, and getting the CL4010 to production.
Beyond 100 Gig
The challenge at higher data rates that 100Gbps is implementing ultra-large ASICs: closing the timings and laying out vast digital circuitry. This is an area the company has been investing in over the last 18 months. “Now we can build 200 million logic gate designs,” says Norouzian.
Moving from 100Gbps to 200Gbps wavelengths will require higher order modulation, says Norouzian, and this is within the realm of its ASIC.
Going to 400Gbps will require using two devices in parallel. One Terabit transmission however will be far harder. “Going to one Terabit requires a whole new decade of development,” he says.
Further reading:
MultiPhy eyes 40 and 100 Gigabit direct-detect and coherent schemes
MultiPhy's Avi Shabtai (left) and Ronen Weinberg
MultiPhy is developing transceiver designs to boost the transmission performance of metro and long-haul 40 and 100 Gigabit-per-second (Gbps) links. The start-up is aiming its advanced digital signal processing (DSP) chips at direct detection and coherent-based modulation schemes.
“We are the only company, as far as we know, who is doing DSP-based semiconductors for the 40G and 100G direct-detect world,” says Avi Shabtai, CEO of Multiphy.
At 40Gbps the main direct-detection schemes are differential phase-shift keying (DPSK) and differential quadrature phase-shift keying (DQPSK), while at 100Gbps several direct-detect modulation schemes are being considered. “The fact that we are doing DSP at 40G and 100G enables us to achieve much better performance than regular hard-detection technology,” says Shabtai.
Established in 2007, the fabless semiconductor start-up raised US$7.2m in its latest funding round in May. MultiPhy is targeting its physical layer chips at module makers and system vendors. “While there is a clear ecosystem involving optical module companies and systems vendors, there is a lot of overlap,” says Shabtai. “You can find module companies that develop components; you can find system companies that skip the module companies, buying components to make their own line cards.”
MultiPhy’s CMOS chips include high-speed analogue-to-digital converters (ADC) and hardware to implement the maximum-likelihood sequence estimation (MLSE) algorithm. The company is operating the MLSE algorithm at “tens of gigasymbols-per-second”, says Shabtai. “We believe we are the only company implementing MLSE at these speeds.”
MultiPhy's office is alongside Finisar's Israeli headquartersMultiPhy will not disclose the exact sampling rate but says it is sampling at the symbol rate rather than at the Nyquist sampling theorem rate of double the symbol rate. Since commercial ADCs for 100Gbps have been announced that sample at 65Gsample/s, it suggests MultiPhy is sampling at up to half that rate.
MLSE is used to compensate for the non-linear impairments of fibre transmission, to improve overall transmission performance. “We implement an anti-aliasing filter at the input to the ADC and we use the MLSE engine to compensate for impairments due to the low-bandwidth sampling,” says Shabtai.
“There is a good chance that 100Gbps will leapfrog 40Gbps coherent deployments”
Avi Shabtai, MultiPhy
MultiPhy benefits from using one-sample-per-symbol in terms of simplifying the chip design and its power consumption but the MLSE algorithm must counter the resulting distortion. Shabtai claims the result is a significant reduction in power consumption compared to the tradition two-samples-per-symbol approach: “Tens of percent – I won’t say the exact number but it is not 10 percent.”
Other chip companies implementing MLSE designs for optical transmission include CoreOptics, which was acquired by Cisco in May, and Clariphy. (See Oclaro and Clariphy)
Does using MLSE make sense for 40Gbps DPSK and DQPSK?
“If you use DSP for DQPSK at 40Gbps you can significantly improve polarisation mode dispersion tolerance, the limiting factor today of DQPSK transceivers,” says Shabtai. MultiPhy expects the 40 Gigabit direct-detect market to shift towards DQPSK, accounting for the bulk of deployments in two years’ time.
Market applications
MultiPhy is delivering two solutions: for 40 and 100Gbps direct-detect, and 40 and 100Gbps coherent designs. The company has not said when it will deliver products but hinted that first it will address the direct-detect market and that chip samples will be available in 2011.
Not only will the samples enhance the reach of DQPSK-modulation based links but also allow the optical component specifications to be relaxed. For example, cheaper 10Gbps optical components can be used which, says MultiPhy, will reduce total design cost by “tens of percent”.
This is noteworthy, says Shabtai, as the direct-detect markets are increasingly cost-sensitive. “Coherent is being positioned as the high-end solution, and there will be pressure on the direct-detect market to show lower cost solutions,” he says.
MultiPhy is eyeing two 100Gbps spaces
MultiPhy’s view is that direct-detect modulation schemes will be deployed for quite some time due to their price and power advantage compared to coherent detection.
Another factor against 40Gbps coherent technology will be the price difference between 40Gbps and 100Gbps coherent schemes. “There is a good chance that 100Gbps will leapfrog 40Gbps coherent deployments,” he says. “The 40Gbps coherent modules will need to go a long way to get to the right price.” MultiPhy says it is hearing about the expense of coherent modules from system vendors and module makers, as well as industry analysts.
Metro and long-haul
The company says it has received several requests for 40Gbps and 100Gbps direct-detect schemes for the metro due to its sensitivity to cost and power consumption. “We are getting to the point in optical communications where one solution does not fit all – that the same solution for long-haul will also suit metro,” says Shabtai.
He believes 100Gbps coherent will become a mainstream solution but will take time for the technology to mature and its costs to come down. It will thus take time before 100Gbps coherent expands beyond long-haul and into the metro. He also expects a different 100Gbps coherent solution to be used in the metro. “The requirements are different – in reach, in power constraints” he says. “The metro will increasingly become a segment, not only for direct-detect but also for coherent.”
Coherent: Already a crowded market
There are at least a dozen companies actively developing silicon for coherent transmission, while half-a-dozen leading system vendors developing designs in-house. In addition, no-one really knows when the 100Gbps market will take off. So how does MultiPhy expect to fare given the fierce competition and uncertain time-to-revenues?
“It is very hard to predict the exact ramp up to high volumes,” says Shabtai. “At the end of the day, 100Gbps will come instead of 10Gbps and when people look back in five and six years’ time, they will say: ‘Gee, who would have expected so much capacity would have been needed?’.”
The big question mark is when will coherent technology ramp and this explains why MultiPhy is also targeting next-generation direct-detect schemes with its technology. “We cannot come to market doing the same thing as everyone else,” says Shabtai. “Having a solution that addresses power consumption based on one-sample-per-symbol gives us a significant edge.”
MultiPhy admits it has received greater market interest following Cisco’s acquisition of CoreOptics. “While Cisco said it would fulfill all previous commitments, still it worried some of CoreOptics’ customers,” says Shabtai. The acquisition also says something else to Shabtai: 100Gbps coherent is a strategic technology.
Did Cisco consider MultiPhy as a potential acquisition target? “First, I can’t comment, and I wasn’t at the company at the time,” says Shabtai.
As for design wins, Shabtai says MultiPhy is in “advanced discussion” with several leading module and system vendor companies concerning its 40Gbps and 100Gbps direct-detect and coherent technologies.
Further reading
See Opnext's multiplexer IC plays its part in 100Gbps trial
