Copper cabling can resolve the cost / power equation

Electrical

Wednesday, 09 July 2008 13:36

Written by Allard Van der Horst

At the same time however, careful management of power often needs to take place in the face of shrinking capital expenditure, which needs to be controlled and managed with equal levels of tenacity.  While there are many more physical and technical factors to consider, the power and cost considerations of data center interconnect need to be balanced.

This article therefore recaps on the main advantages and disadvantages of each of the principal incumbent interconnect options:  10GBASE-SR, CX4 and 10GBASE-T and proposes that emerging active twin-ax interconnect solutions present a radical alternative which can simultaneously tackle both power and capex issues.

10GBASE-SR

When it comes to its reach, physical weight and power consumption, multimode optical fibre interconnect has for more than twenty years created links that tick all the right boxes for high speed data comms; a glance at the simple technology comparison table shown in Table 1 confirms this.

Since 80% of all cables in the data center are less than 30m in length, clearly optical fiber’s got the range aspect more than covered and weight-wise of course, it is the least likely of the available options to pose a weight-bearing problem for cable ducts - be they underground or overhead.

On power consumption too optical fibre interconnects perform well.  Operating at as little as 1.5W, an XFP fibre optic cable module figures well and with newer SFP+ solutions taking power consumption down to 1W or maybe even less - things can only get better.  Plus it really is as easy to terminate in the field as copper cable and let’s face it bend radius really isn’t an issue in a professional environment.

It’s the overall link cost however that’s always been the big issue and for data center managers looking to reduce capital outlay, fibre optic interconnect provides a useful first port of call.  It’s not the medium that’s more or less expensive than copper solutions, it’s the cost of lasers (notably) and transceiver electronics that makes the optical interconnect option cost heavy.  At a link cost of $500, 10G interconnect based on optical fibre is today 2-3 times more than copper alternatives and for the short intra-rack connections, their future viability is being called into question.

10GBASE-CX4

While around half the cost of their optical counterparts, CX4 cables are looking increasingly unattractive from the overall link cost perspective too, as the world’s copper prices soar.  The CX4 cable itself is composed of 8 twin-axial copper pairs, requiring individual pair and overall EMI shielding.  On the upside, since each transmit and receive lane supports 3Gbps operation to arrive at the 10G throughput, equalisation and of course the electro-optic conversion of the fibre optic solution is avoided.

It’s more than reasons of overall link cost however that CX4 has been unpopular, has never really taken off and is now being passed over.  Its thick heavy cable makes it to say the least cumbersome to deploy and reliability is an issue too.  Remember, that unlike the fibre optic alternative, CX4 cable ends are not active modules, they are dumb connectors and cable driver electronics (four off) need to be mounted on the line card.

As shown by the photograph in Figure 1, the CX4 connector is big! and unlike intelligent XFP and SFP+ modules, it also sits proud of the rack facia.  This off-rack protrusion, combined with the bend radius of the cable simply eats into precious data center space, while introducing significant mechanical stresses and reliability issues at the same time.

The power consumption of the CX4 solution is traditionally considered to be low, as the previous generation data center equipment had internal 4-lane (XAUI) interfaces which converted relatively to the CX4 interface.  The move in the current generation switches and server NICs however is to enable high port density by using serial interfaces.  These will have to be converted to the 4-lane CX4 interface at significant cost and potentially several Watts of power.  This development makes CX4 an even less attractive option going forward.

10GBASE-T

Now ratified and interoperability tested, 10GBASE-T is today presenting the latest alternative for data centre interconnects.  This technology uses existing CAT5 copper cable with 8-wire twisted pair operating in a bidirectional communication mode and employs the ubiquitous and very small RJ45 (or similar) connector.

At $500, link cost is similar to the 10GBASE-SR optical solution which means it’s receiving close scrutiny right now, it is however significantly higher in operating power.  Why?  Put simply, it’s because the required signal processing is extremely complex.  Even when implemented in current state-of-the-art 45nm CMOS processes, needing in excess of 4W per 30m link end, 10GBASE-T is very power hungry.

10GBASE-T is not a modular interface, with the RJ45 jack mounted on the face of the equipment and all the cable driver and interface electronics again needing to be placed on the line card.  This makes it unattractive for high-density switches and routers where the investment will have to be made up-front.

Active twin-ax

Enter the new kid on the block.  Well, twin-ax copper cable is actually nothing new of course and it’s long been harnessed in lower speed passive cables.  However, limited in both reach and top speed, such cables have not been able to support the 10G requirement of data centers at the lengths required.

Phyworks is one semiconductor manufacturer that has now come up with the chip technology to enable active twin-ax cables assured of error free transmission at 10G.  Put simply, it has taken the existing PHY2060 equalizer and retimer technology it developed for optical fibre cables and repurposed it for copper cables.

Active twin-ax copper cable assemblies, like that shown in Figure 2, are now being trialled by cable assembly makers and data center equipment manufacturers. Phyworks’ equalization technology enables active interconnect over thin 24 to 30AWG twin-ax cable of up to 30m.  Importantly, the cable link cost is less than a third that of its 10GBASE-SR optical counterpart, while it’s power consumption is the same.

The solution is seen in many ways to ‘de-risk’ data center network design.  Since the chip technology will fit both XFP and SFP+ modules there’s an element of future proofing, plus it means that the new copper cable interconnects can be hot-plugged into existing optical ports to dramatically increase design flexibility.

Furthermore, unlike 10G-BASE-T, the cable driver electronics, the equalizer and retimer technology, is contained in the cable module and not on the line card and so in terms of power consumption, Phyworks’ active twin-ax cable solution is effectively “pay-as-you-grow”.

The real interconnect choice

Cutting to the chase then, if intra-rack data center interconnect was being chosen purely by link cost considerations, then only two solutions would stand-up to scrutiny: 10GBASE-T and active twin-ax.  The long incumbent 10GBASE-SR and CX4 would have long ago fallen by the wayside.

Looking further at the aspect of power consumption, then the choice between these two copper alternatives is stark.  The calculations used to arrive at the table are based on an electricity price of $20.72/kWh, a CO2 emission figure of 1.5 pounds of carbon emissions per kilowatt hour generated by a coal fired power station and a typical data centre cluster having 1600 ports.

It will be seen that in terms of power consumption, operating cost and CO2 emissions, the active twin-ax copper interconnect solution offers a far superior performance to that of 10GBASE-T, one that is also as good as the incumbent 10GBASE-SR optical solution.

We conclude therefore that active twin-ax interconnect can provide a valid alternative to fibre optic interconnect in the data center and succeeds in simultaneously reducing link cost, link power and CO2 emissions.

Figures:

Figure 1:  The different cable ends: left to right: SFF8470 connector for CX4, SFP+ for active twin-ax, SFP+ for optical fibre and RJ45 plug for 10GBASE-T.

Figure 2:  An active twin-ax copper cable assembly using the Phyworks PHY2060 active equaliser and retimer chip.

Table 1: A comparison of interconnect technologies.

Table 2: A comparison of power consumption, operating costs and CO2 emissions.

About the Author: Allard Van Der Horst is the Director of Applications at Phyworks. Allard was previously Manager of IP development at Fujitsu Microelectronics. He has an MSc in Electrical Engineering and IT from the University of Twente (Netherlands).

TrackBack URI for this entry

Comments (0)

Subscribe to this comment's feed