A continuous increase in bandwidth demand, stemming from new video applications, virtualization and convergence, is driving the need for higher data rates within traditional data centers. This is further evidenced by the formation of the Institute of Electrical and Electronics Engineers (IEEE) P802.3ba Task Force that has begun work on a 40-Gigabit Ethernet (GbE) and 100-GbE standard.

The task force will develop the guidance for data centers to migrate from 10-Gigabit Ethernet to 40G- and 100G-Ethernet data rates, keeping in mind that the data center environment requires solutions that enable a smooth migration to 40/100-GbE. This can be achieved using laser-optimized 50 µm multimode fiber (OM3), multifiber optical (MPO/MTP) connectivity and parallel optic transmission. The areas of implementation include all the Ethernet spaces within the data center, including network interfaces between core routers and distribution switches, from the distribution switches to the edge switches, then from the edge switches into the server.

Parallel optics

To achieve 40-GbE and 100-GbE data rates over OM3 multimode fiber, parallel optics is expected to replace traditional duplex fiber serial transmission technology. The switch to parallel optics becomes a requirement when data rate speed requirements increase beyond 16 Gbps. Vertical-cavity surface-emitting lasers (VCSELs) at 850 nm can only be reliably direct-modulated up to a data rate, typically around 20 Gbps. When wide temperature ranges are factored in, the overall reliability decreases even more.

Parallel optics multiplexes or divides the high-data-rate signal across multiple fibers that simultaneously transmit and receive. At the receiver, signals are de-multiplexed back to the original high-data-rate signal. For 40 Gbps signals, 12-fiber MTP-connectorized OM3 fibers are used, with four fibers sending 10 Gbps each for an aggregate bandwidth of 40 Gbps in one direction. Another four fibers receive 10 Gbps each for an aggregate bandwidth of 40 Gbps in the other direction. This scheme will likely be used to support 40 GbE in the data center (Figure 1).

Figure 1: 40 G Parallel Optics Transmission
  
For both 40-GbE and 100-GbE over OM3 fiber, the minimum objectives identified for the standard specify a distance of 100 meters. However, IEEE 802.3ba Task Force discussions are ongoing to consider extending the distance out to 150-200 meters to address the deployment of structured cabling solutions in the data centers.

Analysis completed by Corning determined that the minimum distance of 100 meters only represents approximately 60 percent of the deployed lengths within a typical data center today – therefore, extending the distance capability to 150-200 meters is desired.

The short-reach OM3 parallel optics architecture for the 100-Gbps solution will utilize two 12-fiber connections to support this data rate. 10-Gbps signals will run over 10 fibers in each direction, providing 100 Gbps transmit and receive transmission (Figure 2).

Figure 2: 100 G Parallel Optics Transmission

Why OM3 Fiber?

OM3 is the only multimode fiber included in the IEEE 802.3ba objectives for 40/100 Gigabit Ethernet. OM3 fiber has a standard 50 µm core with an effective modal bandwidth of 2000 MHz-km at 850 nm, and supports 10 GbE transmission (10GBASE-SR) for distances to 300 meters. Data centers currently using OM1 or OM2 will not have a migration path identified to support 40 or 100 GbE. Therefore, the time to install OM3 fiber is at hand (Figure 3).

 
Figure 3: Multimode Fiber Types

The MTP connector will also play an important role in providing a migration path for data center infrastructures to support parallel optics. A likely scenario will be using MTP connectors with parallel optics modules, such as the recently developed quad small-form-factor pluggable (QSFP) transceiver. The QSFP specification was jointly developed by a group of telecom companies to define a highly integrated 4-channel optical transceiver. QSFP modules would only be for 40G and would have a 12-fiber MTP connector interface. Expectation is that 100G will use two 12-fiber MTPs or one 24-fiber MTP that interfaces into the transceiver module.

Traditionally, data centers have used MTP-terminated trunk assemblies to provide connectivity between equipment distribution area (EDA) locations within the data center. At each EDA, the fibers transitioned from the MTP Connector to duplex LC connectors for termination into the electronics with 2-fiber patch cords. With 40 and 100 GbE using parallel optics transmission, transceiver modules will have MTP ports, driving the need for complete MTP connectivity throughout the channel.

Other Considerations

In addition to the connectivity method and fiber type, two additional considerations to be addressed with regards to support of 40 and 100 GbE performance are the total connector loss budget for 40/100 GbE and optical skew conformance.

Mitigating connector loss is best accomplished by ensuring a precise polishing process that results in perfect alignment of the fibers at the connector mating. Polishing techniques are complex for single-fiber connectors, and the intricacies of the process are multiplied for MTP connectivity. MTP end-face fiber protrusion and other geometry characteristics are critically important when implementing cabling infrastructures with MTP connectivity.

The other primary consideration to address with parallel optic transmission is skew rate. Because parallel optic technology requires signals to be multiplexed and de-multiplexed for simultaneous transmission across multiple fibers, the optical pulses may not reach the destination at exactly the same time. Any delay between the fastest and slowest, or skew, can result in latency and bit error rate performance issues.

Skew performance standards have not yet been determined for 40/100 GbE applications. However, there is a skew performance metric within the industry that can be used by vendors as a benchmark to validate cable assembly performance relative to skew for anticipated 40/100 GbE transmission requirements. The skew requirement identified in the InfiniBand standard defines a performance metric for a data rate of 120 Gbps, known as 12X-quadruple data rate (QDR). With InfiniBand 12X-QDR transmission, 12 fibers transmit, and 12 fibers receive 10 Gbps for an aggregate speed of 120 Gbps. This transmission scheme closely mirrors the expected parallel optic requirements for the 100 GbE application.

The 12X QDR InfiniBand specification allows a maximum cable assembly skew performance of 0.75 ns. Applying this criteria, Corning Cable Systems completed skew testing for distances of 300 meters. Testing to extended distances provides assurance of skew performance for distances beyond the minimum 100-meter objective over OM3 fiber specified by IEEE’s 40/100-Gbps task force. Without a standards requirement defined yet, it falls upon the industry to align itself with vendors who have performed due diligence in ensuring products meet performance requirements.

A Final Word

As data centers move into a future where transmission needs exceed the current 10 GbE data rates, a migration to 40 and 100 GbE is inevitable – along with all of its associated requirements. As IEEE moves ahead to develop the guidance for higher data rates, all indications are that the roadmap for data center infrastructures will be comprised of parallel optics technology over OM3 fiber with MTP connectivity between Ethernet electronic ports.

Other technologies seen in the data center include Fibre Channel and InfiniBand. Work is currently underway to migrate Fibre Channel to 16 Gbps and beyond, where future data rates include 32 Gbps, which cannot be supported by duplex fiber serial solutions. InfiniBand technology for optical connectivity solutions in high-performance computer clusters and other applications also includes guidance for parallel optic transmission.

With the writing on the wall, data centers should be driving towards OM3 fiber solutions using MTP connectivity to support future parallel optic transmission needs. With these solutions in place today, the migration to 40 and 100 GbE will be an easy one. The timeline for completion of IEEE’s 40/100 GbE standard is November 2010.

Doug Coleman is the manager of technology and standards at Corning Cable Systems, (www.corning.com/cablesystems), headquartered in Hickory, NC.

Comments (1)

Subscribe to this comment's feed

...
written by jbonja , June 23, 2008

It should be noted that the IEEE 802.3ab Task Force is currently working on the 40 Gb and 100 Gb Ethernet standard, including de-skewing specifications, and expects it to be published by 2010. In May of 2008, CommScope proposed a worst-case skew model for parallel multimode channels, which was accepted by acclamation of the Task Force. De-skewing circuitry will be built to handle the worst-case skew of this model. This means that skew-controlled cabling will not be required to support the specified channels for 40 Gb or 100 Gb Ethernet.

report abuse

vote down

vote up

Votes: +2