SAN connectivity guide - Fibre Channel, Ethernet and iSCSI

By Stephen J. Bigelow, Features Writer | Jul 6, 2009

1 comments [1]

Facebook [2]

LinkedIn [3]

Digg [5]

Email [6]

Print [7]

Thumbnail: 

A storage area network (SAN) organizes a broad assortment of storage devices into a single storage resource that can then be provisioned, allocated and managed for the entire enterprise. Although issues like storage capacity performance and management often receive the most attention, the connectivity between each SAN device plays a critical role in successful SAN deployment.

Each switch and storage system on the SAN must be interconnected -- usually through optical fiber or copper cabling -- and the physical interconnections must support bandwidth levels that can adequately handle the peak data activities that occur. This overview details the role of Fibre Channel, Ethernet and iSCSI connectivity on a SAN.

Fibre Channel
Fibre Channel is the quintessential SAN interconnect and virtually every storage switch and storage platform provides Fibre Channel ports. Multiple Fibre Channel ports support simultaneous data streams, but individual ports can often be aggregated into groups for even higher effective bandwidth. Servers and other devices can also be fitted with Fibre Channel host channel adapters to enable an Fibre Channel interface.

As a serial interface, Fibre Channel bandwidth is denoted in Gbps. Early Fibre Channel implementations ran at 1 Gbps per port before doubling to 2 Gbps and then 4 Gbps. Another transition to 8 Gbps is beginning but mainstream adoption is not expected until mid or late 2009. Also, 10 Gbps is used in interswitch links to connect Fibre Channel switches to each other. Fibre Channel operates with numerous protocols, most notably SAS and IP.

Fibre Channel can use several types of physical media. Twisted pair cable is used to cover relatively short distances at low speeds between Fibre Channel devices. Coaxial cables generally offer better shielding against signal interference and can run across somewhat longer distances. Optical fiber is routinely used to carry the fastest signals across distances up to 10km.

Ethernet
While Ethernet connectivity is generally used on the greater local area network (LAN), its use in the storage area network has been limited by its relatively slow bandwidth.

Traditional Ethernet ports support 10/100 Mbps -- far slower than Fibre Channel. This had limited Ethernet in the SAN to basic management tasks. For example a storage device or switch might include a single Ethernet port that connects the device to the LAN where an administrator can manage the device across it. Ethernet typically uses two protocols; Transmission Control Protocol (TCP) which handles the organization of data into packets and Internet Protocol (IP) which handles the way those data packets are addressed. In fact, the terms "Ethernet" and "TCP/IP" are often used interchangeably.

Ethernet bandwidth is increasing today and 10 Gigabit Ethernet (10 GigE) is widely available for data centers, which boosts performance on the LAN and will eventually make Ethernet use more practical for carrying data on the SAN. One-Gigabit Ethernet is now common on many servers and switches, and the eventual emergence of 10 GigE promises to put Ethernet on par with 10-Gbit Fibre Channel.

Traditional Ethernet LAN deployments used coaxial cables but twisted-pair cabling (e.g. Category 5 or Category 6 Ethernet cables) is the most common LAN cabling. 10 GigE often relies on optical fiber with transmission distances up to 40km, which makes the technology far more expensive and limits its use to network backbones. As copper cabling becomes available for 10 GigE, the technology should see far more use within data centers and SANs.

iSCSI and FCIP
Fibre Channel SANs have long been challenged by deployment expense and management complexity -- often keeping SANs out of reach of smaller IT organizations. The emergence of iSCSI eases these challenges by encapsulating SCSI commands into IP packets for transmission over an Ethernet connection, rather than a Fibre Channel connection. This approach eliminates Fibre Channel in favor of Ethernet, which allows iSCSI to transfer data over LANs, WANs or the Internet and supports storage management over long distances.

In actual practice, a user or application will cause the operating system to generate corresponding SCSI storage commands. Those SCSI commands and data are then encapsulated and IP headers are added to make packets. The packets can then be sent over an ordinary Ethernet connection. The remote end of the iSCSI connection disassembles the encapsulated content and passes the SCSI commands to the SCSI controller and storage device. This also works in reverse, so any data or responses can be sent back to the user or application across the Ethernet connection.

Although it is easier to manage than Fibre Channel, iSCSI still has two disadvantages for storage. At 1 GigE, it does not perform as fast as Fibre Channel. And Ethernet will drop packets during network congestion. These problems may be alleviated soon, thanks to the emergence of 10 GigE and Data Center Ethernet -- a standard in development with the goal of providing Ethernet with quality of service levels, efficient multipathing and lower latencies, while preventing dropped packets.

Another alternative is FCIP. FCIP translates Fibre Channel commands and data into IP packets, which can be exchanged between distant Fibre Channel SANs. It's important to note that FCIP only works to connect Fibre Channel SANs, but iSCSI can run on any Ethernet network.

Storage vendors are working on a Fibre Channel over Ethernet (FCoE) standard to enable SAN and LAN convergence. FCoE would reduce cabling by using converged network adapters in place of Fibre Channel HBAs and NICs. FCoE adoption depends on the availability of Data Center Ethernet and is not expected to be widely adopted before late 2009. Unlike iSCSI, FCoE is not routable and is subject to the distance limitations of Fibre Channel.

1 comments [1]

Facebook [2]

LinkedIn [3]

Digg [5]

Email [6]

Print [7]