In the matter of cognition, the SAN faced a certain obstacle - the inaccessibility of basic information. In the matter of studying other infrastructure products that I have encountered, it’s easier - there are trial versions of the software, the ability to install them on a virtual machine, there are a bunch of tutorials, reference guides and blogs on the topic. Cisco and Microsoft rivet very high-quality textbooks, MS in addition, at the very least, combed his infernal attic pantry called technet, even there is a book on VMware, albeit one (and even in Russian!), And with an efficiency of about 100%. Already on the storage devices themselves, you can get information from seminars, marketing events and documents, forums. On the same storage network - silence and the dead with braids stand. I found two textbooks, but did not dare to buy. This "Storage Area Networks For Dummies" (there is one, it turns out. Very curious English-speaking "dummies" in the target audience, apparently) for 1,500 rubles and "Distributed Storage Networks: Architecture, Protocols and Management" - it looks more reliable, but 8200r at a discount of 40%. Together with this book, Ozon also recommends The Art of Masonry.
I don’t know what to advise to a person who decides to study at least the theory of organizing a data storage network from scratch. As practice has shown, even expensive courses can yield zero at the exit. People in relation to SAN are divided into three categories: those who do not know what it is at all, who know that such a phenomenon simply exists, and those who look at the question "why do two or more factories in a storage network" with such bewilderment as if they were asked something like "why should a square have four corners?"
I’ll try to fill in the gap that was missing for me - to describe the base and to describe is simple. I will consider SAN based on its classic protocol - Fiber Channel.
So SAN - Storage area network - designed to consolidate server disk space on dedicated storage storages. The bottom line is that this way disk resources are used more economically, easier to manage and have greater performance. And in matters of virtualization and clustering, when several servers need access to the same disk space, such storage systems are generally irreplaceable.
By the way, in the SAN terminology, due to the translation into Russian, there is some confusion. SAN in translation means "storage area network" - SHD. However, classically in Russia under the storage system is understood the term "data storage system", that is, it is a disk array ( Storage array), which in turn consists of a Control Unit ( Storage Processor, Storage Controller) and disk shelves ( Disk enclosure) However, in the original Storage Array is only part of the SAN, although at times the most significant. In Russia we get that SHD (data storage system) is part of SHD (data storage network). Therefore, storage devices are usually called storage systems, and the storage network is called SAN (and confused with "Sun", but this is a trifle).
Components and Terms
Technologically, a SAN consists of the following components:1. Nodes
- Disk arrays (data storage systems) - storages (targets)
- Servers - consumers of disk resources (initiators).
2. Network infrastructure
- Switches (and routers in complex and distributed systems)
- Cables
Features
Without going into details, the FC protocol is similar to the Ethernet protocol with WWN addresses instead of MAC addresses. Only, instead of two levels, Ethernet has five (of which the fourth is not yet defined, and the fifth is mapping between FC transport and high-level protocols that are transmitted via this FC - SCSI-3, IP). In addition, FC switches use specialized services, the analogues of which for IP networks are usually located on servers. For example: Domain Address Manager (responsible for assigning Domain ID to the switches), Name Server (stores information about connected devices, a kind of WINS analogue within the switch), etc.For SANs, the key parameters are not only performance, but also reliability. After all, if the database server disappears the network for a couple of seconds (or even minutes) - well, it will be unpleasant, but you can survive. And if the hard drive with the base or the OS falls off at the same time, the effect will be much more serious. Therefore, all SAN components are usually duplicated - ports in storage devices and servers, switches, links between switches and, a key feature of SAN, compared to LAN - duplication at the level of the entire infrastructure of network devices - factories.
Factory (fabric - which is actually translated from English fabric, because the term symbolizes the interwoven connection diagram of network and end devices, but the term has already been established) - a set of switches interconnected by inter-switch links ( ISL - InterSwitch Link).
Highly reliable SANs necessarily include two (and sometimes more) factories, because the factory itself is a single point of failure. Those who have at least once observed the consequences of a ring on the network or the adroit movement of the keyboard, introducing a kernel-level switch or distribution into an unsuccessful firmware or command, understand what this is about.
Factories may have identical (mirror) topology or vary. For example, one factory can consist of four switches, and the other one, and only highly critical nodes can be connected to it.
Topology
The following types of factory topologies are distinguished:Cascade - The switches are connected in series. If there are more than two, then it is unreliable and unproductive.
Ring - closed cascade. More reliable is just a cascade, although with a large number of participants (more than 4), performance will suffer. And a single failure of ISL or one of the switches turns the circuit into a cascade with all the consequences.
Mesh) it happens Full mesh - when each switch connects to each. Characteristically high reliability, performance and price. The number of ports required for inter-switch communications, with the addition of each new switch to the circuit, grows exponentially. With a certain configuration, there simply will be no ports left for the nodes - everyone will be busy under ISL. Partial mesh - any chaotic association of switches.
Center / Peripherals (Core / Edge) - close to the classic LAN topology, but without a distribution level. Often, storages connect to Core switches, and servers connect to Edge. Although storage can be allocated an additional layer (tier) Edge-switches. Storage and servers can also be connected to the same switch to improve performance and reduce response time (this is called localization). This topology is characterized by good scalability and manageability.
Zoning (zoning)
Another SAN-specific technology. This is the definition of initiator-target pairs. That is, which servers to which disk resources can be accessed, so that it does not work out that all servers see all possible disks. This is achieved as follows:- the selected pairs are added to the zones (zones) previously created on the switch;
- zones are placed in zone sets (zone set, zone config) created in the same place;
- zone sets are activated in the factory.
For the initial post on the topic of SAN, I think enough. I apologize for the variegated pictures - there is no way to draw yourself at work yet, but there is no time at home. There was an idea to draw on paper and photograph, but decided that it was better.
Finally, as a postscript, I will list basic SAN factory design guidelines.
- Design the structure so that between the two end devices there are no more than three switches.
- It is desirable that the factory consisted of no more than 31 switches.
- It is necessary to set the Domain ID manually before entering a new switch into the factory - it improves manageability and helps to avoid problems of the same Domain ID, in cases, for example, reconnecting the switch from one factory to another.
- Have multiple equivalent routes between each storage device and the initiator.
- In cases of undefined performance requirements, proceed from the ratio of the number of Nx ports (for end devices) to the number of ISL ports as 6: 1 (EMC recommendation) or 7: 1 (Brocade recommendation). This ratio is called oversubscription.
- Zoning Recommendations:
- use informative names of zones and zone sets;
- use WWPN zoning, not Port-based (based on device addresses, not physical ports of a particular switch);
- each zone is one initiator;
- clean the factory of the "dead" zones. - Have a reserve of free ports and cables.
- Have a reserve of equipment (switches). At the site level - mandatory, possibly at the factory level.
What is it?
Data storage network, or Storage Area Network is a system consisting of the actual storage devices - disk, or RAID - arrays, tape libraries and other things, the data transmission medium and the servers connected to it. It is usually used by large enough companies with well-developed IT infrastructure for reliable data storage and high-speed access to them.
Simplified, storage is a system that allows servers to distribute reliable fast disks of variable capacity from different storage devices.
A bit of theory.
The server can be connected to the data warehouse in several ways.
The first and simplest one is DAS, Direct Attached Storage (direct connection), we put drives into the server, or an array into the server adapter, and we get a lot of gigabytes of disk space with relatively fast access, and when using a RAID array, there is sufficient reliability, although spears on the subject of reliability for a long time.
However, this use of disk space is not optimal - on one server the place runs out, on the other there is still a lot of it. The solution to this problem is NAS, Network Attached Storage (network-attached storage). However, with all the advantages of this solution - flexibility and centralized management - there is one significant drawback - the speed of access, the network of 10 gigabits is not yet implemented in all organizations. And we are approaching a storage network.
The main difference between a SAN and a NAS (in addition to the letter order in abbreviations) is how the connected resources are seen on the server. If the NAS resources are connected to the NFS or SMB protocols, in the SAN we get a connection to the disk, with which we can work at the level of block I / O operations, which is much faster than a network connection (plus an array controller with a large cache adds speed to many operations).
Using SAN, we combine the advantages of DAS - speed and simplicity, and NAS - flexibility and controllability. Plus, we get the ability to scale storage systems until there is enough money, while simultaneously killing several more birds with one stone, which are not immediately visible:
* remove restrictions on the connection range of SCSI devices, which are usually limited to a wire of 12 meters,
* reduce backup time,
* we can boot from SAN,
* in case of refusal from NAS we unload a network,
* we get a high input-output speed due to optimization on the storage system side,
* get the opportunity to connect multiple servers to one resource, then it gives us the following two birds with one stone:
o make full use of the capabilities of VMWare - for example VMotion (virtual machine migration between physical machines) and others like them,
o we can build failover clusters and organize geographically distributed networks.
What does it give?
In addition to developing the budget for optimizing the storage system, we get, in addition to what I wrote above:
* increase in productivity, load balancing and high availability of storage systems due to several ways to access arrays;
* savings on disks by optimizing the location of information;
* accelerated recovery from failures - you can create temporary resources, deploy backup to them and connect the servers to them, and restore information yourself without haste, or transfer resources to other servers and calmly deal with dead iron;
* Reduced backup time - thanks to the high transfer speed, you can back up to the tape library faster, or even take a snapshot (snapshot) from the file system and safely archive it;
* disk space on demand - when we need it - you can always add a couple of shelves to the storage system.
* reduce the cost of storing a megabyte of information - of course, there is a certain threshold from which these systems are cost-effective.
* A reliable place to store mission critical and business critical data (without which the organization cannot exist and function normally).
* I want to mention VMWare separately - all the chips like migrating virtual machines from server to server and other goodies are available only on the SAN.
What does it consist of?
As I wrote above - SHD consists of storage devices, transmission media and connected servers. Let's consider in order:
Data storage systems usually consist of hard drives and controllers, in a self-respecting system, as a rule, only 2 - 2 controllers, 2 paths to each drive, 2 interfaces, 2 power supplies, 2 administrators. Among the most respected system manufacturers, mention should be made of HP, IBM, EMC, and Hitachi. Here I will quote one representative of EMC at the seminar - “HP makes excellent printers. Well, let her make them! ”I suspect that HP also loves EMC. Competition between manufacturers is serious, however, as elsewhere. The consequences of competition are sometimes sane prices per megabyte of storage system and problems with compatibility and support of competitor standards, especially for old equipment.
Data transfer medium. Typically, SANs are built on optics, which currently gives a speed of 4, sometimes 8 gigabits per channel. When building, specialized hubs were used before, now there are more switches, mainly from Qlogic, Brocade, McData and Cisco (I have never seen the last two on the sites). Cables are used traditional for optical networks - single-mode and multi-mode, single-mode longer.
Inside, FCP is used - Fiber Channel Protocol, a transport protocol. Typically, classic SCSI runs inside it, and FCP provides addressing and delivery. There is an option with connecting via a regular network and iSCSI, but it usually uses (and heavily loads) a local, and not dedicated for data transfer network, and requires adapters with iSCSI support, well, the speed is slower than in optics.
There is also a smart word topology, which is found in all textbooks on SAN. There are several topologies, the simplest option is point to point, we connect 2 systems. This is not DAS, but a spherical horse in vacuum is the simplest version of SAN. Next comes the controlled loop (FC-AL), it works on the principle of “pass on” - the transmitter of each device is connected to the receiver of the subsequent one, the devices are closed in a ring. Long chains tend to initialize for a long time.
Well, the final option is a switched fabric (Fabric), it is created using switches. The structure of connections is built depending on the number of connected ports, as in the construction of a local network. The basic principle of construction is that all paths and connections are duplicated. This means that there are at least 2 different paths to each device on the network. Here, the word topology is also used, in the sense of organizing a device connection diagram and connecting switches. In this case, as a rule, switches are configured so that the servers do not see anything other than the resources intended for them. This is achieved by creating virtual networks and is called zoning, the closest analogy is VLAN. Each device on the network is assigned an analog MAC address on the Ethernet network, it is called WWN - World Wide Name. It is assigned to each interface and each resource (LUN) of storage systems. Arrays and switches can distinguish between WWN access for servers.
Server Connect to storage via HBA - Host Bus Adapters. By analogy with network cards, there are one-, two-, and four-port adapters. The best dog breeders recommend installing 2 adapters per server, this allows both load balancing and reliability.
And then resources are cut into the storage systems, they are the LUNs for each server and a place is left in the reserve, everything is turned on, the system installers prescribe the topology, catch glitches in the configuration of switches and access, everything starts and everyone lives happily ever after *.
I specifically do not touch on different types of ports in the optical network, whoever needs it - he already knows or reads, who does not need it - just hammer his head. But as usual, if the port type is incorrectly set, nothing will work.
From experience.
Usually, when creating a SAN, arrays with several types of disks are ordered: FC for high-speed applications, and SATA or SAS for not very fast ones. Thus, 2 disk groups with different megabyte cost are obtained - expensive and fast, and slow and sad cheap. Usually, all databases and other applications with active and fast I / O are hung on the fast one, while file resources and everything else are hung on the slow one.
If a SAN is created from scratch, it makes sense to build it on the basis of solutions from one manufacturer. The fact is that, despite the declared compliance with standards, there are underwater rakes of the equipment compatibility problem, and not the fact that part of the equipment will work with each other without dancing with a tambourine and consulting with manufacturers. Usually, to tackle such problems, it is easier to call an integrator and give him money than to communicate with manufacturers switching arrows.
If the SAN is created on the basis of the existing infrastructure, everything can be difficult, especially if there are old SCSI arrays and a zoo of old equipment from different manufacturers. In this case, it makes sense to call for help from the terrible beast of an integrator who will unravel compatibility problems and make a third villa in the Canary Islands.
Often when creating storage systems, firms do not order manufacturer support for the system. This is usually justified if the company has a staff of competent competent admins (who have already called me a teapot 100 times) and a fair amount of capital, which makes it possible to purchase spare parts in the required quantities. However, competent administrators are usually lured by integrators (I saw it myself), but they don’t allocate money for the purchase, and after failures, a circus begins with the cries of “I'll fire everyone!” Instead of a call to support and the arrival of an engineer with a spare part.
Support usually comes down to replacing dead disks and controllers, and to adding disk shelves and new servers to the system. A lot of trouble happens after a sudden prevention of the system by local specialists, especially after a complete shutdown and disassembly-assembly of the system (and this happens).
About VMWare. As far as I know (virtualization specialists correct me), only VMWare and Hyper-V have functionality that allows you to transfer virtual machines between physical servers on the fly. And for its implementation, it is required that all servers between which the virtual machine moves are connected to the same disk.
About clusters. Similar to the case with VMWare, the systems I know of building failover clusters (Sun Cluster, Veritas Cluster Server) that I know require storage connected to all systems.
While writing an article - they asked me - in which RAIDs do drives usually combine?
In my practice, they usually made either RAID 1 + 0 on each disk shelf with FC disks, leaving 1 spare disk (Hot Spare) and cut LUNs from this piece for tasks, or did RAID5 from slow disks, again leaving 1 disk to replace. But here the question is complex, and usually the way to organize disks in an array is selected for each situation and justified. The same EMC, for example, goes even further, and they have additional array settings for applications that work with it (for example, under OLTP, OLAP). I didn’t dig so deeply with the other vendors, but I guess everyone has a fine-tuning.
* before the first major failure, after which support is usually bought from the manufacturer or supplier of the system.
Since there are no comments in the sandbox, I'll post it on my personal blog.
In the simplest case, a SAN consists of storage systems, switches, and servers connected by optical communication channels. In addition to directly disk storage systems, SANs can also connect disk libraries, tape libraries (tape drives), devices for storing data on optical disks (CD / DVD, etc.), etc.
An example of a highly reliable infrastructure in which servers are connected simultaneously to a local area network (left) and to a storage network (right). Such a scheme provides access to data located on the storage system in case of failure of any processor module, switch, or access path.
Using SAN allows you to provide:
- centralized resource management of servers and storage systems;
- connection of new disk arrays and servers without stopping the operation of the entire storage system;
- use of previously purchased equipment in conjunction with new data storage devices;
- quick and reliable access to data storage devices located at a great distance from the servers * without significant performance losses;
- speed \u200b\u200bup the backup and recovery process - BURA.
History
The development of network technologies has led to the emergence of two network solutions for storage systems - Storage Area Networks (SAN) for exchanging data at the block level supported by client file systems, and servers for storing data at the file level Network Attached Storage (NAS). To distinguish traditional storage from network, another retronym was proposed - Direct Attached Storage (DAS).
DASs, SANs, and NASs that have appeared on the market sequentially reflect the evolving chains of links between applications that use data and bytes on a medium containing this data. Once upon a time, application programs themselves read and wrote blocks, then drivers appeared as part of the operating system. In modern DAS, SAN, and NAS, the chain consists of three links: the first link is the creation of RAID arrays, the second is the processing of metadata to interpret binary data in the form of files and records, and the third is the services for providing data to the application. They differ in where and how these links are implemented. In the case of DAS, storage is "bare", it only provides the ability to store and access data, and everything else is done on the server side, starting with interfaces and drivers. With the advent of SAN, RAID provisioning moves to the storage side; everything else remains the same as with DAS. And the NAS is different in that metadata is also transferred to the storage system to ensure file access, here the client only needs to support data services.
The emergence of SAN was made possible after the Fiber Channel (FC) protocol was developed in 1988 and ANSI was approved as a standard in 1994. The term Storage Area Network dates back to 1999. Over time, the FC gave way to Ethernet, and became widespread IP-SAN with iSCSI connectivity.
The idea of \u200b\u200ba NAS network storage server belongs to Brian Randall of the University of Newcastle and was implemented in machines on a UNIX server in 1983. This idea was so successful that it was picked up by many companies, including Novell, IBM, and Sun, but eventually replaced the leaders of NetApp and EMC.
In 1995, Garth Gibson developed the principles of NAS and created Object Storage Systems (OBS). He began by dividing all disk operations into two groups; one included more frequently performed operations, such as reading and writing, and the other, more rare ones, such as operations with names. Then he proposed, in addition to blocks and files, another container, he called it an object.
OBS is distinguished by a new type of interface, it is called an object. Client data services interact with metadata using the Object API. OBS not only stores data, but also supports RAID, stores metadata related to objects and supports the object interface. DAS, and SAN, and NAS, and OBS coexist in time, but each of the types of access is more consistent with a certain type of data and application.
SAN Architecture
Network topology
SAN is a high-speed data network designed to connect servers to storage devices. A variety of SAN topologies (point-to-point, Arbitrated Loop, and switching) replace traditional server-to-storage bus connections and provide greater flexibility, performance, and reliability compared to them. The basis of the SAN concept is the ability to connect any of the servers to any data storage device that uses the Fiber Channel protocol. The principle of node interaction in a SAN with point-to-point or switching topologies is shown in the figures. In a SAN with Arbitrated Loop topology, data is transferred sequentially from node to node. In order to start data transfer, the transmitting device initiates arbitration for the right to use the data transmission medium (hence the name of the topology - Arbitrated Loop).
The SAN transport is based on the Fiber Channel protocol, using both copper and fiber-optic device connections.
SAN Components
SAN components are divided into the following:
- Data storage resources;
- Devices that implement the SAN infrastructure;
Host bus adaptors
Data storage resources
Storage resources include disk arrays, tape drives, and fiber channel libraries. Storage resources realize many of their capabilities only when they are included in the SAN. So top-class disk arrays can replicate data between arrays over Fiber Channel networks, and tape libraries can transfer data to tape directly from disk arrays with Fiber Channel interface, bypassing the network and servers (Serverless backup). The most popular on the market are disk arrays from EMC, Hitachi, IBM, Compaq (the Storage Works family, inherited by Compaq from Digital), and among tape library manufacturers we should mention StorageTek, Quantum / ATL, IBM.
SAN Infrastructure Devices
SAN devices include Fiber Channel switches (FC switches), hubs (Fiber Channel Hubs), and routers (Fiber Channel-SCSI routers). Hubs are used to combine Fiber Channel Arbitrated Loop devices (FC_AL ) The use of hubs allows you to connect and disconnect devices in the loop without stopping the system, since the hub automatically closes the loop if the device is turned off and automatically opens the loop if a new device was connected to it. Each change of the loop is accompanied by a complex process of its initialization. The initialization process is multi-stage, and until its completion, data exchange in the loop is impossible.
All modern SANs are built on switches that enable a full network connection. Switches can not only connect Fiber Channel devices, but also delimit access between devices, for which so-called zones are created on the switches. Devices placed in different zones cannot exchange information with each other. The number of ports in a SAN can be increased by connecting switches to each other. A group of related switches is called Fiber Channel Fabric or simply Fabric. Connections between switches are called Interswitch Links, or ISL for short.
Software
The software allows for redundant server access paths to disk arrays and dynamic load balancing between paths. For most disk arrays, there is an easy way to determine that the ports available through different controllers belong to the same disk. Specialized software maintains a table of access paths to devices and provides disconnection of paths in the event of an accident, dynamic connection of new paths and load balancing between them. As a rule, manufacturers of disk arrays offer specialized software of this type for their arrays. VERITAS Software manufactures VERITAS Volume Manager software designed for organizing logical disk volumes from physical disks and providing backup access paths to disks, as well as load balancing between them for most known disk arrays.
Protocols Used
Storage networks use low-level protocols:
- Fiber Channel Protocol (FCP), SCSI transport over Fiber Channel. The most commonly used protocol at the moment. There are options for 1 Gbit / s, 2 Gbit / s, 4 Gbit / s, 8 Gbit / s and 10 Gbit / s.
- iSCSI, SCSI transport over TCP / IP.
- FCoE, transporting FCP / SCSI over pure Ethernet.
- FCIP and iFCP, encapsulation and transmission of FCP / SCSI in IP packets.
- HyperSCSI, SCSI transport over Ethernet.
- FICON transport via Fiber Channel (used only by mainframes).
- ATA over Ethernet, ATA transport over Ethernet.
- SCSI and / or TCP / IP transport through InfiniBand (IB).
Benefits
- High reliability of access to data located on external storage systems. Independence of SAN topology from used storage systems and servers.
- Centralized data storage (reliability, security).
- Convenient centralized management of switching and data.
- Transfer intensive I / O traffic to a separate network - LAN offload.
- High speed and low latency.
- Scalability and flexibility of the SAN logical structure
- The geographic dimensions of the SAN, in contrast to the classic DAS, are practically unlimited.
- The ability to quickly distribute resources between servers.
- The ability to build fault-tolerant cluster solutions at no additional cost based on the existing SAN.
- Simple backup scheme - all data is in one place.
- Availability of additional features and services (snapshots, remote replication).
- High security SAN.
Sharing storage systems typically simplifies administration and adds a fair amount of flexibility since cables and disk arrays do not need to be physically transported and switched from one server to another.
Another advantage is the ability to load servers directly from the storage network. With this configuration, you can quickly and easily replace a failed one.
7 SAN building blocks
The previous sections provide an overview of the Fiber Channel topologies and protocol. Now let's look at the various devices and components that are used to create Fiber Channel storage networks. The main structural elements of a SAN include:
■ bus adapters;
■ Fiber Channel cables;
■ connectors;
■ Connectivity devices, which include hubs, switches, and connected architecture switches.
Note that all addressed components within a Fiber Channel-based storage network have unique WWN (World Wide Names), which are analogues of unique MAC addresses. The WWN name in the Fiber Channel specification is a 64-bit number written in the form XX: XX: XX: XX: XX: XX: XX: XX: XX. The IEEE Institute assigns a specific address range to each manufacturer. The manufacturer is responsible for the unique allocation of the assigned addresses.
7.1 Bus adapters
Bus adapter (host bus adapter - HBA) connects to a computer and provides interaction with data storage devices. In the Windows personal computer world, bus adapters typically plug into a PCI bus and can provide connectivity for IDE, SCSI, and Fiber Channel devices. Bus adapters operate under the control of a device driver, i.e. SCSIPort or Storport miniport drivers.
During initialization, the bus adapter port registers with the connected architecture switch (if one is available) and registers the attributes stored on it. Attributes are available to applications using the API from the manufacturer of the switch or bus adapter. The Storage Networking Industry Association (SNIA) is working on a standardized API that supports various manufacturers' APIs.
For a storage network that has serious resiliency requirements, some bus adapter manufacturers provide additional options, such as automatically switching to another bus adapter when the primary fails.
In a split access ring, only two devices can receive and transmit data at a time. Suppose one of them is a bus adapter connected to a node and receiving data from a storage device. But, if this adapter is connected to a storage network based on a switched connected architecture, it can send multiple read requests to multiple storage devices at the same time.
Answers to these requests can come in any order. Typically, a connected architecture switch provides a round-robin service for ports, which further complicates the task of the bus adapter; in this case, the packet arrival order will be such that each subsequent packet will come from a different source.
Bus adapters can solve this problem in one of two ways. The first strategy, called storage and sorting, involves storing data in the node’s memory and then sorting the buffers at the expense of the central processor. Obviously, this is an inefficient approach from the point of view of the central processor and the total load is associated with switching the context every few tens of microseconds. Another strategy is on the fly- implies the use of additional system logic and microcircuits on the bus adapter itself, which allows switching contexts without using CPU cycles. Typically, the time between context switches when using this strategy is a few seconds.
One reservation allows you to send one Fiber Channel frame. Before sending the next frame, the sender must receive a signal Receiver ready. To use the Fiber Channel effectively, it is necessary to transmit several frames at the same time, which will require several reservations, therefore, more memory will be required to receive frames. Some bus adapters have four 1 Kbyte buffers and two 2 Kbytes buffers, although some high-level adapters have 128 and 256 Kbytes to reserve the buffer. Note that this memory usually requires two ports; those. when one memory area receives data from the SAN Fiber Channel, the remaining memory areas can transmit data to the host PCI bus.
In addition, bus adapters play a role in resiliency and disaster recovery architecture, which provides multiple I / O routes to a single storage device.
7.1.1 Windows operating system and bus adapters
In Windows NT and Windows 2000, Fiber Channel adapters are considered SCSI devices, and drivers are created as SCSI miniport drivers. The problem is that the SCSIPort driver is outdated and does not support the capabilities provided by new SCSI devices, not to mention Fiber Channel devices. Therefore, a new Storport driver model was introduced in Windows Server 2003, which should replace the SCSIPort model, especially for SCSI-3 and Fiber Channel devices. Please note that Fiber Channel drives are used by Windows as DAS devices, which is ensured by the level of abstraction provided by the SCSIPort and Storport drivers.
7.1.2 Double routes
Sometimes you need increased performance and reliability, even at the expense of increasing the cost of a turnkey solution. In such cases, the server connects to dual-port drives through several bus adapters and several independent Fiber Channel storage networks. The main idea is to eliminate a single point of failure in the network. In addition, at times when the system is operating normally, multiple routes can be used to balance the load and improve performance.
7.2 Fiber Channel Cable Types
Two types of cables are mainly used: optical and copper. The main advantages and disadvantages of cables are listed below.
■ Copper cables are cheaper than optical.
■ Optical cables support higher data rates than copper cables.
■ Copper cable can be used at a shorter distance, up to 30 meters. In this case, the optical cable can be used at a distance of up to 2 kilometers (multimode cable) or up to 10 kilometers (single mode cable).
■ Copper cable is more susceptible to electromagnetic interference and the mutual influence of other cables.
■ Optical data usually needs to be converted into electrical signals for transmission through the switch and back into optical form for further transmission.
There is only one type of copper cable, in contrast to the optical, which is represented by two types: multimode and single-mode.
For short distances, a multimode cable is used, which has a core with a diameter of 50 or 62.5 microns (microns - micrometers, or one millionth of a meter.) The light wave that is used in a multimode cable has a length of 780 nanometers, which is not supported in single-mode cables. For long distances, a single-mode cable is designed with a core diameter of 9 microns. The single-mode cable uses a light beam with a wavelength of 1300 nanometers. Despite the topic of this chapter (Fiber Channel interface), it is worth mentioning that such cables can be used to build networks based on other interfaces, such as Gigabit Ethernet.
7.3 Connectors
Since several types of cables (and data transfer technologies) are supported by the Fiber Channel interface, devices (for example, bus adapters, communication and storage devices) are available with connectors that support connection to the transmission medium, which is done to reduce overall costs. There are several types of connectors designed for various transmission media and interfaces.
■ Gigabit interface converters (GBIC) support serial and parallel translation of transmitted data. GBIC converters provide the possibility of "hot" connection, i.e. enabling / disabling GBIC does not affect the operation of other ports. Converters use a 20-bit parallel interface.
■ Gigabit link modules (Gigabit link modules - GLM) provide functions similar to GBIC, but require a device shutdown for their installation. On the other hand, they are slightly cheaper than GBIC.
■ Media Interface Adapters are used to convert signals between copper and optical media and vice versa. Media interface adapters are commonly used in bus adapters, but can also be used on switches and hubs.
■ Small Form Factor Adapters (SFFs) allow you to place more connectors of various interfaces on a board of a certain size.
7.4 Interaction devices
Interaction devices interconnect storage network components. These include a variety of devices, ranging from low-cost Fiber Channel hubs to expensive, high-performance and managed switches of a connected architecture.
7.4.1 Split Fiber Channel Hubs
FC-AL hubs are a budget option for connecting multiple Fiber Channel nodes (storage devices, servers, computer systems, other hubs and switches) in a ring configuration. Typically, hubs provide 8 to 16 ports. A hub can support various transmission media, such as copper or optical.
Fiber Channel hubs are passive devices, i.e. any other device in the ring cannot detect their presence. Hubs provide the following features:
■ internal connections that allow any port to connect to any other port;
■ the ability to bypass the port to which a malfunctioning device is connected.
The biggest problem with ports is that they can only support one Fiber Channel connection at a time. The figure shows that if port 1 gained control to establish a session with port 8, no other port will be able to transmit data until the established session ends.
Hubs can be connected to Fiber Channel architecture switches without modification. In addition, you can create a cascade of hubs by connecting two hubs with a cable.
FC-AL hubs are market leaders in the Fiber Channel market, but Fiber Channel connectivity switches are becoming increasingly popular as the cost decreases.
FC-AL hubs are created by companies such as Gadzoox Networks, Emulex and Brocade.
7.4.2 Shared Fiber Channel Ring Switches
The most significant advantage of FC-AL switches over hubs is the simultaneous support of multiple connections, while hubs only support one connection at a time.
Fig. Fiber Channel Hub
The ability to simultaneously support multiple connections is associated with certain difficulties. Devices connected to the ring switch do not even “suspect” their role. Ring switches are involved in both data transfer and ring addressing. The following is additional information on this subject, as well as the role of switches in storage networks and the methods by which vendors add new features to their products.
Figure Fiber Channel Switch
Ring Switches and Data Transfer
A server that intends to access the storage device must send an arbitration request for ring management. In a normal hub-based FC-AL ring, each device receives an arbitration packet before it is returned to the server bus adapter, so that the server gains control of the ring. The ring switch will send a successful control response immediately without sending requests to other nodes. At this point, the bus adapter will send the basic Open packet for the storage device port, which will be transmitted by the ring switch. If the port does not transfer data at this time, there should not be any special problems. Otherwise, conflict situations may occur. To solve this problem, the ring switch must provide buffers for temporarily storing frames for port 7. Some switch vendors provide 32 buffers for each port for this purpose.
Ring Switches and Addressing FC-AL
FC-AL hubs do not play a role in assigning addresses to devices, but only transmit base address frames in a ring. The same can be said of most switches. However, some devices may insist on getting a specific address. Some hubs have the ability to control the initialization order of ports, which allows a specific port to be initialized first, after which the device will be connected to the required port.
Switches and ring initialization
The FC-AL protocol requires reinitialization of the ring when connecting, disconnecting, or reinitializing the device. Such ring initialization can lead to disruption of the existing connection between the other two devices. Some switch manufacturers provide the ability to selectively screen and transmit LIP (Loop Initialization Primitives) packets. This operation is designed to minimize problems, reduce ring reinitialization time, and preserve existing data transfer sessions if possible. At the same time, it is necessary to ensure the uniqueness of device addresses.
If all devices participate in the reinitialization of the ring, duplication of addresses does not occur, since the devices "protect" their addresses. But, if some devices do not participate in the reinitialization of the ring, it is necessary to prevent the assignment of already distributed addresses to the devices participating in the reinitialization of the ring. Uniqueness of addresses is provided by additional logic of the ring switch. When adding a storage device, the LIP packet must be sent to the server, however, LIP does not need to be transmitted to storage devices that never communicate with other storage devices.
Some storage devices can communicate directly with other storage devices, which is used to back up data.
Ring Switches and Connected Architecture
If all the devices in the ring “know” about the connected architecture, the ring switch sends the necessary frames in the usual way, for example Fabric Login frames. If the devices in the ring do not support the connected architecture, the ring switch must independently execute a sufficiently large
workload.
Some vendor ring switches do not support cascading. In addition, some ring switches require a firmware update before connecting to switches of a connected architecture. A number of switches should be upgraded to fully support the connected architecture before connecting to the SAN.
FC-AL switches are manufactured by companies such as Brocade, McDATA, Gadzoox Networks, Vixel, and QLogic.
7.4.3 Fiber Channel Connected Architecture Switches
Fiber Channel Fabric Fabric Switches (FC-SW) switches provide multiple high-speed communications simultaneously with all devices. At the moment, the main switches support a speed of about 1 Gbit / s, while a speed of 2 Gbit / s also ceases to be a curiosity. Basically, single-port connected architecture switches are more expensive than FC-AL hubs and switches, but they provide much more functionality.
Connected architecture switches are more efficient than FC-AL hubs and switches. For example, switches provide the specific services described above, provide flow control with basic management packs, and, more importantly, some switches can emulate FC-AL functions to provide backward compatibility with older devices.
Some connected architecture switches support non-buffered routing. Its essence is that when the frame header is received, the switch quickly finds the destination trigger header while the frame is still being received. The advantage of this approach is the reduction of delays in frame delivery and the absence of the need to store the contents of the frame in the buffer memory. And the disadvantage is the immediate transfer of all frames, including damaged ones.
Connectivity switches play an important role in the security of Fiber Channel-based storage networks.
7.4.4 Comparison of three connection devices
The table shows the functionality and differences between the three types of Fiber Channel devices.
7.4.5 Bridges and routers
Both in this chapter and throughout the article, the terms bridges and routers do not refer to traditional Ethernet bridges and IP routers. In this case, bridges and routers are devices for Fiber Channel, and not for network protocols of the 2nd and 3rd levels.
Bridges are devices that interface between Fiber Channel and legacy protocols such as SCSI. Fiber Channel-SCSI bridges keep your existing investment in SCSI storage devices. These bridges support SCSI and Fiber Channel interfaces and convert data from two protocols. In this way, a new server with Fiber Channel bus adapter installed can access existing SCSI storage devices. Bridges provide an interface between a parallel SCSI bus and a Fiber Channel interface. Routers have similar capabilities, but for multiple SCSI buses and Fiber Channel interfaces. Storage routers, or smart bridges, provide additional capabilities such as masking and LUN mapping, and support SCSI Extended Copy commands. As devices that transmit data, routers use Extended Copy commands for use by storage libraries, which allows you to copy data between the specified target device and the connected library. This feature is also called independent backup (no server).
Examples of router and bridge manufacturers include companies such as Crossroads Systems, Chaparral Network Storage, Advanced Digital Information Corporation (ADIC after the acquisition of Path-light), and MTI.
