One of the things I want to start posting about is fiber channel (sometimes spelled fibre channel). It’s something I was first exposed to back in 1998, and have been dabbling with off and on since then (entirely at work, since it’s pricey stuff). In the past year I’ve been using it much more, and have learned quite a bit about it.
The biggest issue I have is that whenever I google about an issue, I either run into someone who’s using it in a database application, or someone who’s using it at a very low end in a small video configuration. This is a problem because neither of these scenarios fits our situation at work, and I have to experiment to figure out the solution to a problem, and more often than not it’s a shot in the dark — but it gives me an opportunity to really learn it and with luck impart more information about it.
The three-sentence description of fiber channel is basically this. Attaching a disk to a computer happens via some kind of storage interface — usually IDE/ATA (two disks max), SCSI (7 or 15 disks max), or SATA (one disk per port, more with port multipliers). USB and FireWire don’t count, since those convert one of IDE or SATA to USB or FireWire and then attach. Fiber channel is basically the result of someone saying “Hey Beavis, let’s take the disks out of computers, put them somewhere else, and then tie all the computer-to-disk connections together!”
The result of this is you get some of the great features of networks — with the right hardware, you can attach thousands of disks to a single computer. With the right hardware, you can transfer data screamingly fast. With the right software and hardware, you can add multiple links together and increase the speed of the connection. And with the right software, you can share a common set of disks between multiple computers.
The problem is that it brings with it the bad features of disk controllers. Most operating systems will scan for disks when they start up, and whatever they find, that’s what they expect to keep. They don’t like having new disks presented to them after bootup, and they *really* don’t like losing a disk that they found at bootup. Windows is a lot worse — it actually assumes that it owns any disk it sees, and writes a little tag to the beginning of each disk it finds if it doesn’t recognize the existing disk label.
I’ve spent the past year trying to wring performance and reliability out of several different fiber channel configurations at work, for video, database, shared storage, and other configurations, and have mostly succeeded, with lots of help from colleagues and vendors. I’ve learned a lot, and so has everyone involved; and I’ve not found a lot of references to most of the things we’ve learned, so I want to try and share it.
Next post on this topic — an introduction to our fiber channel switches.