Before I delve into how this specific tool works and how it helps when managing multipathed storage in Solaris, I’ll give some background on what multipathing is and how it is implemented in Solaris.
What is multipathing?
Multipathing is a generic term relating to a certain SAN configuration where a host on that SAN (ie; a server) has multiple physical paths to a target (ie; a disk array) on said SAN. A common multipath configuration consists of a server with multiple fibre channel ports, and each port is connected to a separate fibre channel switch in the SAN. Optionally, the target device or disk array may also have multiple physical links in the same manner. In the end, though, the host “sees” multiple instances of the same target – once through each connection to the SAN. It’s up to the host to then manage this. The end result is higher reliability. If a switch were to go down or the optical cable between the host and a switch were damaged, the host’s connectivity to its SAN disks would not be interrupted. IO traffic is also multiplexed over the multiple links, thus providing fail-over and higher throughput.
How multipathing works in Solaris
Multipathing has existed in Solaris since Solaris 8 via an add-on product called Sun SAN Foundation Kit. With the release of Solaris 10, all functionality of the SAN Foundation Kit was integrated into the OS and no longer requires an add-on to function.
When multipathing is enabled, the scsi_vhci driver looks for multiple instances of the same target (eg; a LUN) and upon finding one, creates a “meta” disk device which it manages. IO to and from this device are then multiplexed over however many paths scsi_vhci sees to the LUN.
Below is an example of a host which has two fibre channel controllers (c4 and c5) connected to the sam SAN, and both see the same target LUNs, of which there are four:
[root@ackpthh]/>cfgadm -al -o show_SCSI_LUN Ap_Id Type Receptacle Occupant Condition c4 fc-fabric connected configured unknown c4::60003930000153a5,0 disk connected configured unknown c4::60003930000153da,0 disk connected configured unknown c4::60003930000153eb,0 disk connected configured unknown c4::6000393000015463,0 disk connected configured unknown c5 fc-fabric connected configured unknown c5::60003930000153a5,0 disk connected configured unknown c5::60003930000153da,0 disk connected configured unknown c5::60003930000153eb,0 disk connected configured unknown c5::6000393000015463,0 disk connected configured unknown
In the above cfgadm output, the name of the LUNs are in the following format:
|fibre channel controller||WWN of target||LUN number on the target|
So, the aforementioned scsi_vhci driver sees that LUN 6000393000015463,0 is presented by two controllers. It will then make a new disk device using that device’s WWN, LUN, and other information to construct a globally-unique ID (GUID) device name and then create device entries under /dev/dsk and /dev/rdsk to represent a multipathed view:
[root@ackpthh]/>ls -l /dev/rdsk/c6t600039300001546301000000D52ACC7Ed0* lrwxrwxrwx 1 root root 69 Jul 6 2006 /dev/rdsk/c6t600039300001546301000000D52ACC7Ed0 -> ../../devices/scsi_vhci/disk@g600039300001546301000000d52acc7e:wd,raw lrwxrwxrwx 1 root root 68 Jul 6 2006 /dev/rdsk/c6t600039300001546301000000D52ACC7Ed0s0 -> ../../devices/scsi_vhci/disk@g600039300001546301000000d52acc7e:a,raw lrwxrwxrwx 1 root root 68 Jul 6 2006 /dev/rdsk/c6t600039300001546301000000D52ACC7Ed0s1 -> ../../devices/scsi_vhci/disk@g600039300001546301000000d52acc7e:b,raw ...
You might notice the new controller, c6. This is a “virtual” controller implemented by scsi_vhci since making such virtual disk devices under a real physical controller would be a broken idea since, technically, there is no such physical device. Remember, this is a meta device.
A note about those big, long multipath device names: there’s actually a format to them, so don’t let their length turn you off. The format is as follows, using the above device as an example:
|scsi_vhci controller||Target comprised of real target’s WWN
, LUN number, and SCSI volume UID
(Page 83 data)
|LUN number and then
We can now use the multipath device as we would any other kind of disk device, with format for example:
[root@ackpthh]/>format Searching for disks...done c6t600039300001546301000000D52ACC7Ed0: configured with capacity of 2794.54GB AVAILABLE DISK SELECTIONS: 0. c0t2d0 <DEFAULT cyl 8921 alt 2 hd 255 sec 63> /pci@0,0/pci1022,7450@2/pci1000,3060@3/sd@2,0 1. c6t60003930000153A501000000D52AE0C4d0 <APPLE-Xserve RAID-1.50-2.73TB> /scsi_vhci/disk@g60003930000153a501000000d52ae0c4 2. c6t600039300001546301000000D52ACC7Ed0 <APPLE-Xserve RAID-1.50-2.73TB> /scsi_vhci/disk@g600039300001546301000000d52acc7e 3. c6t60003930000153DA01000000D767690Ed0 <APPLE-Xserve RAID-1.50-1.82TB> /scsi_vhci/disk@g60003930000153da01000000d767690e 4. c6t60003930000153EB01000000D767D3C0d0 <APPLE-Xserve RAID-1.50-2.73TB> /scsi_vhci/disk@g60003930000153eb01000000d767d3c0 Specify disk (enter its number):
The next part will cover mpathadm itself and how it fits into managing these devices.