Windows Knowledge Base

[IBM Storage] SDD/SDDPCM multipath limitations and recommendations for AIX

Often questions arise how many paths to a LUN are supported or recommended. Or if there is a minimum number of paths required or a maximum number of paths allowed for SDD/SDDPCM devices on AIX?

The first source should be the appropriate Multipath Subsystem Device Driver User's Guide
e not supported.

Multipath Subsystem Device Driver User's Guide, V1.8-2.4
http://www.ibm.com/support/docview.wss?rs=540&context=ST52G7&uid=ssg1S7000303
ftp://ftp.software.ibm.com/storage/subsystem/UG/1.8--2.4/SDD_1.8--2.4_User_Guide_English_version.pdf

As the information still is pretty much spread across this (and other) documents I provide a short summary of and reference to the related paragraphs here:

DS4000/DS5000:

Because switching a device to another controller is a time-consuming recovery action and affects I/O performance, you can use this redundancy to avoid an unnecessary controller failover if a path fails. Therefore, configure a minimum of fourpaths for each LUN with two host adapter ports and two storage controller ports where each host adapter port has redundancy to each storage controller port and vice versa. (p.8)

SVC:

At least one path must be configured through each node of a virtualization product group. That is, if only two paths exist, they must go to separate nodes for each I/O group. However, at least two paths to each node are recommended.

DS8k/ESS:

If you need the SDDPCM I/O load-balancing and failover features, ensure that a minimum of two paths to a device are attached. (p.104)

You need to be aware that due to their architecture DS4k/DS5k, DS6000 and SVC have preferred and non-preferred paths where I/O is primarily routed over the preferred paths. Preferred paths are the ones directly going to the controller or node that actually owns the LUN. Only if no preferred path can be used a non-preferred path is selected for I/O to the alternate controller or node. So with subsystems which have preferred and non-preferred paths a minimum of four paths is quite reasonable, especially to provide load-balancing across the preferred paths to a LUN and avoid unnecessary LUN ownership changes on active/passive controller subsystems (such as DS4k/DS5k) in case of a path failure when only a total of two paths is configured.

DS8000 (and ESS) however manages all available paths equally and thus doesn't have any non-preferred paths. Here a minimum of two paths to HAs in different I/O enclosures already provides load-balancing and allows concurrent code loads. Be sure to take at least one HA in an even numbered DS8000 I/O enclosure and one HA in an odd numbered DS8000 I/O enclosure for best performance and availability!

The pcmpath command lists all available paths for each LUN managed by SDDPCM and marks the non-preferred paths with a '*':

# pcmpath query device

Total Dual Active and Active/Asymmetrc Devices : 2

DEV#:   0  DEVICE NAME: hdisk1  TYPE: 2107900  ALGORITHM:  Load Balance
SERIAL: 75KA3628201
==========================================================================
Path#      Adapter/Path Name          State     Mode     Select     Errors
0           fscsi0/path0           OPEN   NORMAL     844158          0
1           fscsi0/path1           OPEN   NORMAL     843779          0
2           fscsi1/path2           OPEN   NORMAL     833541          0
3           fscsi1/path3           OPEN   NORMAL     833459          0

DEV#:   1  DEVICE NAME: hdisk2  TYPE: 2145  ALGORITHM:  Load Balance
SERIAL: 60050768018082E29000000000000008
==========================================================================
Path#      Adapter/Path Name          State     Mode     Select     Errors
0*          fscsi0/path0           OPEN   NORMAL          6          0
1           fscsi0/path1           OPEN   NORMAL    2124784          0
2           fscsi1/path2           OPEN   NORMAL    2148331          0
3*          fscsi1/path3           OPEN   NORMAL          8          0

Total Active/Passive Devices : 1

DEV#:   3  DEVICE NAME: hdisk3  TYPE: 1815      ALGORITHM:  Load Balance
SERIAL: 600A0B8000118FD00000878E49C33524
==========================================================================
Path#      Adapter/Path Name          State     Mode     Select     Errors
0*          fscsi0/path0           OPEN   NORMAL          0          0
1*          fscsi1/path2           OPEN   NORMAL          0          0
2           fscsi0/path1           OPEN   NORMAL      94440          0
3           fscsi1/path3           OPEN   NORMAL      94168          0

For SDD you have the datapath command instead to display the path information for the SDD vpath devices.

For DS4k/DS5k subsystems you additionally have the sddpcm_get_config command to relate AIX hdisks to DS4k/DS5k subsystem volumes:

# sddpcm_get_config -Av
Frame id 0:
Storage Subsystem worldwide name: 60ab8001143a0000049b88968
Controller count: 2
Partition count: 1
Partition 0:
Storage Subsystem Name = 'VIOS_DS4800'
hdisk      LUN #   Ownership          User Label
hdisk3         0   A (non-preferred)  lpar3_vol01


The SDD/SDDCM support matrix and filesets are available here:

Support Matrix for Subsystem Device Driver, Subsystem Device Driver Path Control Module
http://www.ibm.com/support/docview.wss?rs=540&context=ST52G7&dc=DA400&uid=ssg1S7001350

Subsystem Device Driver (SDD) for AIX
http://www.ibm.com/support/docview.wss?rs=540&context=ST52G7&dc=D430&uid=ssg1S4000065

Subsystem Device Driver Path Control Module (SDDPCM) for AIX
http://www.ibm.com/support/docview.wss?rs=540&context=ST52G7&dc=D430&uid=ssg1S4000201


SDD/SDDPCM also require an AIX host attachment fileset to be installed as prerequisite:

Host Attachments for SDD on AIX
http://www.ibm.com/support/docview.wss?rs=540&context=ST52G7&dc=D400&q1=host+script&uid=ssg1S4000106

Host Attachment for SDDPCM on AIX
http://www.ibm.com/support/docview.wss?rs=540&context=ST52G7&dc=D400&q1=host+script&uid=ssg1S4000203


Maximum number of LUNs / paths for SDD (p.39)

For different AIX OS levels, SDD has set different limits on the maximum number of LUNs that can be configured. These limits exist because AIX has resource limitations on the total number of devices that a system can support. In a multipath configuration environment, AIX creates one hdisk device for each path to a physical disk. Increasing the number of paths that are configured to a physical disk increases the number of AIX system hdisk devices that are created and are consuming system resources. This might leave fewer resources for SDD vpath devices to be configured. On the other hand, more SDD vpath devices can be configured if the number of paths to each disk is reduced.

For AIX versions 4.3 and 5.1, AIX has a published limit of 10 000 devices per system. Based on this limitation, SDD limits the total maximum number of SDD vpath devices that can be configured to 600. This number is shared by all SDD-supported storage devices.

For AIX version 5.2 or later, the resource of the AIX operating system is increased. SDD has increased the SDD vpath device limit accordingly. Beginning with SDD 1.6.0.7, SDD supports a combined maximum of 1200 supported storage devices on AIX version 5.2 or later.

Table 7 provides a summary of the maximum number of LUNs and the maximum number of paths allowed when running on a host systems with different operating system levels.



You can have a maximum of 32 paths per SDD vpath device regardless of the number of LUNs that are configured. This means that you can only have a maximum of 32 host adapter ports for your SDD vpath devices. However, configuring more paths than is needed for failover protection might consume too many system resources and degrade system performance. Use the minimum number of paths that are necessary to achieve sufficient redundancy in the SAN environment. The recommended number of paths is 2 - 4. To avoid exceeding the maximum number of paths per SDD vpath device on AIX 5.2 or later, follow the recommendations in Table 8.




Maximum number of devices and paths that SDDPCM supports (p.114)

SDDPCM supports a maximum of 1200 configured devices and a maximum of 16 paths per device. That means the maximum number of host adapter ports that are supported is 16. However, with the round robin or load balance path selection algorithms, configuring more than four paths per device might impact the I/O performance. Use the minimum number of paths necessary to achieve sufficient redundancy in the SAN environment. The recommended number of paths per device is four. In order to support 1200 supported storage device LUNs, system administrators should first determine whether the system has sufficient resources to support a large number of devices.
See “Preparing your system to configure more than 600 supported storage devices or to handle a large amount of I/O after queue depth is disabled” on page 40 for more information.


Maximum number of paths for SVC VDisks

For SVC specific recommendations and limits take a look at the SVC Software Installation and Configuration Guide (SC23-6628-03):

SVC Software Installation and Configuration Guide
http://www.ibm.com/support/docview.wss?rs=591&uid=ssg1S7002605

Node configuration rules (p.77)

You must follow the configuration rules for SAN Volume Controller nodes to ensure that you have a valid configuration.

VDisks Each node presents a virtual disk (VDisk) to the SAN through four ports. Each VDisk is accessible from the two nodes in an I/O group. Each HBA port can recognize up to eight paths to each logical unit (LU) that is presented by the cluster. The hosts must run a multipathing device driver before the multiple paths can resolve to a single device. You can use fabric zoning to reduce the number of paths to a VDisk that are visible by the host. The number of paths through the network from an I/O group to a host must not exceed eight; configurations that exceed eight paths are not supported by IBM