Sunday, July 3, 2011

EMC PowerPath for multipath I/O

Multipath I/O is a fault-tolerance and performance enhancement technique in computer storage area networks, whereby there is more than one physical path between the CPU in a computer storage system. The physical multipath is through the buses, controllers, switches, and bridge devices connecting the mass storage.

Data Center Scenario:
In our Main library data center environment, the same expectation of high system’s availability must be assured to minimize downtime and cutting mean time between failures (MTBF) on its 24x7 operations.Hence, the application services such as: e-lib, ilib, repository and kiosks that are link in a storage area networks required a set up of multiple redundant data paths (multipath) between a host servers and storage systems to avoid interruptions in data flow and anticipation if ever unexpected hardware failure occur. To manage a multipath I/O configuration, IT administrators should ensure that the server OS supports multipath I/O and is configured properly to access data from the storage system and fail over to secondary data paths when necessary.

Technology Implementation:
Our simple implementation of multipathing is using Dell EMC-500X which is connected to two Fibre Channel ports (FC1 -considered the fastest network carrier for large amounts of information) and the rest are physical path components, adapters, cables, and switches — to create logical paths between the server and the storage device. A network interface card (in the iSCSI case) or HBA should be connected by using redundant switch infrastructures to provide continued access to storage in the event of a failure in a storage fabric component. Should one controller, port or switch fail, causing the path to fail, multipathing logic uses an alternate path for I/O so that applications can still access their data. On the other hand the operating system can route I/O through the remaining controller transparently to the application, with no changes visible to the applications, other than perhaps incremental latency.

Option to Multipathing I/O:
There are two Linux multipath I/O available EMC PowerPath and DM multipath though both have each pros and cons; we preferred EMC Powerpath for a variety of ideal and practical reasons. Provide viable and robust multipath I/O capability for Linux operating systems on Dell PowerEdge servers and Dell/EMC storage systems. This multipath software layers can leverage the redundant paths to provide performance enhancing features, such as:
• Traffic shaping
• Automatic path management
• Dynamic reconfiguration
• Round-robin, adaptive failover
• Basic to advance automatic path failover
• Online recovery
• Supports heterogeneous OS(Windows®, Linux, UNIX®, and Novell NetWare))

PowerPath features:
PowerPath does not use a configuration file but rather it comes with its own init script and can be started and stopped from the command line. However, PowerPath relies on how the HBA driver has detected the LUN and created the block devices, and does not use an administrator-supplied configuration file; its enumeration of LUNs can vary from one node to the next in clustered servers. As with DM multipath, the configurations of the following block devices: /dev/sdb, /dev/sdc, /dev/sdd, and /dev/sde are all bundled to /dev/emcpowera. And once started, PowerPath gathers the UUIDs of the block devices and bundles the devices with the same UUID into a single device, /dev/emcpowerX. As PowerPath identifies the LUNs, it enumerates them as /dev/emcpowera, /dev/emcpowerb, and so on.

PowerPath Operations:
The PowerPath, administrators must stop all I/Os to the devices and unmount them just to confirm that all of the /dev/emcpowerX devices are not in use before issuing the command service PowerPath stop. This device can be partitioned, formatted with a file system, and mounted using the following cli commands: fdisk, mke2fs, mkdir and mount.

Powerpath Utilities:
The software features includes powermt, a powerful management utility for its devices. Its man page (man powermt) provides specific information about the utility. After installation, administrators can start PowerPath by issuing the command service PowerPath start. When stopping PowerPath, administrators should be sure that there is no I/O activity— that is, PowerPath should not be in use by any application. Other features of powermt is allowing administrators to display the current settings; set priority, policy (algorithms), and mode; remove a particular HBA or device; and restore a removed HBA or device.

OS : Centos 64 bit
DELL Servers
FC1 card controller
Fiber Optic Network cables
Lan Networks

To implement Powerpath in a Unix/Linux like platform
To implement EMC-Linux-5-1-0.194.rhel5.i386.rpm in an Intel 64 bit CPU architecture


Steps to make File Systems on Red hat Linux connected to EMC .To prepare the system and install PowerPath, follow these steps:

Below is the screenshot of the EMC PowerPath software’s availability; you may choose which version suits to your hardware architecture

EMCpower.LINUX-5.1.0-194.rhel5.i386.rpm (Intel chip 64 bit CPU architecture)

Step 1:
Set up the system for boot over SRP (SCSI remote Protocol-transferring and commands and data ->modprobe). When the system can boot over SRP, proceed with the steps below to get the system boot over PowerPath. First install below mentioned rpms:

1) EMCpower.LINUX-5.1.0-194.rhel.i386.rpm
2) Naviagentcli.noarch.rpm
[root@db1 ~]# rpm –ivh “rpm name”
root@db1 ~]# rpm -ivh EMCpower.LINUX-5.1.0-194.rhel.x86_64.rpm
[root@db1 ~]# rpm -ivh naviagentcli.noarch.rpm
Start two service mentioned below
[root@db1 ~]# /PATH_EMC/ service PowerPath start
[root@db1 ~]# /PATH_EMC/ service naviagent start
[root@db1 ~]# /etc/init.d/PowerPath start

Starting PowerPath:
[root@db1 ~]#

Step 2:
Edit the file /etc/redhat-release and put in the single line
[root@db1 ~]# Starting PowerPath: This package requires RedHat RHEL5."
Red Hat Enterprise Linux Server release 5 (Number of Powerpath version) so that the system is now identifying itself as a RHEL machine.
[root@db1 ~]# cd /etc
[root@db1 ~]# vi rhel5-x86_64

Change the Linux OS release version into the corresponding version of EMCpower.LINUX-5.1.0-

Step 3:
Install PowerPath on the system. Follow the usual EMC PowerPath installation procedure to install PowerPath.
[root@db1 ~]# emcpreg -install (register powerpath)

This will ask you the key.

Please check this key installed on other server connected with EMC by cli command
[root@db1 ~]# emcpreg –list command

This will install key

Check that EMC path is now available or not by a cli command
[root@db1 ~]# cat /proc/partitions

You should see sd* devices in list, if not then reboot server

Step 4:
After PowerPath is installed properly and started, enter the fdisk -l command. You should see output similar to the example below. In this case, only one LUN is assigned to this host, and the LUN is partitioned accordingly.

[root@db1 ~]# fdisk -l

PowerPath is now up and running. The emcpowera* shows the PowerPath pseudo devices.

Step 5:
Creation of file system (Please skip or ignore this steps if it doesn’t support GPT use GNU parted instead)

This will temporarily mount FS
[root@db1 ~]# cat /proc/partitions

[root@db1 ~]# fdisk /dev/emcpowera

[root@db1 ~]# partprobe
[root@db1 ~]# pvcreate /dev/emcpowera1

32M is Physical extend
VOLUME_GROUP is volume group name
[root@db1 ~]# vgcreate -s 32M VOLUME_GROUP /dev/emcpowera1 204800 is size of file system that is 200(GB)*1024
[root@db1 ~]# lvcreate -L 204800M -n EMC /dev/VOLUME_GROUP
Mkfs.ext3 is used to convert the FS in ext3
[root@db1 ~]# mkfs.ext3 /dev/VOLUME_GROUP/EMC

Create mount point e.g. mysql on /
[root@db1 ~]# mount /dev/VOLUME_GROUP/EMC /mysql

Step 6:
Permanent mounting
[root@db1 ~]# edit /etc/fstab
[root@db1 ~]# cd /etc
[root@db1 ~]# vi fstab

/dev/emcpowera1 /emc ext3 auto, nouser, exec

Above entry in fstab will be active on reboot.

Reboot server

Check that FS “emc” is available by
[root@db1 ~]# df –h rw, sync

Step 7:

Testing mysql file copied to PowerPath

[root@db1 ~]# mount /dev/emcpowera1 /mnt/emc/

[root@db1 ~]# mkdir /mnt/emc/mysql
[root@db1 ~]# service mysqld stop
[root@db1 ~]# cp -R /usr/local/mysql/var /mnt/emc/mysql/

Step 8:
Proceed to EMC PowerPath productions.

Details:(1) LUN clustered configuration owned by SPA

Details(2):Basicj highly available SAN configuration with a LUN owned by SPA

EMC PowerPath Remarks:
After the evaluation of the requirements of implementing our storage area networks- multipathing, it is a clever option to proceeds on EMC powerpath features for a various supported reason. Based on our technical experience about practically and simplicity of usage we had dealt it greatly using EMC PowerPath.

Successful deployment of EMC Power Path(Linux Version )

1)Please contact the author(by e-mail) with regards to any set-up deployment/development plan.
2) Reference uses Dell Multipath/Powerpath articles