SSP Administration
PowerVC and SSP
SSP is a fully supported storage provider in PowerVC. SSP was developed much earlier than PowerVC, but its shared setup can fit very well to the cloud nature of PowerVC. After creating an SSP (few clicks in HMC GUI), PowerVC (which is connected to the HMC) will recognize it automatically and without any additional tasks we can start to create LUs and deploy VMs. (There is no strict distinction, but the word LUN is used more for physical volumes attached to VIOS (lspv), and the word LU for virtual disks created in SSP.)
What is important that each VIO server, which is part of the SSP cluster, has to see the same LUNs. The virtual disks (LUs), which are created in SSP, can be found as files in a special filesystem: /var/vio/SSP/SSP_Cluster_1/D_E_F_A_U_L_T_061310. This fs is created during SSP creation and it is available on each VIO server. (These LUs are basically files in that filesystem, and because these LUs are thin provisioned, these files are so called 'sparse files'). SSP commands can be run as padmin on each VIOS.
/var/vio/SSP/SSP_Cluster_1/D_E_F_A_U_L_T_061310/VOL1 <--contains LUs available in PowerVC
/var/vio/SSP/SSP_Cluster_1/D_E_F_A_U_L_T_061310/IM <--contains Images available in PowerVC
cluster -list <--lists available SSP clusters
cluster -status -clustername <CLUSTER_NAME> -verbose <--it will show primary database node as well (grep -p DBN)
lu -list <--lists SSP LUs
snapshot -list <--lists snapshots (images in PowerVC)
lssp -clustername <Cluster_Name> -sp <SSP_Name> -bd <--old command to list LUs (bd is backing device)
-------------------------------------
Adding a new LUN to SSP
If we want to increase the available free space in SSP we need to add a new LUN to it.
1. request a new LUN from SAN team <--should be a shared LUN assigned to each VIO server in SSP
2. cfgmr (or cfgdev as padmin) <--bring up new disk on all VIO, make sure it is the same disk
3. chdev -l hdisk$i -a queue_depth=32 <--set any parameters needed
4. in HMC GUI add new disk to SSP <--on HMC SSP menu choose SSP, then check mark SSP (System Default) --> Action --> Add Capacity
After that PowerVC will automatically recognize, no steps are needed in PowerVC. df -g can be used to monitor available free space in /var/vio/SSP/SSP_Cluster_1/D_E_F_A_U_L_T_061310.
-------------------------------------
Removing leftover LUs from SSP
Sometimes a volume has been deleted in PowerVC, but they are still visible in SSP. These can be removed from SSP commands with lu -remove ... What is important, to make sure these LUs are really not used by anything. (LUs can be used by VMs and also by Images!!!)
1. check LU list on HMC, PowerVC and VIO <--HMC and PowerVC GUI lists LUs, on VIO 'lu -list' can be used
2. lu -list -attr provisioned=false <--on VIO lists LUs which are not assigned to any LPARs
3. lu -remove -clustername <Cluster_Name> -luudid <ID> <--remove a LU from SSP
If there are many LUs this for cycle can be used as well:
$ for i in `lu -list -attr provisioned=false | awk '{print $4}'`; do lu -remove -clustername SSP_Cluster_1 -luudid $i; done
-------------------------------------
PowerVC Images, Snapshots, LUs and SSP
When an Image is created in PowerVC sometimes a file is created in SSP, sometimes not. It depends on how Image creation has been started. This "inconsistency" can lead to problems when we want to find that specific file in SSP which contains our Image in PowerVC.
Images and LUs are stored in 2 different places in SSP:
# ls -l /var/vio/SSP/SSP_Cluster_1/D_E_F_A_U_L_T_061310
drwxr-xr-x 2 root system 8192 Feb 19 11:19 IM <--Images (these are LUs, but "lu" commands will not list them)
drwxr-xr-x 8 root system 512 Feb 03 2018 VIOSCFG
drwxr-xr-x 2 root system 32768 Feb 20 08:47 VOL1 <--LUs
Volume and LU:
Both are referring to the same disk, just Volume is used in PowerVC, and LU is used in SSP commands. When we create a PowerVC Volume, in the background PowerVC will create a new disk in SSP. The end result is a file in the VOL1 directory (/var/vio/SSP/SSP_Cluster_1/D_E_F_A_U_L_T_061310/VOL1). The lu ... SSP commands will list files in VOL1 directory (not in IM).
Image and Snapshot:
PowerVC images are needed to create new AIX VMs. The word Image is used only in PowerVC (not in SSP commands). Images can be created in 2 different ways:
-capturing a VM (in PowerVC VMs menu): a new file is created in IM directory and it takes minutes
-creating from a Volume (in PowerVC Images menu): no new files are created (same LU is used from VOL1), it takes seconds
When an Image is ready, we can use it to create (deploy) new AIX VMs. During deployment a "snapshot" is created. A snapshot is a link to the disk from which it was created, and this snapshot is the "disk" of the new AIX VM. The actual size of this AIX VM in SSP is very minimal (sometimes I could not recognize the change), because it contains the same data as the Image.
When we create a Volume in PowerVC, a LU is created in SSP. This LU can exist without any assignment or dependency. When we modify this LU to be an Image in PowerVC it is still "independent". But when we deploy AIX VMs from this Image, snapshots will be created which are depending on the LU. It means we cannot remove this LU (or Image) file until there are snapshots which are referring to it. (In PowerVC GUI, we can remove images at any time, but this is a "fake" removal. PowerVC will show that it deleted successfully an image, but in the background if snapshots from that image still exist, no free space will be reclaimed, the used storage space will be the same. If we check "snapshot -list" command we could be surprised to find a lots of images in the output, which does not exist in PowerVC anymore, but still exist in SSP.)
-------------------------------------
snapshot -list:
This command displays all Images and its Snapshots. It does not matter how Images were created (from a Volume or from a VM) it will be listed. The output has 2 parts. The first part (which start with "Lu Name") will list Images which are created from Volumes (in VOL1 directory) and its Snapshots. The second part (which start from "Lu(Client Image)Name") will lists Images in IM directory and its Snapshots.
$ snapshot -list
Lu Name Size(mb) ProvisionType %Used Unused(mb) Lu Udid
volume-bb_dd_61-c20cc6.. 153600 THIN 0% 153609 9191dee4a3ba... <--this is a Volume and Image (in VOL1)
Snapshot
72a40f070213e2450b8d19672f22a5dcIMSnap <--this is a VM, shows LUN id (without IBMsnap)
Lu(Client Image)Name Size(mb) ProvisionType %Used Unused(mb) Lu Udid
volume-Image_7241-5c5b80ac-170b153 THIN 0% 153609 48004e96ecc2... <--this is an image in IM
Snapshot
c397bc118de59c4592429b2eb0bba738IMSnap <--this is a VM with LUN id
618afc4622c5286808a8173468ae161bIMSnap <--this is a VM with LUN id
-------------------------------------
lu -list
This command will list all LUs and if a LU is functioning as an Image (Images in VOL1 dir) its Snapshots will be also displayed. Images which are in IM directory (captured from a VM) are not displayed here.
$ lu -list
POOL_NAME: SSP_1
TIER_NAME: System
LU_NAME SIZE(MB) UNUSED(MB) UDID
volume-aix-central-111~ 153600 84140 81d15130e9a76596ad0b3564973d4912
volume-bb_dd_61-c20cc6~ 153600 153609 9191dee4a3ba8fe3c7753af592027aad <--this is a Volume and Image (in VOL1)
SNAPSHOTS
72a40f070213e2450b8d19672f22a5dcIMSnap <--its Snapshot (AIX VM is created from Image)
volume-bb_dd_61_VM-1ce~ 153600 153160 72a40f070213e2450b8d19672f22a5dc <--LU of the deployed AIX (same IDs)
volume-cluster_1-71eda~ 20480 20473 e12ba154470b2c8bd9a54eb588fc9d2e
volume-cluster_2-c8ab7~ 20480 20432 4e36795de9a3f90710ba995b97d7ccbd
-------------------------------------
Image removal if it is listed in snapshot -list command:
1. $ snapshot -list
….
volume-Image_ls-aix-test8_capture_1_volume_1-fc19d9fd-5dac153600 THIN 1% 150942 8256526c4e512b54cbdd689d4e1e321a
2. $ lu -remove -luudid 8256526c4e512b54cbdd689d4e1e321a
Logical unit with udid "8256526c4e512b54cbdd689d4e1e321a" is removed.
After that file will be deleted in IM directory as well
-------------------------------------
Image removal if it is not listed in snapshot commands:
In this case there are files in IM directory, but usual SSP commands will not list them. It is possible to manually remove those files (rm), but it works only if these images (LUNs) are not listed as "LU_UDID_DERIVED_FROM"
1. Checked files in IM direcory:
ls -ltr /var/vio/SSP/SSP_Cluster_1/D_E_F_A_U_L_T_061310/IM
volume-Image_AIX-710404_base2_puppet_capture-445b36df-bbee.b38417a6fd9f8eab66c2f7e6e02818cc
It lists PowerVC images (some were not listed in PowerVC GUI). The characters after the dot (.) showing the LUN id, which can be used in searches.
2. Searching LUN ids in "lu -list verbose":
for i in `ls -ltr /var/vio/SSP/SSP_Cluster_1/D_E_F_A_U_L_T_061310/IM | awk -F'.' '{print $2}'`; do ls /var/vio/SSP/SSP_Cluster_1/D_E_F_A_U_L_T_061310/IM| grep $i; lu -list -verbose| grep -p $i; echo "========="; echo; done
3. Remove files if possible:
If there are no reference for a specific Image file (LUN id) in this verbose command then "rm filename" will work, otherwise, we get this error:
# rm volume-Image_AIX-710404_base_nocloud_capture-0d466084-d9bb.054d6d02cf133e4aef56437f4524f016
rm: 0653-609 Cannot remove volume-Image_AIX-710404_base_nocloud_capture-0d466084-d9bb.054d6d02cf133e4aef56437f4524f016.
Operation not permitted.
-------------------------------------
SSP dd examples
# cd /var/vio/SSP/SSP_Cluster_1/D_E_F_A_U_L_T_061310/IM
To export the volume to a file from SSP:
# dd if=volume-New-SSP-Image-Volume.7e2e5b5738d7adf4be3b64b9b731c2ff of=/tmp/aix7_img bs=1M
To import the volume from a file to SSP:
# dd if=/tmp/aix7_img of=VOL1/volume-New-SSP-Image-Volume.7e2e5b5738d7adf4be3b64b9b731c2ff bs=1M
-------------------------------------
Hi can someone please post the link for vios 2.xx and 3.xx for the LPAR CREATION on real power 6 7 systems. thanks
ReplyDelete