Friday, July 28, 2017

UNIX : Forcibly Unmounting NFS filesystems on HPUX

Friday, July 28, 2017 0 Comments
There comes a time when you have to do what you have to do

When you have tried the usual umount command to unmount a NFS but it has failed due to many reasons (user using the NFS, processes run by user using the NFS, waiting didn't do it, etc)

Here are some alternative steps to unmounting an NFS



1. Attempt to unmount with the usual umount command failed

2. Use the fuser command to find out process IDs, user running/using the NFS

3. Do a ps to find the offending processes

4. Do a fuser with kill option to kill the processed holding up the NFS

5. Unmount the NFS again, now that all other processes have been terminated

And here's an important message when running fuser command


LINUX : Shrink JFS2 File System on LINUX

Friday, July 28, 2017 0 Comments
I once had a request when an engineer incorrectly extended a different file system than requested. These kind of human error always happen because hey, we're not all that perfect (and sometimes misreading and misunderstanding the user's request)

So I had to correct his mistake and get back to the customer because I was on great terms with the team. Yups, that happens too

Anyhow, here goes

What happens if we resized the wrong FS??

You start with the problem

Many thanks, but there is one problem, as per my initial request I needed /oracle/dat4 not /oracle/dat3

Make sure no one is using/accessing the file systems and resize2fs is installed on the server and the FS was created with JFS2

# umount /oracle/dat3

# resize2fs /dev/vg04/lvol2 20464M (this is the original size before we went about clobbering it)

# lvreduce -L 20464M /dev/vg04/lvol2

# mount /oracle/dat3

I had forgotten exactly which OS this Linux machine was running on. Hence the title Linux

UNIX : Replace/Mirror disks with no downtime on HPUX

Friday, July 28, 2017 0 Comments
This tutorial is simple, to replace disks on a server, without a downtime (live) , so mirroring is done to copy data from old disks to new disks  and release the old disks to SAN. I trust you have taken the necessary steps to an ignite, health check, backup, Change Request, etc. Tested on HPUX 11.23

We start by adding new mirror disks to the server

Create a PV for all newly added LUN. We use the raw character of a disk whenever we're adding new a disk to a server. I highlighted this in yellow

super22:/etc# pvcreate /dev/rdsk/c44t10d2
Physical volume "/dev/rdsk/c44t10d2" has been successfully created.

super22:/etc# pvcreate /dev/rdsk/c45t10d2
Physical volume "/dev/rdsk/c45t10d2" has been successfully created.

super22:/etc# pvcreate /dev/rdsk/c44t10d3
Physical volume "/dev/rdsk/c44t10d3" has been successfully created.

super22:/etc# pvcreate /dev/rdsk/c45t10d3
Physical volume "/dev/rdsk/c45t10d3" has been successfully created.

super22:/etc# pvcreate /dev/rdsk/c44t10d4
Physical volume "/dev/rdsk/c44t10d4" has been successfully created.

super22:/etc# pvcreate /dev/rdsk/c45t10d4
Physical volume "/dev/rdsk/c45t10d4" has been successfully created.

super22:/etc# pvcreate /dev/rdsk/c44t10d5
Physical volume "/dev/rdsk/c44t10d5" has been successfully created.

super22:/etc# pvcreate /dev/rdsk/c45t10d5
Physical volume "/dev/rdsk/c45t10d5" has been successfully created.
super22:/etc# 

1) Append in the /etc/lvmpvg and add the new mirror disks. Note that I have not used the raw device character when adding disk to lvmpvg

PVG     PVG1
/dev/dsk/c44t10d2
/dev/dsk/c44t10d3
/dev/dsk/c44t10d4

/dev/dsk/c44t10d5

2) We then extend VG vgSPAN with the new mirror disks

super22:/etc# vgextend vgSPAN /dev/dsk/c44t10d2 /dev/dsk/c44t10d3 /dev/dsk/c44t10d4 /dev/dsk/c44t10d5
Current path "/dev/dsk/c35t8d2" is an alternate link, skip.
Current path "/dev/dsk/c33t8d3" is an alternate link, skip.
Current path "/dev/dsk/c35t8d4" is an alternate link, skip.
Current path "/dev/dsk/c33t8d5" is an alternate link, skip.
Current path "/dev/dsk/c35t8d6" is an alternate link, skip.
Current path "/dev/dsk/c33t8d7" is an alternate link, skip.
Current path "/dev/dsk/c45t9d4" is an alternate link, skip.
Current path "/dev/dsk/c45t9d5" is an alternate link, skip.
Current path "/dev/dsk/c45t9d6" is an alternate link, skip.
Current path "/dev/dsk/c45t9d7" is an alternate link, skip.
Current path "/dev/dsk/c45t10d0" is an alternate link, skip.
Current path "/dev/dsk/c45t10d1" is an alternate link, skip.
Volume group "vgSPAN" has been successfully extended.
Volume Group configuration for /dev/vgSPAN has been saved in /etc/lvmconf/vgSPAN.conf

super22:/etc# vgextend vgSPAN /dev/dsk/c45t10d2 /dev/dsk/c45t10d3 /dev/dsk/c45t10d4 /dev/dsk/c45t10d5
Current path "/dev/dsk/c35t8d2" is an alternate link, skip.
Current path "/dev/dsk/c33t8d3" is an alternate link, skip.
Current path "/dev/dsk/c35t8d4" is an alternate link, skip.
Current path "/dev/dsk/c33t8d5" is an alternate link, skip.
Current path "/dev/dsk/c35t8d6" is an alternate link, skip.
Current path "/dev/dsk/c33t8d7" is an alternate link, skip.
Current path "/dev/dsk/c45t9d4" is an alternate link, skip.
Current path "/dev/dsk/c45t9d5" is an alternate link, skip.
Current path "/dev/dsk/c45t9d6" is an alternate link, skip.
Current path "/dev/dsk/c45t9d7" is an alternate link, skip.
Current path "/dev/dsk/c45t10d0" is an alternate link, skip.
Current path "/dev/dsk/c45t10d1" is an alternate link, skip.
Volume group "vgSPAN" has been successfully extended.
Volume Group configuration for /dev/vgSPAN has been saved in /etc/lvmconf/vgSPAN.conf

super22:/etc# 

View the VG while you're at it

super22:/etc# vgdisplay vgSPAN
--- Volume groups ---
VG Name                     /dev/vgSPAN
VG Write Access             read/write     
VG Status                   available                 
Max LV                      255    
Cur LV                      5      
Open LV                     5      
Max PV                      16     
Cur PV                      16     
Act PV                      16     
Max PE per PV               16000        
VGDA                        32  
PE Size (Mbytes)            32              
Total PE                    34160   
Alloc PE                    14066   
Free PE                     20094   
Total PVG                   2        
Total Spare PVs             0              
Total Spare PVs in use      0    


And here is the list of old disks at the LVs of VG vgSPAN. 

Old DISKS at LVs of VgSPAN:

LV1

--- Logical volumes ---
LV Name                     /dev/vgSPAN/lvol1
VG Name                     /dev/vgSPAN
LV Permission               read/write   
LV Status                   available/syncd           
Mirror copies               0   ----------->Not mirrored yet          
Consistency Recovery        MWC                 
Schedule                    parallel     
LV Size (Mbytes)            4224            
Current LE                  132       
Allocated PE                132         
Stripes                     0       
Stripe Size (Kbytes)        0                   
Bad block                   NONE         
Allocation                  PVG-strict/distributed
IO Timeout (Seconds)        default             

   --- Distribution of logical volume ---
   PV Name            LE on PV  PE on PV  
   /dev/dsk/c33t8d2   22        22        
   /dev/dsk/c35t8d3   22        22        
   /dev/dsk/c33t8d4   22        22        
   /dev/dsk/c35t8d5   22        22        
   /dev/dsk/c33t8d6   22        22        
   /dev/dsk/c35t8d7   22        22  

####################################################################

LV2:

super22:/etc# lvdisplay -v /dev/vgSPAN/lvol2 | more
--- Logical volumes ---
LV Name                     /dev/vgSPAN/lvol2
VG Name                     /dev/vgSPAN
LV Permission               read/write   
LV Status                   available/syncd           
Mirror copies               0 ---------->Not mirrored yet            
Consistency Recovery        MWC                 
Schedule                    parallel     
LV Size (Mbytes)            4224            
Current LE                  132       
Allocated PE                132         
Stripes                     0       
Stripe Size (Kbytes)        0                   
Bad block                   NONE         
Allocation                  PVG-strict/distributed
IO Timeout (Seconds)        default             

   --- Distribution of logical volume ---
   PV Name            LE on PV  PE on PV  
   /dev/dsk/c33t8d2   22        22        
   /dev/dsk/c35t8d3   22        22        
   /dev/dsk/c33t8d4   22        22        
   /dev/dsk/c35t8d5   22        22        
   /dev/dsk/c33t8d6   22        22        
   /dev/dsk/c35t8d7   22        22  

####################################################################

LV3:

super22:/etc# lvdisplay -v /dev/vgSPAN/lvol3 | more
--- Logical volumes ---
LV Name                     /dev/vgSPAN/lvol3
VG Name                     /dev/vgSPAN
LV Permission               read/write   
LV Status                   available/syncd           
Mirror copies               0 --------->Not mirrored yet            
Consistency Recovery        MWC                 
Schedule                    parallel     
LV Size (Mbytes)            71680           
Current LE                  2240      
Allocated PE                2240        
Stripes                     0       
Stripe Size (Kbytes)        0                   
Bad block                   NONE         
Allocation                  PVG-strict/distributed
IO Timeout (Seconds)        default             

   --- Distribution of logical volume ---
   PV Name            LE on PV  PE on PV  
   /dev/dsk/c33t8d2   214       214       
   /dev/dsk/c35t8d3   214       214       
   /dev/dsk/c33t8d4   214       214       
   /dev/dsk/c35t8d5   214       214       
   /dev/dsk/c33t8d6   214       214       
   /dev/dsk/c35t8d7   214       214       
   /dev/dsk/c44t9d4   160       160       
   /dev/dsk/c44t9d5   160       160       
   /dev/dsk/c44t9d6   159       159       
   /dev/dsk/c44t9d7   159       159       
   /dev/dsk/c44t10d0  159       159       
   /dev/dsk/c44t10d1  159       159 

####################################################################

LV4:

super22:/etc# lvdisplay -v /dev/vgSPAN/lvol4 | more
--- Logical volumes ---
LV Name                     /dev/vgSPAN/lvol4
VG Name                     /dev/vgSPAN
LV Permission               read/write   
LV Status                   available/syncd           
Mirror copies               0  ---------->Not mirrored yet            
Consistency Recovery        MWC                 
Schedule                    parallel     
LV Size (Mbytes)            368640          
Current LE                  11520     
Allocated PE                11520       
Stripes                     0       
Stripe Size (Kbytes)        0                   
Bad block                   NONE         
Allocation                  PVG-strict/distributed
IO Timeout (Seconds)        default             

   --- Distribution of logical volume ---
   PV Name            LE on PV  PE on PV  
   /dev/dsk/c33t8d2   694       694       
   /dev/dsk/c35t8d3   694       694       
   /dev/dsk/c33t8d4   694       694       
   /dev/dsk/c35t8d5   694       694       
   /dev/dsk/c33t8d6   694       694       
   /dev/dsk/c35t8d7   694       694       
   /dev/dsk/c44t9d4   1226      1226      
   /dev/dsk/c44t9d5   1226      1226      
   /dev/dsk/c44t9d6   1226      1226      
   /dev/dsk/c44t9d7   1226      1226      
   /dev/dsk/c44t10d0  1226      1226      

   /dev/dsk/c44t10d1  1226      1226      

####################################################################

LV5:

super22:/etc# lvdisplay -v /dev/vgSPAN/lvol5 | more
--- Logical volumes ---
LV Name                     /dev/vgSPAN/lvol5
VG Name                     /dev/vgSPAN
LV Permission               read/write   
LV Status                   available/syncd           
Mirror copies               0 ---------->Not mirrored yet            
Consistency Recovery        MWC                 
Schedule                    parallel     
LV Size (Mbytes)            1344            
Current LE                  42        
Allocated PE                42          
Stripes                     0       
Stripe Size (Kbytes)        0                   
Bad block                   NONE         
Allocation                  PVG-strict/distributed
IO Timeout (Seconds)        default             

   --- Distribution of logical volume ---
   PV Name            LE on PV  PE on PV  
   /dev/dsk/c33t8d2   7         7         
   /dev/dsk/c35t8d3   7         7         
   /dev/dsk/c33t8d4   7         7         
   /dev/dsk/c35t8d5   7         7         
   /dev/dsk/c33t8d6   7         7         
   /dev/dsk/c35t8d7   7         7         


####################################################################

3. Mirror the LVs

super22:/root# lvextend -m 1 /dev/vgSPAN/lvol1 PVG1
The newly allocated mirrors are now being synchronized. This operation will 
take some time. Please wait ....
Logical volume "/dev/vgSPAN/lvol1" has been successfully extended.
Volume Group configuration for /dev/vgSPAN has been saved in /etc/lvmconf/vgSPAN.conf
super22:/root#

(Make LV status is syncd & mirror copy is 1)

super22:/root# lvdisplay /dev/vgSPAN/lvol1 |egrep "LV Status|Mirror copies"
LV Status                   available/syncd           
Mirror copies               1 --------------->Mirrored now      
spthpn22:/root#

super22:/root# lvdisplay -v /dev/vgSPAN/lvol1 | more
--- Logical volumes ---
LV Name                     /dev/vgSPAN/lvol1
VG Name                     /dev/vgSPAN
LV Permission               read/write   
LV Status                   available/syncd           
Mirror copies               1            
Consistency Recovery        MWC                 
Schedule                    parallel     
LV Size (Mbytes)            4224            
Current LE                  132       
Allocated PE                264         
Stripes                     0       
Stripe Size (Kbytes)        0                   
Bad block                   NONE         
Allocation                  PVG-strict/distributed
IO Timeout (Seconds)        default             

   --- Distribution of logical volume ---
   PV Name            LE on PV  PE on PV  
   /dev/rdsk/c33t8d2   22        22        
   /dev/dsk/c35t8d3   22        22        
   /dev/dsk/c33t8d4   22        22        
   /dev/dsk/c35t8d5   22        22        
   /dev/dsk/c33t8d6   22        22        
   /dev/dsk/c35t8d7   22        22        
   /dev/dsk/c44t10d2  33        33 --------->new disks mirrored      
   /dev/dsk/c44t10d3  33        33 --------->new disks mirrored        
   /dev/dsk/c44t10d4  33        33 --------->new disks mirrored        
   /dev/dsk/c44t10d5  33        33 --------->new disks mirrored        


####################################################################

super22:/root# lvextend -m 1 /dev/vgSPAN/lvol2 PVG1
The newly allocated mirrors are now being synchronized. This operation will 
take some time. Please wait ....
Logical volume "/dev/vgSPAN/lvol2" has been successfully extended.
Volume Group configuration for /dev/vgSPAN has been saved in /etc/lvmconf/vgSPAN.conf
spthpn22:/root# 

super22:/root# lvdisplay /dev/vgSPAN/lvol2 |egrep "LV Status|Mirror copies"
LV Status                   available/syncd           
Mirror copies               1 --------------->Mirrored now             
spthpn22:/root# 

super22:/root# lvdisplay -v /dev/vgSPAN/lvol2 | more
--- Logical volumes ---
LV Name                     /dev/vgSPAN/lvol2
VG Name                     /dev/vgSPAN
LV Permission               read/write   
LV Status                   available/syncd           
Mirror copies               1            
Consistency Recovery        MWC                 
Schedule                    parallel     
LV Size (Mbytes)            4224            
Current LE                  132       
Allocated PE                264         
Stripes                     0       
Stripe Size (Kbytes)        0                   
Bad block                   NONE         
Allocation                  PVG-strict/distributed
IO Timeout (Seconds)        default             

   --- Distribution of logical volume ---
   PV Name            LE on PV  PE on PV  
   /dev/dsk/c33t8d2   22        22        
   /dev/dsk/c35t8d3   22        22        
   /dev/dsk/c33t8d4   22        22        
   /dev/dsk/c35t8d5   22        22        
   /dev/dsk/c33t8d6   22        22        
   /dev/dsk/c35t8d7   22        22        
   /dev/dsk/c44t10d2  33        33 --------->new disks mirrored       
   /dev/dsk/c44t10d3  33        33 --------->new disks mirrored         
   /dev/dsk/c44t10d4  33        33 --------->new disks mirrored         
   /dev/dsk/c44t10d5  33        33 --------->new disks mirrored   


####################################################################

super22:/root# lvextend -m 1 /dev/vgSPAN/lvol3 PVG1
The newly allocated mirrors are now being synchronized. This operation will 
take some time. Please wait ....
Logical volume "/dev/vgSPAN/lvol3" has been successfully extended.
Volume Group configuration for /dev/vgSPAN has been saved in /etc/lvmconf/vgSPAN.conf 

super22:/root# lvdisplay /dev/vgSPAN/lvol3 |egrep "LV Status|Mirror copies"
LV Status                   available/syncd           
Mirror copies               1 --------------->Mirrored now            
spthpn22:/root# 

super22:/root# lvdisplay -v /dev/vgSPAN/lvol3 | more
--- Logical volumes ---
LV Name                     /dev/vgSPAN/lvol3
VG Name                     /dev/vgSPAN
LV Permission               read/write   
LV Status                   available/syncd           
Mirror copies               1            
Consistency Recovery        MWC                 
Schedule                    parallel     
LV Size (Mbytes)            71680           
Current LE                  2240      
Allocated PE                4480        
Stripes                     0       
Stripe Size (Kbytes)        0                   
Bad block                   NONE         
Allocation                  PVG-strict/distributed
IO Timeout (Seconds)        default             

   --- Distribution of logical volume ---
   PV Name            LE on PV  PE on PV  
   /dev/dsk/c33t8d2   214       214       
   /dev/dsk/c35t8d3   214       214       
   /dev/dsk/c33t8d4   214       214       
   /dev/dsk/c35t8d5   214       214       
   /dev/dsk/c33t8d6   214       214       
   /dev/dsk/c35t8d7   214       214       
   /dev/dsk/c44t9d4   160       160       
   /dev/dsk/c44t9d5   160       160       
   /dev/dsk/c44t9d6   159       159       
   /dev/dsk/c44t9d7   159       159       
   /dev/dsk/c44t10d0  159       159         
   /dev/dsk/c44t10d1  159       159         
   /dev/dsk/c44t10d2  560       560 --------->new disks mirrored         
   /dev/dsk/c44t10d3  560       560 --------->new disks mirrored         
   /dev/dsk/c44t10d4  560       560 --------->new disks mirrored         
   /dev/dsk/c44t10d5  560       560 --------->new disks mirrored   


####################################################################

super22:/root# lvextend -m 1 /dev/vgSPAN/lvol4 PVG1
The newly allocated mirrors are now being synchronized. This operation will 
take some time. Please wait ....
Logical volume "/dev/vgSPAN/lvol4" has been successfully extended.
Volume Group configuration for /dev/vgSPAN has been saved in /etc/lvmconf/vgSPAN.conf
super22:/root# 

super22:/root# lvdisplay -v /dev/vgSPAN/lvol4 | more
--- Logical volumes ---
LV Name                     /dev/vgSPAN/lvol4
VG Name                     /dev/vgSPAN
LV Permission               read/write   
LV Status                   available/syncd           
Mirror copies               1            ------------->Mirrored now 
Consistency Recovery        MWC                 
Schedule                    parallel     
LV Size (Mbytes)            368640          
Current LE                  11520     
Allocated PE                23040       
Stripes                     0       
Stripe Size (Kbytes)        0                   
Bad block                   NONE         
Allocation                  PVG-strict/distributed
IO Timeout (Seconds)        default             

   --- Distribution of logical volume ---
   PV Name            LE on PV  PE on PV  
   /dev/dsk/c33t8d2   694       694       
   /dev/dsk/c35t8d3   694       694       
   /dev/dsk/c33t8d4   694       694       
   /dev/dsk/c35t8d5   694       694       
   /dev/dsk/c33t8d6   694       694       
   /dev/dsk/c35t8d7   694       694       
   /dev/dsk/c44t9d4   1226      1226      
   /dev/dsk/c44t9d5   1226      1226      
   /dev/dsk/c44t9d6   1226      1226      
   /dev/dsk/c44t9d7   1226      1226      
   /dev/dsk/c44t10d0  1226      1226      
   /dev/dsk/c44t10d1  1226      1226      
   /dev/dsk/c44t10d2  2880      2880 --------->new disks mirrored      
   /dev/dsk/c44t10d3  2880      2880 --------->new disks mirrored       
   /dev/dsk/c44t10d4  2880      2880 --------->new disks mirrored       
   /dev/dsk/c44t10d5  2880      2880 --------->new disks mirrored  


####################################################################

super22:/root# lvextend -m 1 /dev/vgSPAN/lvol5 PVG1
The newly allocated mirrors are now being synchronized. This operation will 
take some time. Please wait ....
Logical volume "/dev/vgSPAN/lvol5" has been successfully extended.
Volume Group configuration for /dev/vgSPAN has been saved in /etc/lvmconf/vgSPAN.conf

super22:/root# lvdisplay /dev/vgSPAN/lvol5 |egrep "LV Status|Mirror copies"
LV Status                   available/syncd           
Mirror copies               1 --------------->Mirrored now             
spthpn22:/root# 

super22:/root# lvdisplay -v /dev/vgSPAN/lvol5 | more
--- Logical volumes ---
LV Name                     /dev/vgSPAN/lvol5
VG Name                     /dev/vgSPAN
LV Permission               read/write   
LV Status                   available/syncd           
Mirror copies               1            
Consistency Recovery        MWC                 
Schedule                    parallel     
LV Size (Mbytes)            1344            
Current LE                  42        
Allocated PE                84          
Stripes                     0       
Stripe Size (Kbytes)        0                   
Bad block                   NONE         
Allocation                  PVG-strict/distributed
IO Timeout (Seconds)        default             

   --- Distribution of logical volume ---
   PV Name            LE on PV  PE on PV  
   /dev/dsk/c33t8d2   7         7         
   /dev/dsk/c35t8d3   7         7         
   /dev/dsk/c33t8d4   7         7         
   /dev/dsk/c35t8d5   7         7         
   /dev/dsk/c33t8d6   7         7         
   /dev/dsk/c35t8d7   7         7         
   /dev/dsk/c44t10d2  11        11 --------->new disks mirrored        
   /dev/dsk/c44t10d3  11        11 --------->new disks mirrored        
   /dev/dsk/c44t10d4  10        10 --------->new disks mirrored        
   /dev/dsk/c44t10d5  10        10 -------->new disks mirrored 



####################################################################

While mirroring is going on, to check the status of mirror-because it takes time, run this command

# lvdisplay -v /dev/vgSPAN/lvolx |grep stale|wc -l

The number of stale PE will decline and when it hits zero that's when you know mirroring has completed 

5. Once all the 5 LVs have been mirrored, you remove/un-mirror the old LUNs from all the five LVs of VG vgSPAN

# lvreduce -m 0 /dev/vgSPAN/lvol1 /dev/dsk/c33t8d2 /dev/dsk/c35t8d3 /dev/dsk/c33t8d4 /dev/dsk/c35t8d5 /dev/dsk/c33t8d6 /dev/dsk/c35t8d7

super22:/root# lvreduce -m 0 /dev/vgSPAN/lvol1 /dev/dsk/c33t8d2 /dev/dsk/c35t8d3 /dev/dsk/c33t8d4 /dev/dsk/c35t8d5 
Logical volume "/dev/vgSPAN/lvol1" has been successfully reduced.
Volume Group configuration for /dev/vgSPAN has been saved in /etc/lvmconf/vgSPAN.conf

# lvreduce -m 0 /dev/vgSPAN/lvol2 /dev/dsk/c33t8d2 /dev/dsk/c35t8d3 /dev/dsk/c33t8d4 /dev/dsk/c35t8d5 /dev/dsk/c33t8d6 /dev/dsk/c35t8d7

super22:/root# /c35t8d3 /dev/dsk/c33t8d4 /dev/dsk/c35t8d5 /dev/dsk/c33t8d6 /dev/dsk/c35t8d7                                     <
Logical volume "/dev/vgSPAN/lvol2" has been successfully reduced.
Volume Group configuration for /dev/vgSPAN has been saved in /etc/lvmconf/vgSPAN.conf


# lvreduce -m 0 /dev/vgSPAN/lvol3 /dev/dsk/c33t8d2 /dev/dsk/c35t8d3 /dev/dsk/c33t8d4 /dev/dsk/c35t8d5 /dev/dsk/c33t8d6 /dev/dsk/c35t8d7 /dev/dsk/c44t9d4 /dev/dsk/c44t9d5 /dev/dsk/c44t9d6 /dev/dsk/c44t9d7 /dev/dsk/c44t10d0 /dev/dsk/c44t10d1

super22:/root# v/dsk/c44t9d6 /dev/dsk/c44t9d7 /dev/dsk/c44t10d0 /dev/dsk/c44t10d1                                               <
Logical volume "/dev/vgSPAN/lvol3" has been successfully reduced.
Volume Group configuration for /dev/vgSPAN has been saved in /etc/lvmconf/vgSPAN.conf
super22:/root# 


# lvreduce -m 0 /dev/vgSPAN/lvol4 /dev/dsk/c33t8d2 /dev/dsk/c35t8d3 /dev/dsk/c33t8d4 /dev/dsk/c35t8d5 /dev/dsk/c33t8d6 /dev/dsk/c35t8d7 /dev/dsk/c44t9d4 /dev/dsk/c44t9d5 /dev/dsk/c44t9d6 /dev/dsk/c44t9d7 /dev/dsk/c44t10d0 /dev/dsk/c44t10d1

super22:/root# v/dsk/c44t9d6 /dev/dsk/c44t9d7 /dev/dsk/c44t10d0 /dev/dsk/c44t10d1                                               <
Logical volume "/dev/vgSPAN/lvol4" has been successfully reduced.
Volume Group configuration for /dev/vgSPAN has been saved in /etc/lvmconf/vgSPAN.conf
super22:/root# 

# lvreduce -m 0 /dev/vgSPAN/lvol5 /dev/dsk/c33t8d2 /dev/dsk/c35t8d3 /dev/dsk/c33t8d4 /dev/dsk/c35t8d5 /dev/dsk/c33t8d6 /dev/dsk/c35t8d7

super22:/root# /c35t8d3 /dev/dsk/c33t8d4 /dev/dsk/c35t8d5 /dev/dsk/c33t8d6 /dev/dsk/c35t8d7                                     <
Logical volume "/dev/vgSPAN/lvol5" has been successfully reduced.
Volume Group configuration for /dev/vgSPAN has been saved in /etc/lvmconf/vgSPAN.conf

super22:/root# 

Running lvreduce command can be tricky, and an error is bound to happen if you don't remove all the disks together in a single command line. If that happens don't panic. There's always a way to reverse engineer a bit

Say for an example I was suppose to remove 6 disks but I only removed 4, so I have to mirror back these disks into the LV it was removed from, and THEN UN-MIRROR them together

5.1. Identify 4 disks not used by lvol1 of VG vgSPAN:

/dev/dsk/c33t8d2 /dev/dsk/c35t8d3 /dev/dsk/c33t8d4 /dev/dsk/c35t8d5

5.2. Mirror the 4 disks to LVOL1 back:

lvextend -m 1 /dev/vgSPAN/lvol1 /dev/dsk/c33t8d2 /dev/dsk/c35t8d3 /dev/dsk/c33t8d4 /dev/dsk/c35t8d5

5.3. Check the mirror sync status:

lvdisplay /dev/vgSPAN/lvol1 |egrep "LV Status|Mirror copies"


5.4. Un-mirror all 6 disks together in a single command line:


lvreduce -m 0 /dev/vgSPAN/lvol1 /dev/dsk/c33t8d6 /dev/dsk/c35t8d7 /dev/dsk/c44t10d2 /dev/dsk/c44t10d3 /dev/dsk/c44t10d4  /dev/dsk/c44t10d5

####################################################################

6. Make sure all the LVs' mirror copy is 0

super22:/root# lvdisplay /dev/vgSPAN/lvol1 |egrep "LV Status|Mirror copies"
LV Status                   available/syncd           
Mirror copies               0
super22:/root# lvdisplay /dev/vgSPAN/lvol2 | egrep "LV Status|Mirror copies"
LV Status                   available/syncd           
Mirror copies               0            
spthpn22:/root#

super22:/root# lvdisplay /dev/vgSPAN/lvol3 |egrep "LV Status|Mirror copies"
LV Status                   available/syncd           
Mirror copies               0            
spthpn22:/root# 

super22:/root#  lvdisplay /dev/vgSPAN/lvol4 |egrep "LV Status|Mirror copies"
LV Status                   available/syncd           
Mirror copies               0            
spthpn22:/root# 

super22:/root#  lvdisplay /dev/vgSPAN/lvol5 |egrep "LV Status|Mirror copies"
LV Status                   available/syncd           
Mirror copies               0            
spthpn22:/root# 

####################################################################

7. Remove/release old LUN from vgSPAN

# vgreduce vgSPAN /dev/dsk/c33t8d2 /dev/dsk/c35t8d2 /dev/dsk/c35t8d3 /dev/dsk/c33t8d3 /dev/dsk/c33t8d4 /dev/dsk/c35t8d4 /dev/dsk/c35t8d5 /dev/dsk/c33t8d5 /dev/dsk/c33t8d6 /dev/dsk/c35t8d6  /dev/dsk/c35t8d7 /dev/dsk/c33t8d7   

        

# vgreduce vgSPAN /dev/dsk/c44t9d4 /dev/dsk/c45t9d4 /dev/dsk/c44t9d5 /dev/dsk/c45t9d5 /dev/dsk/c44t9d6 /dev/dsk/c45t9d6 /dev/dsk/c44t9d7 /dev/dsk/c45t9d7 /dev/dsk/c44t10d0 /dev/dsk/c45t10d0 /dev/dsk/c44t10d1 /dev/dsk/c45t10d1

8. Remove the old disks entries from /etc/lvmpvg 

PVG     PVG0
/dev/dsk/c33t8d2
/dev/dsk/c35t8d3
/dev/dsk/c33t8d4
/dev/dsk/c35t8d5
/dev/dsk/c33t8d6
/dev/dsk/c35t8d7
/dev/dsk/c44t9d4
/dev/dsk/c44t9d5
/dev/dsk/c44t9d6
/dev/dsk/c44t9d7
/dev/dsk/c44t10d0
/dev/dsk/c44t10d1

9.  Issue "rmsf /dev/dsk/cXtXdX" on following disks to completely remove them from the server

rmsf /dev/dsk/c33t8d2
rmsf /dev/dsk/c35t8d2 
rmsf /dev/dsk/c35t8d3
rmsf /dev/dsk/c33t8d3 
rmsf /dev/dsk/c33t8d4
rmsf /dev/dsk/c35t8d4 
rmsf /dev/dsk/c35t8d5
rmsf /dev/dsk/c33t8d5 
rmsf /dev/dsk/c33t8d6
rmsf /dev/dsk/c35t8d6 
rmsf /dev/dsk/c35t8d7
rmsf /dev/dsk/c33t8d7 
rmsf /dev/dsk/c44t9d4
rmsf /dev/dsk/c45t9d4 
rmsf /dev/dsk/c44t9d5
rmsf /dev/dsk/c45t9d5 
rmsf /dev/dsk/c44t9d6
rmsf /dev/dsk/c45t9d6 
rmsf /dev/dsk/c44t9d7
rmsf /dev/dsk/c45t9d7 
rmsf /dev/dsk/c44t10d0
rmsf /dev/dsk/c45t10d0
rmsf /dev/dsk/c44t10d1
rmsf /dev/dsk/c45t10d1

10. Alternatively, you can use the commands rmsf -H H/W PATH instead of the rmsf /dev/dsk/XXX

11. Return the LUNs to SAN team

/dev/rdsk/c33t8d2            e1  08  42  CL1B  04:42  OPEN-V           00045468
/dev/rdsk/c35t8d2            d3  08  42  CL2B  04:42  OPEN-V           00045468
/dev/rdsk/c35t8d3            d3  08  43  CL2B  04:43  OPEN-V           00045468
/dev/rdsk/c33t8d3            e1  08  43  CL1B  04:43  OPEN-V           00045468
/dev/rdsk/c33t8d4            e1  08  44  CL1B  04:44  OPEN-V           00045468
/dev/rdsk/c35t8d4            d3  08  44  CL2B  04:44  OPEN-V           00045468
/dev/rdsk/c35t8d5            d3  08  45  CL2B  04:45  OPEN-V           00045468
/dev/rdsk/c33t8d5            e1  08  45  CL1B  04:45  OPEN-V           00045468
/dev/rdsk/c33t8d6            e1  08  46  CL1B  04:46  OPEN-V           00045468
/dev/rdsk/c35t8d6            d3  08  46  CL2B  04:46  OPEN-V           00045468
/dev/rdsk/c35t8d7            d3  08  47  CL2B  04:47  OPEN-V           00045468
/dev/rdsk/c33t8d7            e1  08  47  CL1B  04:47  OPEN-V           00045468
/dev/rdsk/c44t9d4            e1  09  cc  CL1B  06:09  OPEN-V           00045468
/dev/rdsk/c45t9d4            d3  09  cc  CL2B  06:09  OPEN-V           00045468
/dev/rdsk/c44t9d5            e1  09  cd  CL1B  06:0a  OPEN-V           00045468
/dev/rdsk/c45t9d5            d3  09  cd  CL2B  06:0a  OPEN-V           00045468
/dev/rdsk/c44t9d6            e1  09  ce  CL1B  06:0b  OPEN-V           00045468
/dev/rdsk/c45t9d6            d3  09  ce  CL2B  06:0b  OPEN-V           00045468
/dev/rdsk/c44t9d7            e1  09  cf  CL1B  06:0c  OPEN-V           00045468
/dev/rdsk/c45t9d7            d3  09  cf  CL2B  06:0c  OPEN-V           00045468
/dev/rdsk/c44t10d0           e1  0a  d0  CL1B  06:0d  OPEN-V           00045468
/dev/rdsk/c45t10d0           d3  0a  d0  CL2B  06:0d  OPEN-V           00045468
/dev/rdsk/c44t10d1           e1  0a  d1  CL1B  06:0e  OPEN-V           00045468
/dev/rdsk/c45t10d1           d3  0a  d1  CL2B  06:0e  OPEN-V           00045468

And we're done!