Initial Situation
VG name = test
LV names = testlv1, testlv2
# mount /dev/test/testlv1 /DIR1 # mount /dev/test/testlv2 /DIR2 # ls -l /DIR1 /DIR2 /DIR1: total 14680084 -rw-r--r-- 1 root root 15032385536 Sep 28 13:03 file drwx------ 2 root root 16384 Sep 28 12:39 lost+found /DIR2: total 14680084 -rw-r--r-- 1 root root 15032385536 Sep 28 13:04 file2 drwx------ 2 root root 16384 Sep 28 12:39 lost+found
Problems: LVM Volume Group corrupted
Check Logical Volume status
# lvdisplay Couldn't find device with uuid memdmT-ge9I-Qbki-sV90-D4yU-Kz4Q-ow1pae. --- Logical volume --- LV Path /dev/test/testlv1 LV Name testlv1 VG Name test LV UUID 8jjBN7-m4wa-qxEq-rPL7-ET3E-yI9I-MxuVlo LV Write Access read/write LV Creation host, time base, 2016-09-28 12:37:00 +0200 LV Status NOT available LV Size 15.00 GiB Current LE 3840 Segments 2 Allocation inherit Read ahead sectors auto --- Logical volume --- LV Path /dev/test/testlv2 LV Name testlv2 VG Name test LV UUID 4cXopI-qBvG-hOTZ-E9j6-ktoT-ZeyM-HO3911 LV Write Access read/write LV Creation host, time base, 2016-09-28 12:38:14 +0200 LV Status NOT available LV Size 14.98 GiB Current LE 3835 Segments 2 Allocation inherit Read ahead sectors auto
The /dev/test/testlv1 and /dev/test/testlv2 are in status NOT available.
Check Volume Group status:
# vgdisplay Couldn't find device with uuid memdmT-ge9I-Qbki-sV90-D4yU-Kz4Q-ow1pae. --- Volume group --- VG Name test System ID Format lvm2 Metadata Areas 2 Metadata Sequence No 6 VG Access read/write VG Status resizable MAX LV 0 Cur LV 2 Open LV 0 Max PV 0 Cur PV 3 Act PV 2 VG Size 29.99 GiB PE Size 4.00 MiB Total PE 7677 Alloc PE / Size 7675 / 29.98 GiB Free PE / Size 2 / 8.00 MiB VG UUID cqLVSd-FhG1-bRGz-tj4u-SREG-z90p-pTbRft
The results tells us that something has happened to the integrity of the volume group’s disks: Could not find device with uuid memdmT-ge9I-Qbki-SV90-D4yU-Kz4Q-ow1pae
Check devices status:
# lvs -a -o +devices Couldn't find device with uuid memdmT-ge9I-Qbki-sV90-D4yU-Kz4Q-ow1pae. LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert Devices root base -wi-ao---- 34.38g /dev/sda5(0) swap_1 base -wi-ao---- 1.50g /dev/sda5(8801) testlv1 test -wi-----p- 15.00g /dev/sdb1(0) testlv1 test -wi-----p- 15.00g unknown device(0) testlv2 test -wi-----p- 14.98g /dev/sdb3(0) testlv2 test -wi-----p- 14.98g unknown device(1281) # pvscan Couldn't find device with uuid memdmT-ge9I-Qbki-sV90-D4yU-Kz4Q-ow1pae. PV /dev/sdb1 VG test lvm2 [10.00 GiB / 0 free] PV unknown device VG test lvm2 [10.00 GiB / 8.00 MiB free] PV /dev/sdb3 VG test lvm2 [10.00 GiB / 0 free] PV /dev/sda5 VG base lvm2 [35.88 GiB / 0 free] Total: 4 [65.87 GiB] / in use: 4 [65.87 GiB] / in no VG: 0 [0 ]
A partition (10GB) used by VG test is missed (PV unknow device).
Recovery
In this case we have recreate another sdb2 partition (type LVM)
You may be able to find the UUID for the physical volume that was overwritten by looking in the /etc/lvm/archive directory.
Look in the file VolumeGroupName_xxxx.vg for the last known valid archived LVM metadata for that volume group.
Use the --uuid and --restorefile arguments of the pvcreate command to restore the physical volume.
The following example labels the /dev/sdb2 device as a physical volume with the UUID indicated above, memdmT-ge9I-Qbki-sV90-D4yU-Kz4Q-ow1pae.
This command restores the physical volume label with the metadata information contained in test_00006-246083206.vg, the most recent good archived metatdata for volume group.
The restorefile argument instructs the pvcreate command to make the new physical volume compatible with the old one on the volume group, ensuring that the the new metadata will not be placed where the old physical volume contained data (which could happen, for example, if the original pvcreate command had used the command line arguments that control metadata placement, or it the physical volume was originally created using a different version of the software that used different defaults).
The pvcreate command overwrites only the LVM metadata areas and does not affect the existing data areas.
# pvcreate --uuid "memdmT-ge9I-Qbki-sV90-D4yU-Kz4Q-ow1pae" --restorefile /etc/lvm/archive/test_00006-246083206.vg /dev/sdb2 Couldn't find device with uuid memdmT-ge9I-Qbki-sV90-D4yU-Kz4Q-ow1pae. Physical volume "/dev/sdb2" successfully created
Check the situation now:
# lvs -a -o +devices LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert Devices root base -wi-ao---- 34.38g /dev/sda5(0) swap_1 base -wi-ao---- 1.50g /dev/sda5(8801) testlv1 test -wi------- 15.00g /dev/sdb1(0) testlv1 test -wi------- 15.00g /dev/sdb2(0) testlv2 test -wi------- 14.98g /dev/sdb3(0) testlv2 test -wi------- 14.98g /dev/sdb2(1281) # pvscan WARNING: Volume Group test is not consistent PV /dev/sdb1 VG test lvm2 [10.00 GiB / 0 free] PV /dev/sdb2 VG test lvm2 [10.00 GiB / 8.00 MiB free] PV /dev/sdb3 VG test lvm2 [10.00 GiB / 0 free] PV /dev/sda5 VG base lvm2 [35.88 GiB / 0 free] Total: 4 [65.87 GiB] / in use: 4 [65.87 GiB] / in no VG: 0 [0 ]
The partition has been added to VG test and VG at the moment is not consistent.
testlv1 and testlv2 are now OK but not available infact there is not the “a” to the attribute (see the first result).
Activation LV:
# lvchange -ay /dev/test/testlv1 # lvchange -ay /dev/test/testlv2
Status:
# lvs -a -o +devices LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert Devices root base -wi-ao---- 34.38g /dev/sda5(0) swap_1 base -wi-ao---- 1.50g /dev/sda5(8801) testlv1 test -wi-a----- 15.00g /dev/sdb1(0) testlv1 test -wi-a----- 15.00g /dev/sdb2(0) testlv2 test -wi-a----- 14.98g /dev/sdb3(0) testlv2 test -wi-a----- 14.98g /dev/sdb2(1281)
LV activated. So, now is possible to check if there are files inside.
# mount /dev/test/testlv1 /DIR1 # mount /dev/test/testlv2 /DIR2 # ls -l /DIR1 /DIR2 /DIR1: total 14680084 -rw-r--r-- 1 root root 15032385536 Sep 28 13:03 file drwx------ 2 root root 16384 Sep 28 12:39 lost+found /DIR2: total 14680084 -rw-r--r-- 1 root root 15032385536 Sep 28 13:04 file2 drwx------ 2 root root 16384 Sep 28 12:39 lost+found
Drammatic case: all PV are lost
Example
# lvs -a -o +devices Couldn't find device with uuid 'FmGRh3-zhok-iVI8-7qTD-S5BI-MAEN-NYM5Sk'. Couldn't find all physical volumes for volume group VG. Couldn't find device with uuid 'FmGRh3-zhok-iVI8-7qTD-S5BI-MAEN-NYM5Sk'. Couldn't find all physical volumes for volume group VG. ...
# pvcreate --uuid "FmGRh3-zhok-iVI8-7qTD-S5BI-MAEN-NYM5Sk" --restorefile /etc/lvm/archive/VG_00050.vg /dev/sdh1 Physical volume "/dev/sdh1" successfully created
# vgcfgrestore VG Restored volume group VG
# lvs -a -o +devices LV VG Attr LSize Origin Snap% Move Log Copy% Devices stripe VG -wi--- 300.00G /dev/sdh1 (0),/dev/sda1(0) stripe VG -wi--- 300.00G /dev/sdh1 (34728),/dev/sdb1(0)
The following commands activate the volumes and display the active volumes.
# lvchange -ay /dev/VG/stripe # lvs -a -o +devices LV VG Attr LSize Origin Snap% Move Log Copy% Devices stripe VG -wi-a- 300.00G /dev/sdh1 (0),/dev/sda1(0) stripe VG -wi-a- 300.00G /dev/sdh1 (34728),/dev/sdb1(0)
