Proxmox Copy CEPH Disks to another VM

One of my proxmox hosts failed. It had a VM which was not in any HA group. The storage of that VM was within the shared CEPH pool called “cephpool1” . The major issue is that for the failed host, only the VMID was visible in GUI and not the name of the VM. So i could not immediately well which VMID to restore

  • Available Host – host0
  • Failed host – host1
  • Failed VM name – DesktopXYZ

Step 1: Find the VMID associated with the VM and the VM DATA FIle NAME

ssh root@host0 cd /etc/pve/nodes/host1/qemu-server ls -al grep "DesktopXYZ" *.conf cat 100.conf agent: 1 bootdisk: scsi0 cores: 4 ide2: none,media=cdrom memory: 8192 name: DesktopXYZ net0: virtio=8E:1C:05:00:74:F2,bridge=vmbr0,firewall=1 numa: 0 ostype: l26 scsi0: cephpool1:vm-100-disk-0,size=21G scsihw: virtio-scsi-pci smbios1: uuid=fed3a3a2-8103-45c8-a350-e40512ea512d sockets: 1 vga: vmware,memory=512 vmgenid: 3c35a8c3-f91f-4013-8be2-62783f939417

We now know that the HDD file is : scsi0: cephpool1:vm-100-disk-0,size=21G

Copy this line in a text editor. We will adapt this line for our new VM.

Step 2: Create a new VM on Host0

Use the GUI to create a new VM on host0: meeting same specification as the 100.conf file EXCEPT just 1 GB storage in cephpool1. I specified the VMID as 140. Do not start it. we just need the hardware config, no actual data in there.

Lets see the conf file after creation of VM 140.

ssh root@host0 cd /etc/pve/nodes/host0/qemu-server ls -al cat 140.conf agent: 1 bootdisk: scsi0 cores: 4 ide2: none,media=cdrom memory: 8192 name: DesktopXYZ2 net0: virtio=6A:B0:A1:E0:08:28,bridge=vmbr0,firewall=1 numa: 0 ostype: l26 scsi0: cephpool1:vm-140-disk-0,size=1G scsihw: virtio-scsi-pci smbios1: uuid=110999e7-115c-4271-90f4-c07348e8679c sockets: 1 vga: vmware vmgenid: 86ba6628-8dca-4f7c-ac91-bf7cd1aecbd5

It has the main HDD scsi0 of 1GB size in cephpool1, name being vm-140-disk-0

Step 3: Migrate the data from old to new VM

Use the rbd command to copy data to a new disk-1 for VM 140 from disk-0 of VM 100

# rbd --pool poolName copy SourceDisk poolname/DestinationDisk rbd --pool cephpool1 copy vm-100-disk-0 cephpool1/vm-140-disk-1

Remember to specify the cephpool1 for the destination disk as well.

Step 4: Associate the disk with new VMID 140

Edit the 140.conf file .

Rename: scsi0 to scsi1

scsi0: cephpool1:vm-140-disk-0,size=1G to scsi1: cephpool1:vm-140-disk-0,size=1G

Add new scsi0 device pointing to the new NM disk that has been copied over

scsi0: cephpool1:vm-140-disk-1,size=21G

ssh root@host0 cd /etc/pve/nodes/host0/qemu-server nano 140.conf .... scsi0: cephpool1:vm-140-disk-1,size=21G scsi1: cephpool1:vm-140-disk-0,size=1G ....

We now have two disks in the new VM 140. One 1GB disk and one 21Gb disk that contains all data of older VM.

Step 5: Reboot Happy !

You can / should delete the 1Gb disk

You can now also add the VM to a HA group