Epidemiology & Technology

Proxmox Copy CEPH Disks to another VM

One of my proxmox hosts failed. It had a VM which was not in any HA group. The storage of that VM was within the shared CEPH pool called “cephpool1” . The major issue is that for the failed host, only the VMID was visible in GUI and not the name of the VM. So i could not immediately well which VMID to restore

  • Available Host – host0
  • Failed host – host1
  • Failed VM name – DesktopXYZ

Step 1: Find the VMID associated with the VM and the VM DATA FIle NAME

ssh root@host0

cd /etc/pve/nodes/host1/qemu-server
ls -al
grep "DesktopXYZ" *.conf
cat 100.conf


agent: 1
bootdisk: scsi0
cores: 4
ide2: none,media=cdrom
memory: 8192
name: DesktopXYZ
net0: virtio=8E:1C:05:00:74:F2,bridge=vmbr0,firewall=1
numa: 0
ostype: l26
scsi0: cephpool1:vm-100-disk-0,size=21G
scsihw: virtio-scsi-pci
smbios1: uuid=fed3a3a2-8103-45c8-a350-e40512ea512d
sockets: 1
vga: vmware,memory=512
vmgenid: 3c35a8c3-f91f-4013-8be2-62783f939417Code language: CSS (css)

We now know that the HDD file is : scsi0: cephpool1:vm-100-disk-0,size=21G

Copy this line in a text editor. We will adapt this line for our new VM.

Step 2: Create a new VM on Host0

Use the GUI to create a new VM on host0: meeting same specification as the 100.conf file EXCEPT just 1 GB storage in cephpool1. I specified the VMID as 140. Do not start it. we just need the hardware config, no actual data in there.

Lets see the conf file after creation of VM 140.

ssh root@host0
cd /etc/pve/nodes/host0/qemu-server
ls -al
cat 140.conf

agent: 1
bootdisk: scsi0
cores: 4
ide2: none,media=cdrom
memory: 8192
name: DesktopXYZ2
net0: virtio=6A:B0:A1:E0:08:28,bridge=vmbr0,firewall=1
numa: 0
ostype: l26
scsi0: cephpool1:vm-140-disk-0,size=1G
scsihw: virtio-scsi-pci
smbios1: uuid=110999e7-115c-4271-90f4-c07348e8679c
sockets: 1
vga: vmware
vmgenid: 86ba6628-8dca-4f7c-ac91-bf7cd1aecbd5Code language: CSS (css)

It has the main HDD scsi0 of 1GB size in cephpool1, name being vm-140-disk-0

Step 3: Migrate the data from old to new VM

Use the rbd command to copy data to a new disk-1 for VM 140 from disk-0 of VM 100

# rbd --pool poolName copy SourceDisk poolname/DestinationDisk

rbd --pool cephpool1 copy vm-100-disk-0 cephpool1/vm-140-disk-1Code language: PHP (php)

Remember to specify the cephpool1 for the destination disk as well.

Step 4: Associate the disk with new VMID 140

Edit the 140.conf file .

Rename: scsi0 to scsi1

scsi0: cephpool1:vm-140-disk-0,size=1G to scsi1: cephpool1:vm-140-disk-0,size=1G

Add new scsi0 device pointing to the new NM disk that has been copied over

scsi0: cephpool1:vm-140-disk-1,size=21G

ssh root@host0
cd /etc/pve/nodes/host0/qemu-server
nano 140.conf 


....
scsi0: cephpool1:vm-140-disk-1,size=21G
scsi1: cephpool1:vm-140-disk-0,size=1G
....
Code language: CSS (css)

We now have two disks in the new VM 140. One 1GB disk and one 21Gb disk that contains all data of older VM.

Step 5: Reboot Happy !

You can / should delete the 1Gb disk

You can now also add the VM to a HA group

Related posts