Proxmox Virtual Environment (PVE) is an open source server virtualization management solution based on QEMU/KVM and LXC. You can manage virtual machines, containers, highly available clusters, storage and networks with an integrated, easy-to-use web interface or via CLI. Proxmox VE code is licensed under the GNU Affero General Public License, version 3. The project is developed and maintained by Proxmox Server Solutions GmbH.
Step 1: Upgrade from 3.x to 4.4
In-place upgrades are done with apt, so make sure that you are familiar with apt before you start here.
Preconditions
- upgraded to latest V3.4 version
- reliable access to all configured storages
- healthy cluster
- no VM or CT running (note: VM live migration from 3.4 to 4.x node or vice versa NOT possible)
- valid backup of all OpenVZ containers (needed for the conversion to LXC)
- valid backup of all VM (only needed if something goes wrong)
- Correct repository configuration (accessible both wheezy and jessie)
- at least 1GB free disk space at root mount point
Actions Step by Step
All has to be done on each Proxmox node’s command line (via console or ssh; preferably via console in order to exclude interrupted ssh connections) , some of the steps are optional. If a whole cluster should be upgraded, keep in mind the cluster name and HA configuration like failoverdomains, fencing etc since these have to be restored after upgrade by the new WEB GUI. Again, make sure that you have a valid backup of all CT and VM before you start.
Tip: It is advisable to perform a dry-run of the upgrade first. Install the PVE 3.4 ISO on testing hardware, then upgrade this installation to the latest minor version of PVE 3.4 using the test repo (see Package repositories) then copy/create relevant configurations to the test machine to replicate your production setup as closely as possible.
After a Debian version has reached EOL (end of life), its repositories go to the Debian archive. Therefore we can use this archive to get packages for our distribution. The syntax for our /etc/apt/sources.list is as follows:
deb http://archive.debian.org/debian/ wheezy main non-free contrib deb-src http://archive.debian.org/debian/ wheezy main non-free contrib
Our file /etc/apt/sources.list example:
deb http://archive.debian.org/debian/ wheezy main non-free contrib deb-src http://archive.debian.org/debian/ wheezy main non-free contrib
Our file /etc/apt/sources.list.d/pve-enterprise.list example:
#deb https://enterprise.proxmox.com/debian wheezy pve-enterprise deb http://download.proxmox.com/debian wheezy pve-no-subscription
Get public key:
gpg --keyserver keys.gnupg.net --recv-key 7638D0442B90D010 gpg -a --export 7638D0442B90D010 | apt-key add -
Remove Proxmox VE 3.x packages in order to avoid dependency errors
First make sure that your actual installation is “clean”, tentatively run
apt-get update && apt-get dist-upgrade
Then start the removal:
apt-get remove proxmox-ve-2.6.32 pve-manager corosync-pve openais-pve redhat-cluster-pve pve-cluster pve-firmware
Adapt repository locations and update the apt database, point all to jessie, e.g.:
sed -i 's/wheezy/jessie/g' /etc/apt/sources.list sed -i 's/wheezy/jessie/g' /etc/apt/sources.list.d/pve-enterprise.list
Get public key:
gpg --keyserver keys.gnupg.net --recv-key AA8E81B4331F7F50 gpg -a --export AA8E81B4331F7F50 | apt-key add -
Update the repositories data:
apt-get update
Our file /etc/apt/sources.list example after changes:
deb http://ftp.debian.org/debian jessie main contrib # security updates deb http://security.debian.org/ jessie/updates main contrib
Our file /etc/apt/sources.list.d/pve-enterprise.list example after changes:
#deb https://enterprise.proxmox.com/debian jessie pve-enterprise deb http://download.proxmox.com/debian jessie pve-no-subscription
If there is a backports line then remove it. Currently, pve-manager and ceph-common have unmet dependencies with regards to package versions in the jessie backports repo.
In case Ceph server is used: Ceph repositories for jessie can be found at http://download.ceph.com, therefore /etc/apt/sources.list.d/ceph.list will contain e.g.:
deb http://download.ceph.com/debian-hammer jessie main
You also need to install the Ceph repository key to apt, for details, check the wiki on ceph.com.
Install the new kernel (If you haven’t done that before)
Add repository proxmox jessie location
deb https://enterprise.proxmox.com/debian jessie pve-enterprise deb http://download.proxmox.com/debian jessie pve-no-subscription apt-get update
Check first what the current new kernel’s version is
apt-cache search pve-kernel | sort -V
– at the moment (July 2019) it is 4.4.134-1 – and install it:
apt-get install pve-kernel-4.4.134-1-pve pve-firmware
Upgrade the basic system to Debian Jessie
This action will consume some time – depending on the systems performance, this can take up to 60 min or even more. If you run on SSD, the dist-upgrade can be finished in 5 minutes.
Start with this step to get the initial set of upgraded packages.
apt-get upgrade
Once that’s done, move on to the remaining packages to upgrade, with:
apt-get dist-upgrade
During either of the above, you may be asked to approve of some new packages replacing configuration files. Do with them as you see fit, but they are not relevant to the Proxmox upgrade.
Reboot the system in order to activate the new kernel.
reboot
Install Proxmox VE 4.x
Finally, install the new Proxmox VE 4.x packages with one single command:
apt-get install proxmox-ve
Then you should purge configuration files from packages which are no longer needed (Note: purging vzctl will delete all files in /var/lib/vz/private and /var/lib/vz/root , only run this if you have backed up your openvz containers.):
dpkg --purge vzctl dpkg --purge redhat-cluster-pve
Remove the old kernel (not a must, but recommended), e.g. (the kernel version has to be adapted to the currently installed one – there can be more old kernels too. Use dpkg –list | grep pve-kernel to find any 2.6.* kernels to remove):
apt-get remove pve-kernel-2.6.*
After these commands, proxmox gives some error message and ignore them.
Now let’s remove all the packages marked as rc.
dpkg --list |grep "^rc" | cut -d " " -f 3 | xargs dpkg --purge
Reboot system,
reboot
Again,
apt-get install proxmox-ve
Finally, reboot and test if all is working as expected.
Optional: OpenVZ conversion
Convert the previously backed up containers to LXC, following the HowTo on Convert OpenVZ to LXC
You can also remove the obsolete OpenVZ container data from your local storage.
rm -f /etc/pve/openvz/<ct-id>.conf rm -R <storage-path>/private/*
Cluster upgrade
It is not possible to mix Proxmox VE 3.x and earlier with Proxmox VE 4.x cluster
Due to the new corosync 2.x, the cluster has to be re-established again. Please use the same clustername.
at the first node
pvecm create <clustername>
at all other nodes:
pvecm add <first-node´s-IP> -force
The HA configuration (fail-over, fencing etc.) has to be re-configured manually, now supported from WEB GUI, see High Availability Cluster 4.x
After upgrading the last node remove the V3.x cluster data:
rm /etc/pve/cluster.conf
Info: As long as you do not delete the files on the link below, Proxmox starts up with the same virtual machines.
/var/lib/vz/images/* –VM’s data files are here.
/etc/pve/qemu-server/*.conf –VM’s conf files are here.
Step 2: Upgrade from 4.X to 5.4
In-place upgrades are done with apt-get, so make sure that you are familiar with apt before you start here.
Preconditions
- upgraded to latest V 4.4
- reliable access to all configured storages
- healthy cluster
- no VM or CT running
- valid backup of all VM (needed if something goes wrong)
- Correct repository configuration
- at least 1GB free disk space at root mount point
- ensure your /boot partition, if any, has enough space for a new kernel (min 60MB) – e.g., by removing old unused kernels (see pveversion -v)
- if using Ceph, you should be already running the Ceph Luminous version, but see the caveat above
Actions Step by Step
All has to be done on each Proxmox VE node’s command line (via console or ssh; preferably via console in order to exclude interrupted ssh connections). Again, make sure that you have a valid backup of all CT and VM before you start. Further, no changes to any guest, VM nor CT, should be made during the upgrade process.
Add the PVE repositories to you installation
First make sure that your actual installation has the latest package of the Proxmox VE 4.4 release:
apt-get update && apt-get dist-upgrade
Update the Debian repository entry to stretch.
sed -i 's/jessie/stretch/g' /etc/apt/sources.list
Update the Proxmox VE repository entry to stretch.
sed -i 's/jessie/stretch/g' /etc/apt/sources.list.d/pve-enterprise.list
Add the Proxmox VE repository key:
wget http://download.proxmox.com/debian/proxmox-ve-release-5.x.gpg -O /etc/apt/trusted.gpg.d/proxmox-ve-release-5.x.gpg
More information about Package_Repositories
Replace ceph.com repositories with proxmox.com ceph repositories This step is only necessary if you have a ceph cluster on your PVE installation.
echo "deb http://download.proxmox.com/debian/ceph-luminous stretch main" > /etc/apt/sources.list.d/ceph.list
If there is a backports line then remove it. Currently the upgrade has not been tested when packages from the backports repository are installed.
Update the repositories data:
apt-get update
Upgrade the basic system to Debian Stretch and PVE 5.0
Before you can upgrade the system, you have to remove SysVinit.
apt purge insserv sysv-rc initscripts openrc
Start with this step to get the initial set of upgraded packages:
apt-get dist-upgrade
This action will consume some time – depending on the systems performance, this can take up to 60 min or even more. If you run on SSD, the dist-upgrade can be finished in 5 minutes.
During either of the above, you may be asked to approve of some new packages replacing configuration files. Do with them as you see fit, but they are not relevant to the Proxmox upgrade.
Now let’s remove all the packages marked as rc.
dpkg --list |grep "^rc" | cut -d " " -f 3 | xargs dpkg --purge
Reboot the system in order to use the new PVE kernel.
reboot
Update appliance templates.
pveam update
Breaking Changes in 5.0
Configuration defaults
Default display switched from ‘cirrus’ to ‘std’
The default display is now ‘std’ (Standard VGA card with Bochs VBE extensions), changed from the ‘cirrus’ type. Cirrus has security bugs and ‘std’ is the default since qemu 2.2
To still be able to simply live migrate VMs to another PVE 4, or an already upgraded PVE 5 host – without off time, ensure that your Proxmox VE 4 Node is up to date, i.e. you ran an:
apt update apt full-upgrade
cycle, with valid Debian Jessie and Proxmox VE 4 repositories configured!
If you are using older package versions, for example qemu-server older than version 4.0-111, you will run into problems!
Sources:
1. https://pve.proxmox.com/wiki/Upgrade_from_3.x_to_4.0
2. https://pve.proxmox.com/wiki/Upgrade_from_4.x_to_5.0