Configure default boot image in Grub2
Today I have upgraded my Fedora server from version 33 to 38 but all did not go as planned.
I have OpenZFS installed to handle my storage and the current released version of OpenZFS (2.2.2) is not compatible with the latest kernel (6.7). I was stuck with my new kernel and unable to access my data.
For reference here is the error message I was getting:
Output:
And:
Output:
Troubleshooting
The first thing I did was to check what kernel version is supported by the latest ZFS release (2.2.2) and I found out that the latest supported kernel is 6.6. I missed it by one minor version.
Well that's not a big deal, I can just install the previous kernel and boot on it, right?
Let's see what kernels are available:
Output:
Install previous kernel
Not really what I was expecting. I was hoping to see the previous kernel version 6.6 but it's not there.
Let's try to install the 6.2.9 version and see what happens:
Let's reboot and see if it works.
Output:
After rebooting I was still booting on the latest kernel 6.7.
Let's plug a monitor and see what's happening during the boot process.
Grub2
During the boot process, I got the Grub2 menu and I was able to select the previous kernel version 6.2.9 but it was not the default boot image. I had to manually select it every time I rebooted the server.
Troubleshooting again
Let's have a look at the Grub2 configuration to see what's the default configuration:
Output:
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL_OUTPUT="console"
GRUB_CMDLINE_LINUX="resume=/dev/mapper/fedora_nubs-swap rd.lvm.lv=fedora_nubs/root rd.lvm.lv=fedora_nubs/swap rhgb quiet"
GRUB_DISABLE_RECOVERY="true"
GRUB_ENABLE_BLSCFG=true
Hum. GRUB_DEFAULT=saved. Loooks like the default boot image is saved somewhere. Let's have a look at the Grub2 configuration file:
Output:
set default="${saved_entry}"
if [ "${prev_saved_entry}" ]; then
set saved_entry="${prev_saved_entry}"
save_env saved_entry
set prev_saved_entry=
save_env prev_saved_entry
function savedefault {
saved_entry="${chosen}"
save_env saved_entry
The grub2 configuration file is quite complex and I don't want to mess with it but it seeams that it sets the default boot image to the saved_entry variable.
I found out there is a binary to work with grub2 environment variables: grub2-editenv.
Output:
saved_entry=8bfb6e63623d4ee6aab1634ebafdd5d4-5.6.15-200.fc31.x86_64
kernelopts=root=/dev/mapper/fedora_nubs-root ro resume=/dev/mapper/fedora_nubs-swap rd.lvm.lv=fedora_nubs/root rd.lvm.lv=fedora_nubs/swap rhgb quiet systemd.unified_cgroup_hierarchy=0
boot_success=0
boot_indeterminate=1
5.6 ! That's an old kernel version. I guess my previous upgrade from Fedora 31 was not as clean as I thought.
How can I change the default boot image ? And what is this long hexadecimal string at the beginning of the saved_entry variable ?
Changing the default boot image
We first need to find the list of available boot images. I found another handy binary to do that grubby.
Output:
index=0
kernel="/boot/vmlinuz-6.7.3-100.fc38.x86_64"
args="ro resume=/dev/mapper/fedora_nubs-swap rd.lvm.lv=fedora_nubs/root rd.lvm.lv=fedora_nubs/swap rhgb quiet systemd.unified_cgroup_hierarchy=0"
root="/dev/mapper/fedora_nubs-root"
initrd="/boot/initramfs-6.7.3-100.fc38.x86_64.img"
title="Fedora Linux (6.7.3-100.fc38.x86_64) 38 (Server Edition)"
id="8bfb6e63623d4ee6aab1634ebafdd5d4-6.7.3-100.fc38.x86_64"
index=1
kernel="/boot/vmlinuz-6.5.12-100.fc37.x86_64"
args="ro resume=/dev/mapper/fedora_nubs-swap rd.lvm.lv=fedora_nubs/root rd.lvm.lv=fedora_nubs/swap rhgb quiet systemd.unified_cgroup_hierarchy=0"
root="/dev/mapper/fedora_nubs-root"
initrd="/boot/initramfs-6.5.12-100.fc37.x86_64.img"
title="Fedora Linux (6.5.12-100.fc37.x86_64) 37 (Server Edition)"
id="8bfb6e63623d4ee6aab1634ebafdd5d4-6.5.12-100.fc37.x86_64"
index=2
kernel="/boot/vmlinuz-6.2.9-300.fc38.x86_64"
args="ro resume=/dev/mapper/fedora_nubs-swap rd.lvm.lv=fedora_nubs/root rd.lvm.lv=fedora_nubs/swap rhgb quiet systemd.unified_cgroup_hierarchy=0"
root="/dev/mapper/fedora_nubs-root"
initrd="/boot/initramfs-6.2.9-300.fc38.x86_64.img"
title="Fedora Linux (6.2.9-300.fc38.x86_64) 38 (Server Edition)"
id="8bfb6e63623d4ee6aab1634ebafdd5d4-6.2.9-300.fc38.x86_64"
index=3
kernel="/boot/vmlinuz-0-rescue-8bfb6e63623d4ee6aab1634ebafdd5d4"
args="ro resume=/dev/mapper/fedora_nubs-swap rd.lvm.lv=fedora_nubs/root rd.lvm.lv=fedora_nubs/swap rhgb quiet systemd.unified_cgroup_hierarchy=0"
root="/dev/mapper/fedora_nubs-root"
initrd="/boot/initramfs-0-rescue-8bfb6e63623d4ee6aab1634ebafdd5d4.img"
title="Fedora (0-rescue-8bfb6e63623d4ee6aab1634ebafdd5d4) 30 (Thirty)"
id="8bfb6e63623d4ee6aab1634ebafdd5d4-0-rescue"
Wait a minute. The output list is really similar to what I have on my monitor during the boot process. And the id field as the same format as the saved_entry variable.
I guess if I change the saved_entry variable to the id of the boot image I want to boot on, it should work.
Checking the saved_entry variable:
Output:
saved_entry=8bfb6e63623d4ee6aab1634ebafdd5d4-6.2.9-300.fc38.x86_64
kernelopts=root=/dev/mapper/fedora_nubs-root ro resume=/dev/mapper/fedora_nubs-swap rd.lvm.lv=fedora_nubs/root rd.lvm.lv=fedora_nubs/swap rhgb quiet systemd.unified_cgroup_hierarchy=0
boot_success=1
boot_indeterminate=1
Looks good. Time for the final test. Rebooting the server...
I see a the Grub2 menu and the default selected image is the one I set. Don't move ... 5 ... 4 ... 3 ... 2 ... 1 ... 0.
It boots !
Last check:
Output:
And my ZFS ?
Output:
Argh. I forgot to reinstall the ZFS modules for the new kernel. Let's do that.
Output:
It's alive !
Conclusion
I hope this guide will help you to configure the default boot image in Grub2. It's a simple task but it can be tricky if you don't know where to look. Also be very careful when playing with Grub2, you can easily break your system. Always have a backup plan (I didn't ... YOLO).