Bug 15716 (initramfs, kernelpanic, switch_root)

Summary:	initrdscripts ignores virtual runtime "base-utils", hard dependency on busybox
Product:	[Build System, Metadata & Runtime] OE-Core	Reporter:	jbk <jbk>
Component:	core	Assignee:	Christos Gavros <gavrosc>
Status:	RESOLVED FIXED	QA Contact:
Severity:	normal
Priority:	Medium+	CC:	gavrosc, kexin.hao, meta.mr.watcher, meta.watcher, randy.macleod, yoann.congal
Version:	5.99
Target Milestone:	5.2 M3
Hardware:	Other
OS:	x86_64
Whiteboard:
OS type for building Yocto:	---	Type of Regression:	---
Verified:		Documentation change:	Don't know

Description jbk 2025-01-14 10:13:24 UTC

I recently had an issue with including an initramfs in my build, which I tracked down to the following interaction between the "finish" initrdscript and the virtual runtime for "base-utils" used in the initramfs images of oe-core.

The issue:
The initrdscript "finish" of the initramfs-framework is not provider-/runtime-agnostic. It requires busybox as virtual runtime for "base-utils". Changing the base-utils to "packagegroup-core-base-utils" breaks the aforementioned "finish" script, leading to a kernel panic as the initramfs cannot transition to the final userspace / final rootfs.

The cause:
Reason is line 44, executing the switch_root command. switch_root is either provided by busybox or by util-linux, while the latter is shipped with packagegroup-core-base-utils. (Or maybe even a custom/another implementation in rare cases).
However: The invokation as in line 44 of the "finish" script needs the "-c" option , which is busybox specific and not supported by the util-linux implementation.
See switch_root as implemented by util-linux: https://man.archlinux.org/man/core/util-linux/switch_root.8.en
And as implemented by busybox: https://www.busybox.net/downloads/BusyBox.html

Hence, when not using busybox as base-utils provider, inlcuding a initramfs based on the initrdscripts shipped with oe-core, it fails to transition to the final userspace, leading to a kernel panic.

Meaning:
1. Neither core-image-initramfs-boot nor core-image-minimal-initramfs consider this behavior. Both include the base-utils as specified in the virtual runtime (${VIRTUAL-RUNTIME_base-utils}). As soon as you change the virtual runtime in your build, those images break, leading to a kernel panic upon boot.
2. You have to either use "busybox" as hard-dependency in your own initramfs or ship your own initrdscripts

Version: I worked with the current master, however I see that the busybox specific switch_root was used ever since the "finish" script was included in oe-core. Same with the initramfs images shipped with oe-core, which seem to have used the virtual runtime for base-utils for quite a while, therefore breaking when not using busybox as base-utils.

Reproduce as follows:

- Chane your base-util provier:
PREFERRED_PROVIDER_virtual/base-utils = "packagegroup-core-base-utils"
VIRTUAL-RUNTIME_base-utils ?= "packagegroup-core-base-utils"

- Include one of the initramfs images shipped with oe-core in your build:
INITRAMFS_IMAGE = "core-image-minimal-initramfs"

- Build an image of your choice, including the initramfs
- Boot will lead to a kernel panic

Possible resolution:
- Either make the initrdscripts provider-agnostic by removing the "-c" option, or by only using the "-c" option when busybox is the switch_root implementation
- Or use busybox as dependency in the initramfs images shipped with oe-core, possibly mentioning in the docs that you cannot use another virtual provider for base-utils in conjunction with the initrdscripts as shipped by oe-core.

I hope I provided enough information to confirm and possibly resolve this bug.

Comment 1 Randy MacLeod 2025-01-16 15:42:57 UTC

Kevin may have an idea about how to handle this bug but I'm just CCing him for now. Other people are welcome to submit a patch if they have one.

Comment 2 Christos Gavros 2025-03-15 12:54:35 UTC

hi jbk,

thank you for the description. It is very detailed. I tried to reproduce the issue.

I set the following variables in local.conf:
PREFERRED_PROVIDER_virtual/base-utils = "packagegroup-core-base-utils"
VIRTUAL-RUNTIME_base-utils ?= "packagegroup-core-base-utils"
INITRAMFS_IMAGE = "core-image-minimal-initramfs"
INITRAMFS_MAXSIZE = "160000"

execute: bitbake core-image-minimal

Then i tried to use qemu( runqemu qemux86_64 ) but i couldnt reproduce it.

- Are the above settings ok? if yes then probably qemu can not reproduce this behavior

- Did you use another hardware? 

Br
Christos

Comment 3 Randy MacLeod 2025-03-20 14:59:27 UTC

JBK - ping.

Comment 4 Christos Gavros 2025-03-21 11:49:02 UTC

hi

i reproduced it using qemu.

add lines in local.conf:
PREFERRED_PROVIDER_virtual/base-utils = "packagegroup-core-base-utils"
VIRTUAL-RUNTIME_base-utils ?= "packagegroup-core-base-utils"
INITRAMFS_IMAGE = "core-image-minimal-initramfs"
INITRAMFS_MAXSIZE = "160000"
INITRAMFS_IMAGE_BUNDLE = "1"
IMAGE_FSTYPES += "cpio.gz"

Run: bitbake core-image-minimal-initramfs
Run: qemu-system-x86_64 -kernel tmp/deploy/images/qemux86-64/bzImage \
  -initrd tmp/deploy/images/qemux86-64/core-image-minimal-initramfs-qemux86-64.cpio.gz \
  -nographic -append "console=ttyS0 rdinit=/bin/sh"

The result is kernel panic. 
If i set busybox as provider (PREFERRED_PROVIDER_virtual/base-utils and VIRTUAL-RUNTIME_base-utils) then boot is ok.

I will prepare a patch the upcoming week.
Randy and Richard thank you for the support yesterday :).

Br
Christos

Comment 5 Christos Gavros 2025-03-28 18:07:10 UTC

hi

i think the right way to reproduce it is the following (and not in the previous comment i made):

add in local.conf:
PREFERRED_PROVIDER_virtual/base-utils = "packagegroup-core-base-utils"
VIRTUAL-RUNTIME_base-utils ?= "packagegroup-core-base-utils"
INITRAMFS_IMAGE = "core-image-minimal-initramfs"
INITRAMFS_MAXSIZE = "160000"

Run: bitbake core-image-minimal

Run qemu manually including initramfs:
 sudo qemu-system-x86_64 \
  -device virtio-net-pci,netdev=net0,mac=52:54:00:12:34:02 \
  -netdev tap,id=net0,ifname=tap0,script=no,downscript=no \
  -object rng-random,filename=/dev/urandom,id=rng0 \
  -device virtio-rng-pci,rng=rng0 \
  -drive file=/home/cg/poky/build/tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs.ext4,if=virtio,format=raw \
  -usb \
  -device usb-tablet \
  -usb \
  -device usb-kbd \
  -cpu IvyBridge \
  -machine q35 \
  -smp 4 \
  -m 256 \
  -serial mon:vc \
  -serial null \
  -device virtio-vga \
  -display sdl,show-cursor=on \
  -kernel /home/cg/poky/build/tmp/deploy/images/qemux86-64/bzImage \
  -initrd /home/cg/poky/build/tmp/deploy/images/qemux86-64/core-image-minimal-initramfs-qemux86-64.cpio.gz \
  -append 'root=/dev/vda rw ip=192.168.7.2::192.168.7.1:255.255.255.0::eth0:off:8.8.8.8 net.ifnames=0 oprofile.timer=1 tsc=reliable no_timer_check rcupdate.rcu_expedited=1 swiotlb=0'

The result is that boot stops with message:
"Switch_root: invalid option — ‘c’
Try ‘switch_root –help’ for more information"

If i set busybox as provider (PREFERRED_PROVIDER_virtual/base-utils and VIRTUAL-RUNTIME_base-utils) then boot is ok.

Comment 6 Christos Gavros 2025-04-13 16:59:38 UTC

The patch is merged in the master:

https://git.openembedded.org/openembedded-core/commit/?id=0f745024fd40518f98390008b4f613d5641df416