| Summary: | AB-INT PTEST RISCV64: glib-2.0 ptest: in glib/gi-compile-repository.py.test | ||
|---|---|---|---|
| Product: | [QA/Testing] Package Testing (ptest) | Reporter: | João Marcos Costa <joaomarcos.costa> |
| Component: | ptest | Assignee: | Trevor Gamblin <tgamblin> |
| Status: | RESOLVED FIXED | QA Contact: | |
| Severity: | normal | ||
| Priority: | Medium | CC: | luca.ceresoli, randy.macleod, ross.burton |
| Version: | unspecified | ||
| Target Milestone: | 6.0 | ||
| Hardware: | RISCV | ||
| OS: | Multiple | ||
| Whiteboard: | AB-INT | ||
| OS type for building Yocto: | --- | Type of Regression: | --- |
| Verified: | Documentation change: | Don't know | |
|
Description
João Marcos Costa
2025-06-03 09:14:46 UTC
https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/65/steps/12/logs/stdio qemuriscv64-ptest fedora40-vk-2 https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/69/steps/12/logs/stdio qemuriscv64-ptest rocky9-vk-1 https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/87/steps/12/logs/stdio qemuriscv64-ptest rocky9-vk-1 https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/105/steps/12/logs/stdio qemuriscv64-ptest alma9-vk-2 https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/129/steps/12/logs/stdio Failed ptests:{'glib-2.0': ['glib/codegen.py.test', 'glib/gi-compile-repository.py.test']} qemuriscv64-ptest rocky9-vk-2 https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/225/steps/12/logs/stdio qemuriscv64-ptest stream9-vk-1 https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/235/steps/12/logs/stdio qemuriscv64-ptest ubuntu2204-vk-3 https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/242/steps/12/logs/stdio qemuriscv64-ptest stream9-vk-1 https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/266/steps/12/logs/stdio qemuriscv64-ptest debian12-vk-8 https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/267/steps/12/logs/stdio Failed ptests:{'glib-2.0': ['glib/codegen.py.test', 'glib/gi-compile-repository.py.test']} qemuriscv64-ptest stream9-vk-1 https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/257/steps/13/logs/stdio qemuriscv64-ptest rocky8-vk-1 https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/275/steps/12/logs/stdio qemuriscv64-ptest ubuntu2404-vk-2 https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/387/steps/12/logs/stdio qemuriscv64-ptest ubuntu2204-vk-4 The actual failure log is:
Executing: glib/gi-compile-repository.py.test
ok 3 __main__.TestGICompileRepositoryForGLib.test_write_failure
# gi-compile-repository: /usr/bin/gi-compile-repository
# tmpdir: /tmp/tmpihahk969
# Running: ['/usr/bin/gi-compile-repository', '/usr/share/gir-1.0/GLib-2.0.gir', '--output', 'this-is/not/a-good-output/invalid.typelib']
# Return code: 1
# Output:
#
# Error:
# Failed to open ‘this-is/not/a-good-output/invalid.typelib.tmp’: No such file or directory
ok 4 __main__.TestGICompileRepositoryForGObject.test_compile
# gir path set to [PosixPath('/usr/share/gir-1.0'), PosixPath('/usr/share/gir-1.0')]
# gi-compile-repository: /usr/bin/gi-compile-repository
# tmpdir: /tmp/tmpokedaq57
# Running: ['/usr/bin/gi-compile-repository', '/usr/share/gir-1.0/GObject-2.0.gir', '--output', '/tmp/tmpokedaq57/GObject-2.typelib', '--includedir',
'/usr/share/gir-1.0', '--includedir', '/usr/share/gir-1.0']
# Return code: 0
# Output:
#
# Error:
Executing: glib/gi-compile-repository.py.test
ok 5 __main__.TestGICompileRepositoryForGObject.test_write_failure
# gi-compile-repository: /usr/bin/gi-compile-repository
# tmpdir: /tmp/tmpffrqausj
# Running: ['/usr/bin/gi-compile-repository', '/usr/share/gir-1.0/GObject-2.0.gir', '--output', 'this-is/not/a-good-output/invalid.typelib', '--includ
edir', '/usr/share/gir-1.0', '--includedir', '/usr/share/gir-1.0']
# Return code: 1
# Output:
#
# Error:
# Failed to open ‘this-is/not/a-good-output/invalid.typelib.tmp’: No such file or directory
Executing: glib/gi-compile-repository.py.test
Executing: glib/gi-compile-repository.py.test
not ok 6 __main__.TestGICompileRepositoryForGio.test_compile
# gir path set to [PosixPath('/usr/share/gir-1.0'), PosixPath('/usr/share/gir-1.0')]
# gi-compile-repository: /usr/bin/gi-compile-repository
# tmpdir: /tmp/tmp30mr81o4
# Running: ['/usr/bin/gi-compile-repository', '/usr/share/gir-1.0/Gio-2.0.gir', '--output', '/tmp/tmp30mr81o4/Gio-2.typelib', '--includedir', '/usr/sh
are/gir-1.0', '--includedir', '/usr/share/gir-1.0']
---
message: |
Traceback (most recent call last):
File "/usr/libexec/installed-tests/glib/gi-compile-repository.py", line 98, in test_compile
result = self.runTestProgram(argv)
File "/usr/libexec/installed-tests/glib/gi-compile-repository.py", line 81, in runTestProgram
return super().runTestProgram(argv, **kwargs)
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
File "/usr/libexec/installed-tests/glib/testprogramrunner.py", line 127, in runTestProgram
info = subprocess.run(
argv,
...<7 lines>...
check=False,
)
File "/usr/lib/python3.13/subprocess.py", line 556, in run
stdout, stderr = process.communicate(input, timeout=timeout)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.13/subprocess.py", line 1222, in communicate
stdout, stderr = self._communicate(input, endtime, timeout)
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.13/subprocess.py", line 2129, in _communicate
self._check_timeout(endtime, orig_timeout, stdout, stderr)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.13/subprocess.py", line 1269, in _check_timeout
raise TimeoutExpired(
...<2 lines>...
stderr=b''.join(stderr_seq) if stderr_seq else None)
subprocess.TimeoutExpired: Command '['/usr/bin/gi-compile-repository', '/usr/share/gir-1.0/Gio-2.0.gir', '--output', '/tmp/tmp30mr81o4/Gio-2.typ
elib', '--includedir', '/usr/share/gir-1.0', '--includedir', '/usr/share/gir-1.0']' timed out after 10 seconds
...
Executing: glib/gi-compile-repository.py.test
Executing: glib/gi-compile-repository.py.test
ok 7 __main__.TestGICompileRepositoryForGio.test_write_failure
# gi-compile-repository: /usr/bin/gi-compile-repository
# tmpdir: /tmp/tmp273bhhzh
# Running: ['/usr/bin/gi-compile-repository', '/usr/share/gir-1.0/Gio-2.0.gir', '--output', 'this-is/not/a-good-output/invalid.typelib', '--includedir
', '/usr/share/gir-1.0', '--includedir', '/usr/share/gir-1.0']
# Return code: 1
# Output:
#
# Error:
# Failed to open ‘this-is/not/a-good-output/invalid.typelib.tmp’: No such file or directory
1..7
FAIL: glib/gi-compile-repository.py.test (Child process exited with code 1)
It looks like the actual fail is the timeout: subprocess.TimeoutExpired: Command '['/usr/bin/gi-compile-repository', '/usr/share/gir-1.0/Gio-2.0.gir', '--output', '/tmp/tmp30mr81o4/Gio-2.typ elib', '--includedir', '/usr/share/gir-1.0', '--includedir', '/usr/share/gir-1.0']' timed out after 10 seconds I've only been able to reproduce the glib/codegen.py.test failure thus far, but at least on my end, I see errors like this:
# Error:
# ERROR: Bad signature "{vs}". "v" is not a valid type for dictionary keys at position 1.
# /tmp/tmpn8lp0iav/tmpq465on9h.xml:
# <node>
# <interface name="BadTypes">
# <property type="(ss(s{{sv}s}))" name="BadPropertyType" access="read" />
# </interface>
# </node>
# Running: ['/usr/bin/gdbus-codegen', '/tmp/tmpn8lp0iav/tmpq465on9h.xml', '--output', '-', '--body']
# Return code: 1
# Output:
#
# Error:
# ERROR: Bad signature "(ss(s{{sv}s}))". "{" is not a valid type for dictionary keys at position 6.
# /tmp/tmpn8lp0iav/tmpflf56tap.xml:
# <node>
# <interface name="BadTypes">
# <property type="{s" name="BadPropertyType" access="read" />
# </interface>
# </node>
# Running: ['/usr/bin/gdbus-codegen', '/tmp/tmpn8lp0iav/tmpflf56tap.xml', '--output', '-', '--body']
# Return code: 1
I'll add some logs as attachments.
Or not - the logs are way too big, even if I strip them down to just the codegen test. But I do see the same sorts of lines suggesting it's just a timeout, e.g.:
|FAIL: glib/codegen.py.test (Child process killed by signal 9)
and
|# {Executing: glib/codegen.py.test
|Test timed out after 300 seconds
For the record, I was able to produce these errors by building core-image-ptest-glib-2.0 with MACHINE="qemuriscv64" set in local.conf, then doing the following as suggested by rburton: taskset --cpu-list 0 runqemu nographic snapshot *** Bug 15154 has been marked as a duplicate of this bug. *** FYI, from the duplicate issue, a previous fix was merged: https://git.openembedded.org/openembedded-core/commit/?id=8de47e5f3837a9c87c3cbf8dc45f9e90110eda1e https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/456/steps/12/logs/stdio qemuriscv64-ptest rocky9-vk-1 No luck reproducing this outside the Yocto qemuriscv64 build environment yet, but I'm also not certain that my attempts have been right yet - the default test output for glib-2.0 seems a bit different than what we're witnessing from manual ptest/testimage, which is a problem if we're assuming maybe there's an issue related to the volume of output. I'm currently running the tests in an x86-64 Fedora 42 container image with the following packages installed: git pkg-config gcc meson gnome-desktop-testing and the https://gitlab.gnome.org/GNOME/glib.git repository checked out. I have run: meson setup _build meson compile -C _build meson install -C _build There are four test failures, none of which match the ones we're seeing, and they all fail with SIGABRT rather than SIGKILL. This probably isn't the right approach but it may be useful as a hint at what not to do. Looks like there's an intermittent issue with one of the same tests reported upstream (for Windows): https://gitlab.gnome.org/GNOME/glib/-/issues/3733 Seems like they think the culprit could be related to buffering... https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/486/steps/12/logs/stdio Failed ptests:{'glib-2.0': ['glib/codegen.py.test']} qemuriscv64-ptest rocky9-vk-1 Started looking into possible differences in the actual QEMU configuration to see if there was anything that could influence this error. What I found: meta/conf/machine/include/riscv/qemuriscv.inc: QB_SERIAL_OPT = "-device virtio-serial-device -chardev null,id=virtcon -device virtconsole,chardev=virtcon" QB_TCPSERIAL_OPT = " -device virtio-serial-device -chardev socket,id=virtcon,port=@PORT@,host=127.0.0.1,nodelay=on -device virtconsole,chardev=virtcon" meta/conf/machine/qemuarm64.conf: QB_SERIAL_OPT = "-device virtio-serial-pci -chardev null,id=virtcon -device virtconsole,chardev=virtcon" QB_TCPSERIAL_OPT = "-device virtio-serial-pci -chardev socket,id=virtcon,port=@PORT@,host=127.0.0.1,nodelay=on -device virtconsole,chardev=virtcon" I don't know enough about the internals of these to say if there's a buffering difference, but I did test changing the qemuriscv64 setting to match the qemuarm64 one. I am still able to make the codegen test fail if I'm running with taskset to force the emulator to only use one CPU, but so far I can't make the test fail without that limitation. It wasn't readily reproducible in this way before, either, so I'm still skeptical this is the issue. Just to be sure, I've also tested manual bumps of the memory supplied to QEMU up to -m 8192, and I still see the failure, e.g.: taskset --cpu-list 0 runqemu nographic snapshot qemuparams="-m 8192 -smp 4" Some more data from runqemu invocations for comparison. qemuarm64: runqemu - INFO - Using preconfigured tap device tap0 runqemu - INFO - If this is not intended, touch /tmp/qemu-tap-locks/tap0.skip to make runqemu skip tap0. runqemu - INFO - Network configuration: ip=192.168.7.2::192.168.7.1:255.255.255.0::eth0:off:8.8.8.8 net.ifnames=0 runqemu - INFO - Running /home/tgamblin/workspace/yocto/poky/build/tmp/work/x86_64-linux/qemu-helper-native/1.0/recipe-sysroot-native/usr/bin/qemu-system-aarch64 -device virtio-net-pci,netdev=net0,mac=52:54:00:12:34:02 -netdev tap,i d=net0,ifname=tap0,script=no,downscript=no -object rng-random,filename=/dev/urandom,id=rng0 -device virtio-rng-pci,rng=rng0 -drive id=disk0,file=/home/tgamblin/workspace/yocto/poky/build/tmp/deploy/images/qemuarm64/core-image-ptest- glib-2.0-qemuarm64.rootfs-20250918002422.ext4,if=none,format=raw -device virtio-blk-pci,drive=disk0 -device qemu-xhci -device usb-tablet -device usb-kbd -machine virt -cpu cortex-a57 -smp 4 -m 1024 -snapshot -serial mon:stdio -seri al null -nographic -device virtio-gpu-pci -kernel /home/tgamblin/workspace/yocto/poky/build/tmp/deploy/images/qemuarm64/Image -append 'root=/dev/vda rw mem=1024M ip=192.168.7.2::192.168.7.1:255.255.255.0::eth0:off:8.8.8.8 net.ifnam es=0 console=ttyAMA0 console=hvc0 swiotlb=0 ' versus qemuriscv64: runqemu - INFO - Using preconfigured tap device tap0 runqemu - INFO - If this is not intended, touch /tmp/qemu-tap-locks/tap0.skip to make runqemu skip tap0. runqemu - INFO - Network configuration: ip=192.168.7.2::192.168.7.1:255.255.255.0::eth0:off:8.8.8.8 net.ifnames=0 runqemu - INFO - Running /home/tgamblin/workspace/yocto/poky/build/tmp/work/x86_64-linux/qemu-helper-native/1.0/recipe-sysroot-native/usr/bin/qemu-system-riscv64 -device virtio-net-device,netdev=net0,mac=52:54:00:12:34:02 -netdev ta p,id=net0,ifname=tap0,script=no,downscript=no -object rng-random,filename=/dev/urandom,id=rng0 -device virtio-rng-pci,rng=rng0 -drive id=disk0,file=/home/tgamblin/workspace/yocto/poky/build/tmp/deploy/images/qemuriscv64/core-image-p test-glib-2.0-qemuriscv64.rootfs-20250917180545.ext4,if=none,format=raw -device virtio-blk-device,drive=disk0 -device qemu-xhci -device usb-tablet -device usb-kbd -machine virt -cpu rva22s64 -smp 4 -m 1024 -snapshot -serial mon:std io -serial null -nographic -device bochs-display -bios /home/tgamblin/workspace/yocto/poky/build/tmp/deploy/images/qemuriscv64/fw_jump.elf -kernel /home/tgamblin/workspace/yocto/poky/build/tmp/deploy/images/qemuriscv64/Image -append 'root=/dev/vda rw mem=1024M ip=192.168.7.2::192.168.7.1:255.255.255.0::eth0:off:8.8.8.8 net.ifnames=0 console=ttyS0 console=hvc0 earlycon=sbi swiotlb=0 ' Notable differences: - the obvious '-cpu' change (cortex-a57 vs rva22s64) - qemuarm64 has no '-bios' option provided - console=ttyAMA0 for qemuarm64, console=ttyS0 for qemuriscv64 - '-device virtio-gpu-pci' for qemuarm64, '-device bochs-display' for qemuriscv64 (after '-nographic') - '-device virtio-net-pci...' for qemuarm64, '-device virtio-net-device' for qemuriscv64 It was actually quite hard to find useful information about what bochs-display is, even though it seems unlikely to be the problem. A test with QB_GRAPHICS = "-device bochs-display" removed from meta/conf/machine/include/riscv/qemuriscv.inc confirms that (it still failed). Diffing the two image manifests, qemuriscv64's image has three packages that qemuarm64 doesn't: kbd 2.8.0 kernel-image-uimage-6.16.4-yocto-standard 6.16.4+git0+6cd9824a84_01bcf423b0 keymaps 1.0 The test timeout for glib-2.0 tests was doubled in: https://git.openembedded.org/openembedded-core/commit/?id=f634098ed6c5674d81028a7ea8e18a7a93a77fab Neither of the glib-2.0 intermittent test failures has been seen for at least a month (this one for three months), so let's close this. We can reopen it if it starts appearing again. |