Bug 15220 - AB-INT: strace ptest failures: sh: write error: No space left on device
Summary: AB-INT: strace ptest failures: sh: write error: No space left on device
Status: RESOLVED FIXED
Alias: None
Product: Package Testing (ptest)
Classification: QA/Testing
Component: ptest (show other bugs)
Version: unspecified
Hardware: x86 Multiple
: Medium+ normal
Target Milestone: 5.0 M3
Assignee: Alexandre Belloni
QA Contact:
URL:
Whiteboard: AB-INT
Depends on:
Blocks:
 
Reported: 2023-09-28 08:32 UTC by Alexandre Belloni
Modified: 2024-03-19 08:19 UTC (History)
6 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alexandre Belloni 2023-09-28 08:32:40 UTC
Many tests are sometimes failing because the image doesn't have any space left:

FAIL: printpath-umovestr.gen.test
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
make: *** [Makefile:17936: printpath-umovestr.gen.log] Error 1
sh: write error: No space left on device
FAIL: read-write.gen.test
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
make: *** [Makefile:17936: read-write.gen.log] Error 1
sh: write error: No space left on device
FAIL: readahead.gen.test
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
make: *** [Makefile:17936: readahead.gen.log] Error 1
sh: write error: No space left on device
FAIL: readlink.gen.test
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
make: *** [Makefile:17936: readlink.gen.log] Error 1
sh: write error: No space left on device
FAIL: readdir.gen.test
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
make: *** [Makefile:17936: readdir.gen.log] Error 1
sh: write error: No space left on device
FAIL: reboot.gen.test
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
make: *** [Makefile:17936: reboot.gen.log] Error 1
sh: write error: No space left on device
FAIL: readlinkat.gen.test
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
make: *** [Makefile:17936: readlinkat.gen.log] Error 1
sh: write error: No space left on device
FAIL: recv-MSG_TRUNC.gen.test
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
make: *** [Makefile:17936: recv-MSG_TRUNC.gen.log] Error 1
sh: write error: No space left on device
FAIL: recvfrom.gen.test
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
make: *** [Makefile:17936: recvfrom.gen.log] Error 1
sh: write error: No space left on device
FAIL: recvfrom-MSG_TRUNC.gen.test
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
make: *** [Makefile:17936: recvfrom-MSG_TRUNC.gen.log] Error 1
sh: write error: No space left on device
FAIL: recvmmsg-timeout.gen.test
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
make: *** [Makefile:17936: recvmmsg-timeout.gen.log] Error 1
sh: write error: No space left on device
FAIL: recvmsg.gen.test
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
make: *** [Makefile:17936: recvmsg.gen.log] Error 1
../build-aux/test-driver: line 111: can't create regex.gen.log: No space left on device
make: *** [Makefile:17936: regex.gen.log] Error 1
sh: write error: No space left on device
FAIL: remap_file_pages.gen.test
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
make: *** [Makefile:17936: remap_file_pages.gen.log] Error 1
sh: write error: No space left on device
FAIL: remap_file_pages-Xabbrev.gen.test
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
make: *** [Makefile:17936: remap_file_pages-Xabbrev.gen.log] Error 1
sh: write error: No space left on device
FAIL: remap_file_pages-Xraw.gen.test
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
make: *** [Makefile:17936: remap_file_pages-Xraw.gen.log] Error 1
sh: write error: No space left on device
FAIL: remap_file_pages-Xverbose.gen.test
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
make: *** [Makefile:17936: remap_file_pages-Xverbose.gen.log] Error 1
sh: write error: No space left on device
FAIL: rename.gen.test
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
make: *** [Makefile:17936: rename.gen.log] Error 1
sh: write error: No space left on device
FAIL: renameat.gen.test
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
sh: write error: No space left on device
make: *** [Makefile:17936: renameat.gen.log] Error 1
sh: write error: No space left on device
FAIL: renameat2.gen.test
../build-aux/test-driver: line 140: can't create renameat2.gen.trs: No space left on device
../build-aux/test-driver: line 141: can't create renameat2.gen.trs: No space left on device
../build-aux/test-driver: line 142: can't create renameat2.gen.trs: No space left on device
../build-aux/test-driver: line 143: can't create renameat2.gen.trs: No space left on device
make: *** [Makefile:17936: renameat2.gen.log] Error 1


https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/5677/steps/12/logs/stdio
Comment 1 Richard Purdie 2023-09-28 16:12:15 UTC
I suspect this is due to the parallelism that was enabled. The space wasn't increased.

The space is set in the ptest recipe:

IMAGE_ROOTFS_EXTRA_SPACE:virtclass-mcextend-strace = "1024288"
Comment 2 Alexandre Belloni 2023-10-05 20:16:56 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/5662/steps/12/logs/stdio

qemux86-64-ptest fedora38-ty-6
Comment 3 Alexandre Belloni 2023-10-05 21:21:23 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/5692/steps/12/logs/stdio

qemux86-64-ptest fedora38-ty-4
Comment 4 Alexandre Belloni 2023-10-05 21:23:09 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/5682/steps/12/logs/stdio

qemux86-64-ptest fedora38-ty-1
Comment 5 Alexandre Belloni 2023-10-19 14:52:02 UTC
fc424293b22b strace: further clean up of ptest folders


https://git.yoctoproject.org/poky/commit/?id=8fa77bf2f1f782ad42ef8d6131b7947840e881e6
Comment 6 Alexandre Belloni 2023-11-18 10:32:51 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/5910/steps/12/logs/stdio

qemux86-64-ptest alma9-ty-1
Comment 7 Alexandre Belloni 2023-12-11 18:18:55 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/5942/steps/13/logs/stdio

qemux86-64-ptest stream8-ty-1
Comment 8 Alexandre Belloni 2023-12-11 22:51:08 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/5876/steps/12/logs/stdio

qemux86-64-ptest fedora38-ty-6
Comment 9 Alexandre Belloni 2023-12-11 22:54:37 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/5667/steps/12/logs/stdio

qemux86-64-ptest fedora38-ty-1
Comment 10 Alexandre Belloni 2023-12-11 23:04:45 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/5880/steps/12/logs/stdio

qemux86-64-ptest ubuntu2004-ty-1
Comment 11 Alexandre Belloni 2023-12-14 23:23:41 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/5985/steps/13/logs/stdio

qemux86-64-ptest opensuse154-ty-3
Comment 12 Alexandre Belloni 2023-12-18 17:06:00 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/6054/steps/12/logs/stdio

qemux86-64-ptest fedora38-ty-4

this one had many many tests failing, I'm not sure why as the latest patch should have been making enough space for those to run.
Comment 13 João Marcos Costa 2024-01-17 10:22:35 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/6111/steps/13/logs/stdio

qemux86-64-ptest opensuse154-ty-2
Comment 14 João Marcos Costa 2024-01-17 10:51:23 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/6082/steps/12/logs/stdio

qemux86-64-ptest fedora38-ty-6
Comment 15 João Marcos Costa 2024-01-17 10:54:31 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/6077/steps/13/logs/stdio

qemux86-64-ptest opensuse154-ty-2
Comment 16 João Marcos Costa 2024-01-17 12:54:37 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/5956/steps/12/logs/stdio

qemux86-64-ptest alma9-ty-1
Comment 17 João Marcos Costa 2024-01-17 13:07:03 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/5954/steps/12/logs/stdio

qemux86-64-ptest fedora38-ty-4
Comment 18 João Marcos Costa 2024-01-17 13:51:59 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/5836/steps/13/logs/stdio

qemux86-64-ptest opensuse153-ty-1
Comment 19 João Marcos Costa 2024-01-17 14:01:06 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/5811/steps/12/logs/stdio

qemux86-64-ptest fedora38-ty-4
Comment 20 João Marcos Costa 2024-01-17 14:15:54 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/5761/steps/12/logs/stdio

qemux86-64-ptest fedora38-ty-4
Comment 21 João Marcos Costa 2024-01-17 14:59:01 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/5677/steps/12/logs/stdio

qemux86-64-ptest fedora38-ty-6
Comment 22 João Marcos Costa 2024-01-17 15:04:01 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/5657/steps/12/logs/stdio

qemux86-64-ptest fedora38-ty-1
Comment 23 João Marcos Costa 2024-01-17 15:22:09 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/5723/steps/12/logs/stdio

qemux86-64-ptest ubuntu2204-ty-3
Comment 24 João Marcos Costa 2024-01-17 15:25:03 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/5721/steps/12/logs/stdio

qemux86-64-ptest ubuntu2004-ty-1
Comment 25 João Marcos Costa 2024-01-17 15:39:48 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/5739/steps/12/logs/stdio

qemux86-64-ptest fedora38-ty-5
Comment 26 João Marcos Costa 2024-01-17 15:47:10 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/5737/steps/12/logs/stdio

qemux86-64-ptest fedora38-ty-6
Comment 27 João Marcos Costa 2024-01-17 15:51:15 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/5736/steps/13/logs/stdio

qemux86-64-ptest opensuse154-ty-2
Comment 28 João Marcos Costa 2024-01-17 15:53:34 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/5732/steps/12/logs/stdio

qemux86-64-ptest fedora38-ty-2
Comment 29 Alexis Lothoré 2024-02-28 09:24:44 UTC
A series introducing basic guest/host disk space statistics has been merged on master and now allows to get disk statistics on failed runtime tests.
To retrieve the corresponding stats, heads public artifacts server (https://autobuilder.yocto.io/pub/) and navigate to non-release/<job_id>/testresults/<builder>/artifacts
Comment 30 Alexandre Belloni 2024-03-19 08:19:38 UTC
No occurrence after growing the rootfs size.