Bug 16163

https://autobuilder.yoctoproject.org/valkyrie/#/builders/61/builds/2988/steps/13/logs/stdio
qemuarm64-ptest debian13-vk-arm1

https://autobuilder.yoctoproject.org/valkyrie/#/builders/73/builds/2967/steps/13/logs/stdio
qemux86-64-ptest stream9-vk-1

Tim's guess is that the system is running low on memory.

https://autobuilder.yoctoproject.org/valkyrie/#/builders/73/builds/3087/steps/13/logs/stdio
qemux86-64-ptest fedora42-vk-1

qemux86-64-ptest stream9-vk-1 master-next&master completed at 2026-02-13 17:34:19+00:00
https://valkyrie.yocto.io/pub/non-release/20260213-93/testresults/qemux86-64-ptest/
https://autobuilder.yoctoproject.org/valkyrie/#/builders/73/builds/3091/steps/13/logs/stdio

qemuarm64-ptest debian13-vk-arm1 mathieu/master-next completed at 2026-02-17 17:35:11+00:00
https://valkyrie.yocto.io/pub/non-release/20260217-110/testresults/qemuarm64-ptest/
https://autobuilder.yoctoproject.org/valkyrie/#/builders/61/builds/3061/steps/13/logs/stdio

Created attachment 5188 [details]
cffi_ptest_histogram.png

Created attachment 5189 [details]
cffi_durations_final.json

My current theory is that the default ptest-runner timeout of 450 seconds which is hard-coded in https://git.openembedded.org/openembedded-core/tree/meta/lib/oeqa/runtime/cases/ptest.py#n62:

status, output = self.target.run('ptest-runner -t 450 -d \"{}\"'.format(' '.join(ptest_dirs)), 0)

Attached are a histogram PNG and json file generated with the help of Claude Opus 4.6.

Here is the summary of that analysis:

Here's the histogram of ptestresult.sections.python3-cffi.duration from the last month of commits. The chart is attached as cffi_ptest_histogram.png.

Key observations:

29 data points from Jan 19 - Feb 19 2026
The distribution is bimodal — there are two clusters:
~318-340s (the dominant cluster, 13 runs) — these appear to be the "normal" fast runs
~420-446s (secondary cluster, 9 runs) — these are ~2 minutes slower
Min: 318s, Max: 446s, Mean: 372s, Median: 344s, Std dev: 51s

The ~120s gap between the two modes suggests some environmental or test-configuration difference that causes roughly half the runs to take significantly longer.

qemuarm64-ptest debian13-vk-arm1 mathieu/master-next completed at 2026-02-20 20:58:44+00:00
https://valkyrie.yocto.io/pub/non-release/20260220-76/testresults/qemuarm64-ptest/
https://autobuilder.yoctoproject.org/valkyrie/#/builders/61/builds/3081/steps/13/logs/stdio

qemuarm64-ptest debian13-vk-arm1 mathieu/master-next-tests&mathieu/master-next completed at 2026-02-26 14:11:13+00:00
https://valkyrie.yocto.io/pub/non-release/20260226-91/testresults/qemuarm64-ptest/
https://autobuilder.yoctoproject.org/valkyrie/#/builders/61/builds/3123/steps/13/logs/stdio

Submitted:
https://lists.openembedded.org/g/openembedded-core/message/232111

The test run on the AutoBuilder cluster that proves this works is:
https://autobuilder.yoctoproject.org/valkyrie/#/builders/61/builds/3136
https://valkyrie.yocto.io/pub/non-release/20260227-113/testresults/qemuarm64-ptest/core-image-ptest-python3-cffi/
where you can see:
PATH=/usr/sbin:/sbin:/usr/bin:/bin; ptest-runner -t 600 -d "/usr/lib"
in https://valkyrie.yocto.io/pub/non-release/20260227-113/testresults/qemuarm64-ptest/core-image-ptest-python3-cffi/log.do_testimage.2323256.20260227191837

yocto-docs submission:
https://lists.yoctoproject.org/g/docs/message/9014

Not with python3-cffi but very similar looking ptest-runner timeouts:
whinlatter qemuriscv64-ptest fedora42-vk-1 https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/1146

Log extract:
           'PASS: mini-record-range\n'
           'PASS: mini-server-name\n'
           'PASS: mini-record-failure\n'
           '[    0.000000] Booting Linux on hartid 3\n'
           '[    0.000000] Linux version 6.16.11-yocto-standard '
           '(oe-user@oe-host) (riscv64-poky-linux-gcc (GCC) 15.2.0, GNU ld '

@Tim: Do you have an explanation as to why we do not see "TIMEOUT: ..." lines in the logs?
If ptest-runner kills the test because of a timeout, it should print those just after DURATION:
https://git.yoctoproject.org/ptest-runner2/tree/utils.c#n536
  fprintf(fp, "DURATION: %d\n", (int) duration);
  if (timedout) {
  	fprintf(fp, "TIMEOUT: %s\n", ptest_dir);
  	rc += 1;
  }

But in the logs linked from this bug (and the one I added), I do not see it:
                 'Swap:             0           0           0\n'
                 '\n'
                 'ERROR: Exited from signal Killed (9)\n'
                 'DURATION: 450\n'}
ptests which had no test results:
['python3-cffi']

Merged:
https://git.openembedded.org/openembedded-core/commit/?id=37982f5be43ab1fc646cfea18f3e92b760dfebc4
https://git.openembedded.org/openembedded-core/commit/?id=2110bbc2c3d0ac5c8018e79448c3a30015888b5d
https://git.openembedded.org/openembedded-core/commit/?id=d3e757f21403554e7064c5fa2b353080d14d2ce7

Yoann,

I think we get information way too late from the way we run ptests. Glad to see you addressing the timeout message in the ptest-runner itself:

https://patchwork.yoctoproject.org/project/yocto/patch/20260302174500.357368-2-yoann.congal@smile.fr/

Should we change the text of this bug and keep it open to track other packages that need longer timeouts? Or should we start another umbrella bug to track individual packages...

Stable fix strategy:
Wait for a ptest-runner upgrade to merge on master (I'll send that), then backport it to whinlatter.

The idea is: The critical fix for this bug is in ptest-runner v2.5.0:
main.c: Set PYTHONUNBUFFERED in the environment
https://git.yoctoproject.org/ptest-runner2/commit/?id=4a8661eb7af6f7d97dc769296e96a68327e30bd7

We decided to close this bug, as the patches to fix the immediate problem have merged (See comment 15). New bugs should be filed for any new timeout discoveries on a per package basis.

As Yoann noted in comment 17, a related fix in ptest-runner itself (2.5.0 tag) will help resolve the AutoBuilder failures (or help identify them better).