Bug 16386 - Investigate reviving ptest coverage for systemd
Summary: Investigate reviving ptest coverage for systemd
Status: IN PROGRESS REVIEW
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: core (show other bugs)
Version: 6.2
Hardware: x86 Multiple
: Medium enhancement
Target Milestone: 6.1
Assignee: Himani Barde
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2026-08-06 05:56 UTC by Himani Barde
Modified: 2026-09-04 08:09 UTC (History)
4 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Himani Barde 2026-08-06 05:56:19 UTC
systemd ptest support was dropped in 2018 when the recipe switched to meson:
https://git.openembedded.org/openembedded-core/commit/?id=906230a73b3ccfa4afd2a19a6b0aa18cd1d5fa08

The commit message noted: "This new version has dropped ptest support, as
there's no easy way to do this in the framework of meson."

Since then, meson has matured significantly. Additionally, upstream systemd
now includes tests relevant to user unit handling:

  systemd.git test/ directory:
  - test/units/TEST-26-SYSTEMCTL.sh (user unit tests with --global)
  - test/units/TEST-55-OOMD.sh (user unit coverage)
  - test/units/TEST-91-LIVEUPDATE.sh (user unit session management)

This enhancement is to investigate:
1. Whether meson now supports running systemd tests in a ptest-compatible way
2. Feasibility of re-enabling ptest for the systemd recipe
3. Including user unit test coverage

Related: Bug 7800 (systemd.bbclass user unit support - now fixed on master)
https://bugzilla.yoctoproject.org/show_bug.cgi?id=7800
Comment 1 Ross Burton 2026-08-06 14:48:35 UTC
Definitely several parts to this:

1) can we get meson to build but not run the test suites
2) how can we copy the test suite into the packages
3) how to run a meson test suite

It would be good to discuss this with the upstream meson maintainers. Last time I tried they were not interested in installable test suites.

Something that might be useful: meson has an 'introspect' command that can dump machine-readable JSON for various bits of the build setup. If that can tell you what the tests are called then it might be possible to automate some of the building/installation/running.
Comment 2 Himani Barde 2026-09-04 08:07:51 UTC
Investigation results: ptest revival is feasible (PoC working)

Tested against systemd 257.6 (meson). Answering the three points from comment 1:

1. Can meson build tests without running them? 
Yes. ninja builds the test binaries at compile time; meson test runs them separately. Meson also writes machine-readable test metadata to <builddir>/meson-info/intro-tests.json (the meson introspect --tests data you suggested) — it enumerates all 1651 tests with name/suite/cmd. So the build-vs-run split that was hard in 2018 is now clean.

2. How to copy the test suite into packages? 
This is now solved upstream. systemd's meson has an install-tests option (meson_options.txt, default false). Enabling it installs the unit-test binaries into ${prefix}/lib/systemd/tests/unit-tests/ plus a run-unit-tests.py runner into ${prefix}/lib/systemd/tests/. 
No manual copying of binaries needed — this is the key thing that changed since the 2018 removal. (So the "installable test suites" concern with upstream meson is moot for systemd specifically; it already ships install_data(...) behind if install_tests.)

3. How to run the suite? 
systemd's installed run-unit-tests.py already globs and runs unit-tests/test-*, printing PASS:/SKIP: per test (rc 77 = skip). That maps almost directly to ptest's expected PASS:/FAIL:/SKIP: output, so a thin run-ptest wrapper is enough.

PoC: a small systemd_%.bbappend — inherit ptest, EXTRA_OEMESON:append = " -Dinstall-tests=true", a do_install_ptest that copies ${prefix}/lib/systemd/tests/* into ${PTEST_PATH}, and a run-ptest wrapping run-unit-tests.py. 
Building systemd -c install produced 313 installed test-* unit-test binaries plus the runner. This covers the systemd:test/libsystemd C unit suites (~230 tests) and the systemd:dist help/version checks (111).

On the user-unit tests (TEST-26-SYSTEMCTL, TEST-55-OOMD, TEST-91-LIVEUPDATE): these are integration tests under test/units/ that expect a booted system via systemd's integration/mkosi harness, not standalone unit-test binaries. They're a separate, harder effort from the meson unit-test suite and probably shouldn't block a first ptest landing. Suggest tracking integration-test-in-ptest separately. (Note: TEST-91-LIVEUPDATE isn't present in 257.6; it's newer.)

Recommendation: re-enable ptest for systemd targeting the meson unit-test suites via install-tests. 
Happy to send a patch.
Comment 3 Himani Barde 2026-09-04 08:09:30 UTC
Patch submitted to openembedded-core: "systemd: re-enable ptest support" (2026-09-04).

It implements the approach described above — enables the meson install-tests option, packages the installed unit-test tree into systemd-ptest, and adds a run-ptest wrapper driving run-unit-tests.py. Verified to parse against oe-core master (systemd 261.2); PoC build done on 257.6 (313 test binaries + runner install cleanly).

As noted, the test/units/*.sh integration tests (TEST-26/55/91) are out of scope for this change and would be a separate integration-test-in-ptest effort.

Message-ID: 20260904080215.2501426-1-HimaniRamesh.Barde@windriver.com