Bug 13376 - Need better perl module dependency tooling
Summary: Need better perl module dependency tooling
Status: IN PROGRESS IMPLEMENTATION
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: core (show other bugs)
Version: 5.99
Hardware: x86 Multiple
: Medium enhancement
Target Milestone: 6.99
Assignee: Tim Orling
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2019-06-04 16:03 UTC by Tim Orling
Modified: 2026-04-23 14:59 UTC (History)
4 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Don't know


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Tim Orling 2019-06-04 16:03:26 UTC
The core perl dependencies in perl-rdepends.txt are not sufficient for catching the runtime dependencies (as uncovered in ptests). Perl modules, i.e. in meta-oe and meta-perl, also have gaps in dependencies. Unfortunately, the way these dependencies are determined currently is a painfully slow, iterative process of build/run/test/repeat until all dependencies have been met. Testing includes simply running "perl -m<module::name>" or running ptests (which can be enabled with the ptest-perl.bbclass).

In #13200, RP mentioned the use of /usr/lib/rpm/perl.req from the rpm package, which does help catch some dependencies, but misses for instance dependencies that are conditional.

There are a number of tools on CPAN for dependency determination, including tools that cpan or cpanminus use themselves.

https://metacpan.org/pod/Module::ScanDeps
https://metacpan.org/pod/Carton
https://metacpan.org/release/Test-Prereq
https://metacpan.org/release/Module-Extract-Use
https://metacpan.org/release/Perl-PrereqScanner
https://metacpan.org/pod/release/RJBS/Dist-Zilla-5.048/lib/Dist/Zilla/Plugin/AutoPrereq.pm

Worth noting:
http://deps.cpantesters.org/

An informative stackoverflow discussion which uncovered most of the above:
https://stackoverflow.com/questions/358891/how-do-i-find-the-module-dependencies-of-my-perl-script/359687


It is worth mentioning that meta-cpan uses cpan itself to help determine dependencies, but is not in a state that would be ready to merge into oe-core.
Comment 1 Yi Fan Yu 2021-02-22 14:58:44 UTC
also interested for this to get a fix:

had to manually update perl dependencies for around an hour when updating tcdump. and it was highly frustrating because of modules depending on other modules.
Comment 2 Tim Orling 2021-03-03 23:13:39 UTC
I have a proof-of-concept in the works [1]

This is a “simple” approach relying on perl.req from rpm. I described the workflow concept recently for lmsensors [2]

That issue highlights the need for an additional use case: perl in the shebang for a script in a {sub}package, which the current implementation would not catch. This would require a regex on every file installed.

This needs more work and time has been elusive, hence bumped to 3.4 (optimism).
In particular, the current approach only works for a clean build... package data is not populated in a usable fashion for sstate derived sysroots.

[1] http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=timo/perl-deps_13376
[2] https://lists.yoctoproject.org/g/yocto/message/52570
Comment 3 Tim Orling 2021-03-03 23:23:23 UTC
I have another approach, implemented for recipetool/devtool [1] which relies on querying the metacpan [2] api [3]. This is also still just a proof-of-concept and needs more work.

[1] http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=timo/recipetool-perl-v3
[2] https://metacpan.org/
[3] https://github.com/metacpan/metacpan-api/blob/master/docs/API-docs.md
Comment 4 Tim Orling 2022-10-24 22:41:58 UTC
Bump. Might get around to it again.
Comment 5 Tim Orling 2022-10-31 00:06:31 UTC
Super simple check for dependencies on target (in this case for libio-*-perl recipe):

for path in $(find /usr/lib/perl5/vendor_perl/5.36.0/IO -name *.pm | sed -e 's~/usr/lib/perl5/vendor_perl/5.36.0/~~' -e 's~/~::~g' -e 's~.pm~~'); do perl -M${path} -e'print'; done
Comment 6 Tim Orling 2026-04-23 14:59:22 UTC
Still valid. Moving to 6.99.