Bug 13376

Summary:	Need better perl module dependency tooling
Product:	[Build System, Metadata & Runtime] OE-Core	Reporter:	Tim Orling <tim.orling>
Component:	core	Assignee:	Tim Orling <tim.orling>
Status:	IN PROGRESS IMPLEMENTATION ---	QA Contact:
Severity:	enhancement
Priority:	Medium	CC:	meta.mr.watcher, meta.watcher, randy.macleod, yf3yu
Version:	5.99
Target Milestone:	6.99
Hardware:	x86
OS:	Multiple
Whiteboard:
OS type for building Yocto:	---	Type of Regression:	---
Verified:		Documentation change:	Don't know

Description Tim Orling 2019-06-04 16:03:26 UTC

The core perl dependencies in perl-rdepends.txt are not sufficient for catching the runtime dependencies (as uncovered in ptests). Perl modules, i.e. in meta-oe and meta-perl, also have gaps in dependencies. Unfortunately, the way these dependencies are determined currently is a painfully slow, iterative process of build/run/test/repeat until all dependencies have been met. Testing includes simply running "perl -m<module::name>" or running ptests (which can be enabled with the ptest-perl.bbclass).

In #13200, RP mentioned the use of /usr/lib/rpm/perl.req from the rpm package, which does help catch some dependencies, but misses for instance dependencies that are conditional.

There are a number of tools on CPAN for dependency determination, including tools that cpan or cpanminus use themselves.

https://metacpan.org/pod/Module::ScanDeps
https://metacpan.org/pod/Carton
https://metacpan.org/release/Test-Prereq
https://metacpan.org/release/Module-Extract-Use
https://metacpan.org/release/Perl-PrereqScanner
https://metacpan.org/pod/release/RJBS/Dist-Zilla-5.048/lib/Dist/Zilla/Plugin/AutoPrereq.pm

Worth noting:
http://deps.cpantesters.org/

An informative stackoverflow discussion which uncovered most of the above:
https://stackoverflow.com/questions/358891/how-do-i-find-the-module-dependencies-of-my-perl-script/359687


It is worth mentioning that meta-cpan uses cpan itself to help determine dependencies, but is not in a state that would be ready to merge into oe-core.

Comment 1 Yi Fan Yu 2021-02-22 14:58:44 UTC

also interested for this to get a fix:

had to manually update perl dependencies for around an hour when updating tcdump. and it was highly frustrating because of modules depending on other modules.

Comment 2 Tim Orling 2021-03-03 23:13:39 UTC

I have a proof-of-concept in the works [1]

This is a “simple” approach relying on perl.req from rpm. I described the workflow concept recently for lmsensors [2]

That issue highlights the need for an additional use case: perl in the shebang for a script in a {sub}package, which the current implementation would not catch. This would require a regex on every file installed.

This needs more work and time has been elusive, hence bumped to 3.4 (optimism).
In particular, the current approach only works for a clean build... package data is not populated in a usable fashion for sstate derived sysroots.

[1] http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=timo/perl-deps_13376
[2] https://lists.yoctoproject.org/g/yocto/message/52570

Comment 3 Tim Orling 2021-03-03 23:23:23 UTC

I have another approach, implemented for recipetool/devtool [1] which relies on querying the metacpan [2] api [3]. This is also still just a proof-of-concept and needs more work.

[1] http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=timo/recipetool-perl-v3
[2] https://metacpan.org/
[3] https://github.com/metacpan/metacpan-api/blob/master/docs/API-docs.md

Comment 4 Tim Orling 2022-10-24 22:41:58 UTC

Bump. Might get around to it again.

Comment 5 Tim Orling 2022-10-31 00:06:31 UTC

Super simple check for dependencies on target (in this case for libio-*-perl recipe):

for path in $(find /usr/lib/perl5/vendor_perl/5.36.0/IO -name *.pm | sed -e 's~/usr/lib/perl5/vendor_perl/5.36.0/~~' -e 's~/~::~g' -e 's~.pm~~'); do perl -M${path} -e'print'; done

Comment 6 Tim Orling 2026-04-23 14:59:22 UTC

Still valid. Moving to 6.99.