13946 2020-06-15 08:13:55 +0000 Intermittent FileNotFoundError during perl do_package_write_ipk on autobuilder 2020-08-13 14:49:46 +0000 1 1 7 Build System, Metadata & Runtime OE-Core devtools / tool chain 3.1 x86 Multiple RESOLVED FIXED backport 3.1 AB-INT Medium+ normal 3.2 M2 1 steve JPEWhacker meta.mr.watcher meta.watcher randy.macleod richard.purdie ross.burton tim.orling --- --- No (bug/feature does not impact docs) oldest_to_newest 87482 0 steve 2020-06-15 08:13:55 +0000 Typically seen in oe-selftest, two recent examples: https://errors.yoctoproject.org/Errors/Details/416411/ https://errors.yoctoproject.org/Errors/Details/416412/ ERROR: perl-5.30.1-r0 do_package_write_ipk: Fatal errors occurred in subprocesses: [Errno 2] No such file or directory: '/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-48585/tmp/work/core2-64-poky-linux/perl/5.30.1-r0/packages-split/perl-module-x86-64-linux-storable': Traceback (most recent call last): File "/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/lib/oe/utils.py", line 280, in run ret = self._target(*self._args, **self._kwargs) File "/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/classes/package_ipk.bbclass", line 103, in ipk_write_pkg os.chdir(root) FileNotFoundError: [Errno 2] No such file or directory: '/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-48585/tmp/work/core2-64-poky-linux/perl/5.30.1-r0/packages-split/perl-module-x86-64-linux-storable' ERROR: Logfile of failure stored in: /home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-48585/tmp/work/core2-64-poky-linux/perl/5.30.1-r0/temp/log.do_package_write_ipk.4708 NOTE: recipe perl-5.30.1-r0: task do_package_write_ipk: Failed 87493 1 richard.purdie 2020-06-16 13:08:26 +0000 The perl-module-x86-64-linux-* modules should never be created in the first place. I have a patch out for review which fixes that: diff --git a/meta/recipes-devtools/perl/perl_5.30.2.bb b/meta/recipes-devtools/perl/perl_5.30.2.bb index 26138ea9e5..804520d885 100644 --- a/meta/recipes-devtools/perl/perl_5.30.2.bb +++ b/meta/recipes-devtools/perl/perl_5.30.2.bb @@ -306,7 +306,7 @@ python split_perl_packages () { do_split_packages(d, libdir, r'Module/([^\/]*)\.pm', '${PN}-module-%s', 'perl module %s', recursive=True, allow_dirs=False, match_path=True, prepend=False) do_split_packages(d, libdir, r'Module/([^\/]*)/.*', '${PN}-module-%s', 'perl module %s', recursive=True, allow_dirs=False, match_path=True, prepend=False) do_split_packages(d, libdir, r'.*linux/([^\/].*)\.(pm|pl|e2x)', '${PN}-module-%s', 'perl module %s', recursive=True, allow_dirs=False, match_path=True, prepend=False) - do_split_packages(d, libdir, r'(^(?!(CPAN\/|CPANPLUS\/|Module\/|unicore\/)[^\/]).*)\.(pm|pl|e2x)', '${PN}-module-%s', 'perl module %s', recursive=True, allow_dirs=False, match_path=True, prepend=False) + do_split_packages(d, libdir, r'(^(?!(CPAN\/|CPANPLUS\/|Module\/|unicore\/|.*linux\/)[^\/]).*)\.(pm|pl|e2x)', '${PN}-module-%s', 'perl module %s', recursive=True, allow_dirs=False, match_path=True, prepend=False) # perl-modules should recommend every perl module, and only the # modules. Don't attempt to use the result of do_split_packages() as some however, this does not explain why the storable module is present in some builds and not others. The only way that can be explained is if /usr/lib64/perl5/5.30.2/Storable.pm somehow sometimes is installed as: /usr/lib64/perl5/5.30.2/x86_64-linux/Storable.pm but I cannot see how that is happening at a look through the perl build system. It could be some kind of host system "contamination" but I can't see what/where. 87517 2 JPEWhacker 2020-06-18 08:21:53 +0000 It's probably worth adding a check in the recipe to specifically look for cases where this file gets created and fail the build instead of producing bad sstate 87888 3 ross.burton 2020-08-06 08:14:10 +0000 Did a build of perl on my arm64 box for an arm64 target: perl-module-storable: /usr/lib/perl5/5.32.0/aarch64-linux/Storable.pm 87889 4 ross.burton 2020-08-06 08:19:43 +0000 Is the cause that sometimes there is a companion .so and sometimes not? For me: perl-module-storable: /usr/lib/perl5/5.32.0/aarch64-linux/Storable.pm /usr/lib/perl5/5.32.0/aarch64-linux/auto/Storable/Storable.so So the aarch64-linux bit is appropriate if there's also a .so. 87893 5 richard.purdie 2020-08-06 14:58:36 +0000 I think I may have the comment backwards and I meant: """ /usr/lib64/perl5/5.30.2/x86_64-linux/Storable.pm somehow sometimes is installed as: /usr/lib64/perl5/5.30.2/Storable.pm """ since my local build has "/5.30.2/x86_64-linux/Storable.pm" as does everyone elses. The test we should perhaps add is to detect if /usr/lib64/perl5/5.30.2/Storable.pm ever is installed and error if so as that is the build we need to find if it happens. 87909 6 ross.burton 2020-08-11 09:54:20 +0000 Has this happened recently? 87918 7 richard.purdie 2020-08-12 13:56:18 +0000 If it did, it would no longer show up due to my other fix. This is why I'd like to add the sanity test before we close this. 87920 8 JPEWhacker 2020-08-12 17:39:09 +0000 https://lists.openembedded.org/g/openembedded-core/message/141421 should add a check for the invalid state at do_install() 87928 9 JPEWhacker 2020-08-13 08:04:33 +0000 The patch will prevent this issue from occurring silently, so we can close this. If the issue occurs again, a new bug should be opened. 87943 10 steve 2020-08-13 14:49:46 +0000 I added Joshua's patch to my dunfell test builds and got this error twice: on oe-selftest-centos and oe-selftest-fedora. Both oe-selftest-ubuntu and oe-selftest-debian were successful. https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/1245 https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/1229 I'll open a new bug too