Bug 13946 - Intermittent FileNotFoundError during perl do_package_write_ipk on autobuilder
Summary: Intermittent FileNotFoundError during perl do_package_write_ipk on autobuilder
Status: RESOLVED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: devtools / tool chain (show other bugs)
Version: 3.1
Hardware: x86 Multiple
: Medium+ normal
Target Milestone: 3.2 M2
Assignee: Joshua Watt
QA Contact:
URL:
Whiteboard: backport 3.1 AB-INT
Depends on:
Blocks:
 
Reported: 2020-06-15 08:13 UTC by Steve Sakoman
Modified: 2020-08-13 14:49 UTC (History)
6 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Steve Sakoman 2020-06-15 08:13:55 UTC
Typically seen in oe-selftest, two recent examples:

https://errors.yoctoproject.org/Errors/Details/416411/
https://errors.yoctoproject.org/Errors/Details/416412/

ERROR: perl-5.30.1-r0 do_package_write_ipk: Fatal errors occurred in subprocesses:
[Errno 2] No such file or directory: '/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-48585/tmp/work/core2-64-poky-linux/perl/5.30.1-r0/packages-split/perl-module-x86-64-linux-storable': Traceback (most recent call last):
  File "/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/lib/oe/utils.py", line 280, in run
    ret = self._target(*self._args, **self._kwargs)
  File "/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/classes/package_ipk.bbclass", line 103, in ipk_write_pkg
    os.chdir(root)
FileNotFoundError: [Errno 2] No such file or directory: '/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-48585/tmp/work/core2-64-poky-linux/perl/5.30.1-r0/packages-split/perl-module-x86-64-linux-storable'


ERROR: Logfile of failure stored in: /home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-48585/tmp/work/core2-64-poky-linux/perl/5.30.1-r0/temp/log.do_package_write_ipk.4708
NOTE: recipe perl-5.30.1-r0: task do_package_write_ipk: Failed
Comment 1 Richard Purdie 2020-06-16 13:08:26 UTC
The perl-module-x86-64-linux-* modules should never be created in the first place. I have a patch out for review which fixes that:

diff --git a/meta/recipes-devtools/perl/perl_5.30.2.bb b/meta/recipes-devtools/perl/perl_5.30.2.bb
index 26138ea9e5..804520d885 100644
--- a/meta/recipes-devtools/perl/perl_5.30.2.bb
+++ b/meta/recipes-devtools/perl/perl_5.30.2.bb
@@ -306,7 +306,7 @@ python split_perl_packages () {
     do_split_packages(d, libdir, r'Module/([^\/]*)\.pm', '${PN}-module-%s', 'perl module %s', recursive=True, allow_dirs=False, match_path=True, prepend=False)
     do_split_packages(d, libdir, r'Module/([^\/]*)/.*', '${PN}-module-%s', 'perl module %s', recursive=True, allow_dirs=False, match_path=True, prepend=False)
     do_split_packages(d, libdir, r'.*linux/([^\/].*)\.(pm|pl|e2x)', '${PN}-module-%s', 'perl module %s', recursive=True, allow_dirs=False, match_path=True, prepend=False)
-    do_split_packages(d, libdir, r'(^(?!(CPAN\/|CPANPLUS\/|Module\/|unicore\/)[^\/]).*)\.(pm|pl|e2x)', '${PN}-module-%s', 'perl module %s', recursive=True, allow_dirs=False, match_path=True, prepend=False)
+    do_split_packages(d, libdir, r'(^(?!(CPAN\/|CPANPLUS\/|Module\/|unicore\/|.*linux\/)[^\/]).*)\.(pm|pl|e2x)', '${PN}-module-%s', 'perl module %s', recursive=True, allow_dirs=False, match_path=True, prepend=False)
 
     # perl-modules should recommend every perl module, and only the
     # modules. Don't attempt to use the result of do_split_packages() as some


however, this does not explain why the storable module is present in some builds and not others. The only way that can be explained is if

/usr/lib64/perl5/5.30.2/Storable.pm

somehow sometimes is installed as:

/usr/lib64/perl5/5.30.2/x86_64-linux/Storable.pm

but I cannot see how that is happening at a look through the perl build system. It could be some kind of host system "contamination" but I can't see what/where.
Comment 2 Joshua Watt 2020-06-18 08:21:53 UTC
It's probably worth adding a check in the recipe to specifically look for cases where this file gets created and fail the build instead of producing bad sstate
Comment 3 Ross Burton 2020-08-06 08:14:10 UTC
Did a build of perl on my arm64 box for an arm64 target:

perl-module-storable:
        /usr/lib/perl5/5.32.0/aarch64-linux/Storable.pm
Comment 4 Ross Burton 2020-08-06 08:19:43 UTC
Is the cause that sometimes there is a companion .so and sometimes not?

For me:

perl-module-storable:
        /usr/lib/perl5/5.32.0/aarch64-linux/Storable.pm
        /usr/lib/perl5/5.32.0/aarch64-linux/auto/Storable/Storable.so

So the aarch64-linux bit is appropriate if there's also a .so.
Comment 5 Richard Purdie 2020-08-06 14:58:36 UTC
I think I may have the comment backwards and I meant:

"""
/usr/lib64/perl5/5.30.2/x86_64-linux/Storable.pm

somehow sometimes is installed as:

/usr/lib64/perl5/5.30.2/Storable.pm
"""

since my local build has "/5.30.2/x86_64-linux/Storable.pm" as does everyone elses.

The test we should perhaps add is to detect if /usr/lib64/perl5/5.30.2/Storable.pm ever is installed and error if so as that is the build we need to find if it happens.
Comment 6 Ross Burton 2020-08-11 09:54:20 UTC
Has this happened recently?
Comment 7 Richard Purdie 2020-08-12 13:56:18 UTC
If it did, it would no longer show up due to my other fix. This is why I'd like to add the sanity test before we close this.
Comment 8 Joshua Watt 2020-08-12 17:39:09 UTC
https://lists.openembedded.org/g/openembedded-core/message/141421 should add a check for the invalid state at do_install()
Comment 9 Joshua Watt 2020-08-13 08:04:33 UTC
The patch will prevent this issue from occurring silently, so we can close this. If the issue occurs again, a new bug should be opened.
Comment 10 Steve Sakoman 2020-08-13 14:49:46 UTC
I added Joshua's patch to my dunfell test builds and got this error twice: on oe-selftest-centos and oe-selftest-fedora.  Both oe-selftest-ubuntu and oe-selftest-debian were successful.

https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/1245

https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/1229

I'll open a new bug too