Bug 14668

Summary: Regression do_package_write_rpm error since 3.1.11
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Chanho Park <parkch98>
Component: coreAssignee: Ross Burton <ross.burton>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium+ CC: akuster808, meta.mr.watcher, meta.watcher, pavel, randy.macleod, ross.burton, steve
Version: 3.1.12   
Target Milestone: 3.1.16   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Don't know

Description Chanho Park 2021-12-20 02:44:59 UTC 
I found a regression when I updated yocto dunfell from 3.1.11 to 3.1.12.
Sorry for attaching the errors here instead of attachment.

I ran "git bisect" of poky between yocto-3.1.11 and yocto-3.1.12. Finally, I found below commit makes the error.
[62c874086a9f3f335f879264f14fb5d10226800e] oeqa/runtime: search sys.path explicitly for modules

I found similar problems in the community.
- https://lore.kernel.org/all/1tcH.1635971217520897935.hiAz@lists.openembedded.org/
- https://stackoverflow.com/questions/69726046/bitbake-build-fails-because-of-a-python-function


<----------------------- Failure Log -------------------->
ERROR: libxml2-2.9.10-r0 do_package_write_rpm: Error executing a python function in exec_func_python() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:do_package_rpm(d)
     0003:
File: '/home/chanho61.park/inhouse/dunfell_release_s/sources/poky/meta/classes/package_rpm.bbclass', lineno: 712, function: do_package_rpm
     0708:
     0709:    # Build the rpm package!
     0710:    d.setVar('BUILDSPEC', cmd + "\n" + cleanupcmd + "\n")
     0711:    d.setVarFlag('BUILDSPEC', 'func', '1')
 *** 0712:    bb.build.exec_func('BUILDSPEC', d)
     0713:
     0714:    if d.getVar('RPM_SIGN_PACKAGES') == '1':
     0715:        bb.build.exec_func("sign_rpm", d)
     0716:}
File: '/home/chanho61.park/inhouse/dunfell_release_s/sources/poky/bitbake/lib/bb/build.py', lineno: 254, function: exec_func
     0250:    with bb.utils.fileslocked(lockfiles):
     0251:        if ispython:
     0252:            exec_func_python(func, d, runfile, cwd=adir)
     0253:        else:
 *** 0254:            exec_func_shell(func, d, runfile, cwd=adir)
     0255:
     0256:    try:
     0257:        curcwd = os.getcwd()
     0258:    except:

File: '/home/chanho61.park/inhouse/dunfell_release_s/sources/poky/bitbake/lib/bb/build.py', lineno: 455, function: exec_func_shell
     0451:    with open(fifopath, 'r+b', buffering=0) as fifo:
     0452:        try:
     0453:            bb.debug(2, "Executing shell function %s" % func)
     0454:            with open(os.devnull, 'r+') as stdin, logfile:
 *** 0455:                bb.process.run(cmd, shell=False, stdin=stdin, log=logfile, extrafiles=[(fifo,readfifo)])
     0456:        finally:
     0457:            os.unlink(fifopath)
     0458:
     0459:    bb.debug(2, "Shell function %s finished" % func)
File: '/home/chanho61.park/inhouse/dunfell_release_s/sources/poky/bitbake/lib/bb/process.py', lineno: 184, function: run
     0180:
     0181:    if pipe.returncode != 0:
     0182:        if log:
     0183:            # Don't duplicate the output in the exception if logging it
 *** 0184:            raise ExecutionError(cmd, pipe.returncode, None, None)
     0185:        raise ExecutionError(cmd, pipe.returncode, stdout, stderr)
     0186:    return stdout, stderr
Exception: bb.process.ExecutionError: Execution of /home/chanho61.park/inhouse/dunfell_release_s/build_sys/tmp/work/aarch64-poky-linux-musl/libxml2/2.9.10-r0/temp/run.BUILDSPEC.112574' failed with exit code 1

ERROR: Logfile of failure stored in: /home/chanho61.park/inhouse/dunfell_release_s/build_sys/tmp/work/aarch64-poky-linux-musl/libxml2/2.9.10-r0/temp/log.do_package_write_rpm.112574

ERROR: Task (/home/chanho61.park/inhouse/dunfell_release_s/sources/poky/meta/recipes-core/libxml/libxml2_2.9.10.bb:do_package_write_rpm) failed with exit code '1'









<---------------------- log.do_package_write_rpm.112574------------------->

Provides: libxml2-python = 2.9.10-r0 libxml2-python(aarch-64) = 2.9.10-r0 libxml2mod.so()(64bit)

Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1

Checking for unpackaged file(s): /home/chanho61.park/inhouse/dunfell_release_s/build_sys/tmp/work/aarch64-poky-linux-musl/libxml2/2.9.10-r0/recipe-sysroot-native/usr/bin/../../usr/lib/rpm/check-files /home/chanho61.park/inhouse/dunfell_release_s/build_sys/tmp/work/aarch64-poky-linux-musl/libxml2/2.9.10-r0/package

error: create archive failed: cpio: write failed - Invalid argument             

Finished binary package job, result 2, filename /home/chanho61.park/inhouse/dunfell_release_s/build_sys/tmp/work/aarch64-poky-linux-musl/libxml2/2.9.10-r0/deploy-rpms/aarch64/libxml2-2.9.10-r0.aarch64.rpm

Finished binary package job, result 0, filename (null)                          

Finished binary package job, result 0, filename (null)                          

Wrote: /home/chanho61.park/inhouse/dunfell_release_s/build_sys/tmp/work/aarch64-poky-linux-musl/libxml2/2.9.10-r0/deploy-rpms/aarch64/libxml2-utils-2.9.10-r0.aarch64.rpm

Finished binary package job, result 0, filename /home/chanho61.park/inhouse/dunfell_release_s/build_sys/tmp/work/aarch64-poky-linux-musl/libxml2/2.9.10-r0/deploy-rpms/aarch64/libxml2-utils-2.9.10-r0.aarch64.rpm

Wrote: /home/chanho61.park/inhouse/dunfell_release_s/build_sys/tmp/work/aarch64-poky-linux-musl/libxml2/2.9.10-r0/deploy-rpms/aarch64/libxml2-dev-2.9.10-r0.aarch64.rpm

Finished binary package job, result 0, filename /home/chanho61.park/inhouse/dunfell_release_s/build_sys/tmp/work/aarch64-poky-linux-musl/libxml2/2.9.10-r0/deploy-rpms/aarch64/libxml2-dev-2.9.10-r0.aarch64.rpm

Wrote: /home/chanho61.park/inhouse/dunfell_release_s/build_sys/tmp/work/aarch64-poky-linux-musl/libxml2/2.9.10-r0/deploy-rpms/aarch64/libxml2-python-2.9.10-r0.aarch64.rpm

Finished binary package job, result 0, filename /home/chanho61.park/inhouse/dunfell_release_s/build_sys/tmp/work/aarch64-poky-linux-musl/libxml2/2.9.10-r0/deploy-rpms/aarch64/libxml2-python-2.9.10-r0.aarch64.rpm

Wrote: /home/chanho61.park/inhouse/dunfell_release_s/build_sys/tmp/work/aarch64-poky-linux-musl/libxml2/2.9.10-r0/deploy-rpms/aarch64/libxml2-doc-2.9.10-r0.aarch64.rpm

Finished binary package job, result 0, filename /home/chanho61.park/inhouse/dunfell_release_s/build_sys/tmp/work/aarch64-poky-linux-musl/libxml2/2.9.10-r0/deploy-rpms/aarch64/libxml2-doc-2.9.10-r0.aarch64.rpm

Wrote: /home/chanho61.park/inhouse/dunfell_release_s/build_sys/tmp/work/aarch64-poky-linux-musl/libxml2/2.9.10-r0/deploy-rpms/aarch64/libxml2-src-2.9.10-r0.aarch64.rpm

Finished binary package job, result 0, filename /home/chanho61.park/inhouse/dunfell_release_s/build_sys/tmp/work/aarch64-poky-linux-musl/libxml2/2.9.10-r0/deploy-rpms/aarch64/libxml2-src-2.9.10-r0.aarch64.rpm

Wrote: /home/chanho61.park/inhouse/dunfell_release_s/build_sys/tmp/work/aarch64-poky-linux-musl/libxml2/2.9.10-r0/deploy-rpms/aarch64/libxml2-dbg-2.9.10-r0.aarch64.rpm

Finished binary package job, result 0, filename /home/chanho61.park/inhouse/dunfell_release_s/build_sys/tmp/work/aarch64-poky-linux-musl/libxml2/2.9.10-r0/deploy-rpms/aarch64/libxml2-dbg-2.9.10-r0.aarch64.rpm

Wrote: /home/chanho61.park/inhouse/dunfell_release_s/build_sys/tmp/work/aarch64-poky-linux-musl/libxml2/2.9.10-r0/deploy-rpms/aarch64/libxml2-ptest-2.9.10-r0.aarch64.rpm

Finished binary package job, result 0, filename /home/chanho61.park/inhouse/dunfell_release_s/build_sys/tmp/work/aarch64-poky-linux-musl/libxml2/2.9.10-r0/deploy-rpms/aarch64/libxml2-ptest-2.9.10-r0.aarch64.rpm

                                                                                

                                                                                

RPM build errors:                                                               

    Deprecated external dependency generator is used!                           

    Deprecated external dependency generator is used!                           

    Deprecated external dependency generator is used!                           

    Deprecated external dependency generator is used!                           

    Deprecated external dependency generator is used!                           

    Deprecated external dependency generator is used!                           

    Deprecated external dependency generator is used!                           

    Deprecated external dependency generator is used!                           

    create archive failed: cpio: write failed - Invalid argument
Comment 1 Ross Burton 2022-02-08 11:23:18 UTC 
There's no way that a change to oeqa will cause that issue.

How are you reproducing the failure? Is it just bitbake libxml2? Does it happen every time, or occasionally?  This recipe, or any recipe?
Comment 2 Chanho Park 2022-02-23 03:29:04 UTC 
(In reply to comment #1)
> There's no way that a change to oeqa will cause that issue.
> 
> How are you reproducing the failure? Is it just bitbake libxml2? Does it
> happen every time, or occasionally?  This recipe, or any recipe?

The same errors can be seen from perl as well. Once I executed "bitbake -c cleansstate perl" and build it again, the errors were not reproduced anymore.
So, I only can reproduce this issue once I wipe out tmp directory and sstate cache directory.

Below error was coming from rpmbuild on do_package_write_rpm stage.

> error: create archive failed: cpio: write failed - Invalid argument
Comment 3 akuster 2022-02-24 16:14:52 UTC 
MV may be seeing this. Let me double check.
Comment 4 akuster 2022-02-24 16:53:56 UTC 
There is a patch in stable/dunfell-nut that may be helpful. MV has passed that along to its customer to test as well.

https://git.yoctoproject.org/poky-contrib/commit/?h=stable/dunfell-nut&id=7b27c85ed93cd6d193971a0d535bfa47454801f9
Comment 5 Chanho Park 2022-03-02 08:39:00 UTC 
(In reply to comment #4)
> There is a patch in stable/dunfell-nut that may be helpful. MV has passed
> that along to its customer to test as well.
> 
> https://git.yoctoproject.org/poky-contrib/commit/?h=stable/dunfell-
> nut&id=7b27c85ed93cd6d193971a0d535bfa47454801f9

Great. The issue cannot be reproduced anymore with this patch. I saw it was included in dunfell branch and it will be release on 3.1.15 release, right?
Comment 6 akuster 2022-03-02 16:03:28 UTC 
(In reply to comment #5)
> (In reply to comment #4)
> > There is a patch in stable/dunfell-nut that may be helpful. MV has passed
> > that along to its customer to test as well.
> > 
> > https://git.yoctoproject.org/poky-contrib/commit/?h=stable/dunfell-
> > nut&id=7b27c85ed93cd6d193971a0d535bfa47454801f9
> 
> Great. The issue cannot be reproduced anymore with this patch. I saw it was
> included in dunfell branch and it will be release on 3.1.15 release, right?

This patch did fix our Customer's issue. Yes I believe it should be if I understand the release processes.
Comment 7 Ross Burton 2022-03-28 16:01:21 UTC 
Fixed in 3.1.15.