Bug 14955 - eSDK/devtool/create-spdx: if you enable create-spdx it breaks the SDK
Summary: eSDK/devtool/create-spdx: if you enable create-spdx it breaks the SDK
Status: RESOLVED FIXED
Alias: None
Product: eSDK
Classification: Yocto Project Subprojects
Component: eSDK (show other bugs)
Version: 4.1
Hardware: All Multiple
: Medium+ normal
Target Milestone: 4.3 M1
Assignee: David Reyna
QA Contact: Francisco Pedraza
URL:
Whiteboard:
Depends on: 15089
Blocks:
  Show dependency tree
 
Reported: 2022-11-01 19:32 UTC by Robert Berger
Modified: 2023-06-08 20:00 UTC (History)
3 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Don't know


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Robert Berger 2022-11-01 19:32:22 UTC
It looks like this makes it into the eSDK local.conf:

INHERIT += "create-spdx"

When I create via devtool a recipe, add it to the workspace and build an image thw following happens:

The eudev:do_create_spdx sig is computed to be d2c17c4ba999740eb989112eeafcb8b01a5bba3e9b955bd1a5b3dc54ec7ffe0a, but the sig is locked to 56e2f86f60a335c1d6d06463fcdd5196f6f1883da678fd331c9e8f36d2794acb in SIGGEN_LOCKEDSIGS_t-armv7at2hf-neon
Removing 2 stale sstate objects for arch multi_v7_ml: 100% |###################################################################################################################| Time: 0:00:00
NOTE: Executing Tasks
ERROR: core-image-minimal-1.0-r0 do_rootfs: Error executing a python function in exec_func_python() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:image_combine_spdx(d)
     0003:
File: '/opt/ext-sdk/resy/4.1.66/multi-v7-ml/imx6q-phytec-mira-rdk-nand/layers/poky-master/meta/classes/create-spdx.bbclass', lineno: 843, function: image_combine_spdx
     0839:    imgdeploydir = Path(d.getVar("IMGDEPLOYDIR"))
     0840:    img_spdxid = oe.sbom.get_image_spdxid(image_name)
     0841:    packages = image_list_installed_packages(d)
     0842:
 *** 0843:    combine_spdx(d, image_name, imgdeploydir, img_spdxid, packages)
     0844:
     0845:    def make_image_link(target_path, suffix):
     0846:        if image_link_name:
     0847:            link = imgdeploydir / (image_link_name + suffix)
File: '/opt/ext-sdk/resy/4.1.66/multi-v7-ml/imx6q-phytec-mira-rdk-nand/layers/poky-master/meta/classes/create-spdx.bbclass', lineno: 1004, function: combine_spdx
     1000:                for ref in doc.externalDocumentRefs:
     1001:                    ref_path = deploy_dir_spdx / "by-namespace" / ref.spdxDocument.replace("/", "_")
     1002:                    collect_spdx_document(ref_path)
     1003:
 *** 1004:            collect_spdx_document(image_spdx_path)
     1005:
     1006:            index["documents"].sort(key=lambda x: x["filename"])
     1007:
     1008:            index_str = io.BytesIO(json.dumps(
File: '/opt/ext-sdk/resy/4.1.66/multi-v7-ml/imx6q-phytec-mira-rdk-nand/layers/poky-master/meta/classes/create-spdx.bbclass', lineno: 1002, function: collect_spdx_document
     0998:                    })
     0999:
     1000:                for ref in doc.externalDocumentRefs:
     1001:                    ref_path = deploy_dir_spdx / "by-namespace" / ref.spdxDocument.replace("/", "_")
 *** 1002:                    collect_spdx_document(ref_path)
     1003:
     1004:            collect_spdx_document(image_spdx_path)
     1005:
     1006:            index["documents"].sort(key=lambda x: x["filename"])
File: '/opt/ext-sdk/resy/4.1.66/multi-v7-ml/imx6q-phytec-mira-rdk-nand/layers/poky-master/meta/classes/create-spdx.bbclass', lineno: 1002, function: collect_spdx_document
     0998:                    })
     0999:
     1000:                for ref in doc.externalDocumentRefs:
     1001:                    ref_path = deploy_dir_spdx / "by-namespace" / ref.spdxDocument.replace("/", "_")
 *** 1002:                    collect_spdx_document(ref_path)
     1003:
     1004:            collect_spdx_document(image_spdx_path)
     1005:
     1006:            index["documents"].sort(key=lambda x: x["filename"])
File: '/opt/ext-sdk/resy/4.1.66/multi-v7-ml/imx6q-phytec-mira-rdk-nand/layers/poky-master/meta/classes/create-spdx.bbclass', lineno: 1002, function: collect_spdx_document
     0998:                    })
     0999:
     1000:                for ref in doc.externalDocumentRefs:
     1001:                    ref_path = deploy_dir_spdx / "by-namespace" / ref.spdxDocument.replace("/", "_")
 *** 1002:                    collect_spdx_document(ref_path)
     1003:
     1004:            collect_spdx_document(image_spdx_path)
     1005:
     1006:            index["documents"].sort(key=lambda x: x["filename"])
File: '/opt/ext-sdk/resy/4.1.66/multi-v7-ml/imx6q-phytec-mira-rdk-nand/layers/poky-master/meta/classes/create-spdx.bbclass', lineno: 1002, function: collect_spdx_document
     0998:                    })
     0999:
     1000:                for ref in doc.externalDocumentRefs:
     1001:                    ref_path = deploy_dir_spdx / "by-namespace" / ref.spdxDocument.replace("/", "_")
 *** 1002:                    collect_spdx_document(ref_path)
     1003:
     1004:            collect_spdx_document(image_spdx_path)
     1005:
     1006:            index["documents"].sort(key=lambda x: x["filename"])
File: '/opt/ext-sdk/resy/4.1.66/multi-v7-ml/imx6q-phytec-mira-rdk-nand/layers/poky-master/meta/classes/create-spdx.bbclass', lineno: 1002, function: collect_spdx_document
     0998:                    })
     0999:
     1000:                for ref in doc.externalDocumentRefs:
     1001:                    ref_path = deploy_dir_spdx / "by-namespace" / ref.spdxDocument.replace("/", "_")
 *** 1002:                    collect_spdx_document(ref_path)
     1003:
     1004:            collect_spdx_document(image_spdx_path)
     1005:
     1006:            index["documents"].sort(key=lambda x: x["filename"])
File: '/opt/ext-sdk/resy/4.1.66/multi-v7-ml/imx6q-phytec-mira-rdk-nand/layers/poky-master/meta/classes/create-spdx.bbclass', lineno: 1002, function: collect_spdx_document
     0998:                    })
     0999:
     1000:                for ref in doc.externalDocumentRefs:
     1001:                    ref_path = deploy_dir_spdx / "by-namespace" / ref.spdxDocument.replace("/", "_")
 *** 1002:                    collect_spdx_document(ref_path)
     1003:
     1004:            collect_spdx_document(image_spdx_path)
     1005:
     1006:            index["documents"].sort(key=lambda x: x["filename"])
File: '/opt/ext-sdk/resy/4.1.66/multi-v7-ml/imx6q-phytec-mira-rdk-nand/layers/poky-master/meta/classes/create-spdx.bbclass', lineno: 972, function: collect_spdx_document
     0968:                    return
     0969:
     0970:                visited_docs.add(path)
     0971:
 *** 0972:                with path.open("rb") as f:
     0973:                    doc, sha1 = oe.sbom.read_doc(f)
     0974:                    f.seek(0)
     0975:
     0976:                    if doc.documentNamespace in visited_docs:
File: '/opt/ext-sdk/resy/4.1.66/multi-v7-ml/imx6q-phytec-mira-rdk-nand/buildtools/sysroots/x86_64-resysdk-linux/usr/lib/python3.10/pathlib.py', lineno: 1119, function: open
     1115:        the built-in open() function does.
     1116:        """
     1117:        if "b" not in mode:
     1118:            encoding = io.text_encoding(encoding)
 *** 1119:        return self._accessor.open(self, mode, buffering, encoding, errors,
     1120:                                   newline)
     1121:
     1122:    def read_bytes(self):
     1123:        """
Exception: FileNotFoundError: [Errno 2] No such file or directory: '/opt/ext-sdk/resy/4.1.66/multi-v7-ml/imx6q-phytec-mira-rdk-nand/tmp/deploy/spdx/multi-v7-ml/by-namespace/http:__spdx.org_spdxdoc_recipe-icecc-create-env-native-485b3afa-6c5d-584d-b84d-ed3c7fe6a84a'

ERROR: Logfile of failure stored in: /opt/ext-sdk/resy/4.1.66/multi-v7-ml/imx6q-phytec-mira-rdk-nand/tmp/work/multi_v7_ml-resy-linux-gnueabi/core-image-minimal/1.0-r0/temp/log.do_rootfs.125572
ERROR: Task (/opt/ext-sdk/resy/4.1.66/multi-v7-ml/imx6q-phytec-mira-rdk-nand/layers/poky-master/meta/recipes-core/images/core-image-minimal.bb:do_rootfs) failed with exit code '1'
NOTE: Tasks Summary: Attempted 1877 tasks of which 1876 didn't need to be rerun and 1 failed.
NOTE: The errors for this build are stored in /opt/ext-sdk/resy/4.1.66/multi-v7-ml/imx6q-phytec-mira-rdk-nand/tmp/log/error-report/error_report_20221101192140.txt
You can send the errors to a reports server by running:
  send-error-report /opt/ext-sdk/resy/4.1.66/multi-v7-ml/imx6q-phytec-mira-rdk-nand/tmp/log/error-report/error_report_20221101192140.txt [-s server]
NOTE: The contents of these logs will be posted in public if you use the above command with the default server. Please ensure you remove any identifying or proprietary information when prompted before sending.

Summary: 1 task failed:
  /opt/ext-sdk/resy/4.1.66/multi-v7-ml/imx6q-phytec-mira-rdk-nand/layers/poky-master/meta/recipes-core/images/core-image-minimal.bb:do_rootfs
Summary: There were 2 WARNING messages.
Summary: There was 1 ERROR message, returning a non-zero exit code.


If I comment out from local.conf this line 

# INHERIT += "create-spdx"

the image builds.

The eudev:do_packagedata sig is computed to be 54e01735bc08b494d71c1f39aecae893368e000c4e53e40ede63da901b43b2a3, but the sig is locked to fec55a328154cc7d4685528610e1097c783bdd9285d014f1700a499a79aac440 in SIGGEN_LOCKEDSIGS_t-armv7at2hf-neon
Removing 2 stale sstate objects for arch multi_v7_ml: 100% |###################################################################################################################| Time: 0:00:00
NOTE: Executing Tasks
NOTE: Tasks Summary: Attempted 1579 tasks of which 1571 didn't need to be rerun and all succeeded.

Summary: There were 2 WARNING messages.
INFO: Successfully built core-image-minimal. You can find output files in /opt/ext-sdk/resy/4.1.66/multi-v7-ml/imx6q-phytec-mira-rdk-nand/tmp/deploy/images/multi-v7-ml
+ ls tmp/deploy/images/multi-v7-ml/core-image-minimal-multi-v7-ml-20221101192528.rootfs.tar.bz2 tmp/deploy/images/multi-v7-ml/core-image-minimal-multi-v7-ml.tar.bz2
tmp/deploy/images/multi-v7-ml/core-image-minimal-multi-v7-ml-20221101192528.rootfs.tar.bz2  tmp/deploy/images/multi-v7-ml/core-image-minimal-multi-v7-ml.tar.bz2
Comment 1 Randy MacLeod 2022-11-03 14:42:35 UTC
Perhaps something to do with the lock signatures.
Comment 2 Joshua Watt 2023-03-30 14:27:26 UTC
Robert,

Can you reproduce this and attach the local.conf from your eSDK? I think something else might be going on here.

Thanks
Comment 3 Joshua Watt 2023-03-30 14:28:16 UTC
Changing to NEEDINFO
Comment 4 Randy MacLeod 2023-04-20 14:58:15 UTC
Add depends.
Comment 5 Richard Purdie 2023-06-08 20:00:27 UTC
We've fixed a load of issues like this in master and have enabled create_spdx by default. This means eSDKs are now being built and tested on the autobuilder with this enabled so this should all be working now.