15909 2025-06-18 04:36:25 +0000 crate://crates.io is not a valid download_location for SPDX 2.2 2025-11-06 06:08:07 +0000 1 1 7 Build System, Metadata & Runtime OE-Core oe-core other 5.0.15 All Multiple RESOLVED FIXED Medium+ normal 5.3 M4 1 balsup Maganuru.Jayasurya ccasciato JPEWhacker Maganuru.Jayasurya randy.macleod sundeep.kokkonda --- --- Don't know oldest_to_newest 102283 0 balsup 2025-06-18 04:36:25 +0000 After moving from dunfell to scarthgap, I am seeing validation errors in the generated SPDX files, e.g. [sbom] package download_location must be a valid URL or download location according to the specification, but is: crate://crates.io/windows_i686_gnu/0.52.3 [sbom] package download_location must be a valid URL or download location according to the specification, but is: crate://crates.io/windows_aarch64_msvc/0.52.3 [sbom] package download_location must be a valid URL or download location according to the specification, but is: crate://crates.io/windows_aarch64_gnullvm/0.52.3 I am also finding that SPDX generation behavior has changed between dunfell and scarthgap, with SPDX files not being generated where expected. This seems related to certain tasks not being built if they are only runtime dependencies (possibly related to https://git.openembedded.org/openembedded-core/commit/?id=220d29a5abb5be8b80c64d8f2f48949bc8c534a3 ). A workaround seems to be explicitly adding --runall create_runtime_spdx --runall create_spdx to the bitbake command (similar to the workaround suggested in the given url to use --runall build) 102284 1 balsup 2025-06-18 04:37:41 +0000 I presume create-spdx class should massage the url similar to how the fetcher does it, e.g. if host == 'crates.io': host = 'crates.io/api/v1/crates' ud.url = "https://%s/%s/%s/download" % (host, name, version) 102288 2 balsup 2025-06-18 15:36:25 +0000 Should the fetcher itself support a function to translate a given (fetcher) URL into a list of download locations? This problem first presented itself with gitsm, and there is a translation in fetch_data_to_uri (although it seems to simply ignore the submodules). There are bound to be fetchers in the future that do their own thing, so supporting an optional fetcher method to return a list of download locations (or perhaps mappings between download locations and sets of downloaded files) would seem to make sense, rather than putting exceptions into spdx_common.py. 102303 3 randy.macleod 2025-06-19 14:40:46 +0000 Internally spdx has mappings of varioius URLS. Rust wasn't in dunfell so it's likely that there's just a missing mapping for crates.io. So yes, we should have a translation function as Billie suggested. Can you do that Billie ? -- YP bug triage 102329 4 5124 balsup 2025-06-23 15:52:59 +0000 Created attachment 5124 Workaround for crate fetcher URLs being invalid SPDX URLs 102330 5 balsup 2025-06-23 15:55:18 +0000 Let me get the required approvals here, and then I'll take a look at a long term solution. In the meantime, I've attached our scarthgap workaround. 102352 6 randy.macleod 2025-06-26 17:53:32 +0000 Hi Billie, Thanks for the patch / workaround. It might help someone who happens to read this defect. I'll move this bug to accepted and hopefully it won't take too long for you to get local approval to contribute to Yocto so the patch can be merged to master and then back-ported as needed. 102546 7 balsup 2025-08-05 15:21:31 +0000 I am still in the waiting for business approval state (which is unlikely to come until I finish some internal projects). The given workaround was sufficient for us to proceed, and deprioritized work to have a generic solution going forward. 102610 8 randy.macleod 2025-08-14 16:24:23 +0000 Joshua, Should Sundeep have someone submit the work-around or is there a better approach? 102798 9 JPEWhacker 2025-09-18 15:04:45 +0000 Yes, the munging of the URI is correct for crates, but please do it in the fetch_data_to_uri function in spdx_common.py. Ideally, the actual URI comes from the fetcher and we just need to know to use that for crates (or others) 102829 10 JPEWhacker 2025-09-25 15:07:07 +0000 See above comment 102928 11 Maganuru.Jayasurya 2025-10-14 13:00:58 +0000 (In reply to Joshua Watt from comment #10) > See above comment Hi Joshua, I've made the equivalent changes of the workaround, to fetch_data_to_uri function of spdx_common.py and manually verified the generated spdx.json (for the recipe python3-bcrypt). The software_downloadLocation is now correctly set to a valid URL, e.g.: "software_downloadLocation": "https://crates.io/api/v1/crates/windows_i686_gnu/0.48.5/download" instead of the invalid crate://crates.io/... format previously used for Rust crates. Should I be validating the spdx.json output using any specific tool before sending a patch to poky master? Or is this manual check sufficient to move forward? 103191 12 sundeep.kokkonda 2025-11-06 06:08:07 +0000 Patch merged - https://git.openembedded.org/openembedded-core/commit/?id=7cadbd1a22e18847d03b5baa902f5581d3e0aafa 5124 2025-06-23 15:52:59 +0000 2025-06-23 15:52:59 +0000 Workaround for crate fetcher URLs being invalid SPDX URLs file_15909.txt text/plain 1128 balsup Y29tbWl0IDBhODZkNzJlNWEzZTMxNWYwNmU4YjIxZmY0ZTIxYmFkZmIzOGZjNjENCkF1dGhvcjog U3plIEhvIEhlbGlvcyBUc29pIDxoZXRzb2lAY2lzY28uY29tPg0KRGF0ZTogICBXZWQgSnVuIDE4 IDIyOjM0OjUyIDIwMjUgLTA3MDANCg0KICAgIFNwZWNpYWwgaGFuZGxpbmcgZm9yIGNyYXRlOiBV UklzIGluIFNQRFggZ2VuZXJhdGlvbg0KICAgIA0KICAgIGNyYXRlOi8vIGlzIG5vdCBhIHZhbGlk IFVSTCBmb3IgU1BEWC4gIFJlcGxhY2UgaXQgd2l0aCBjb250ZW50cw0KICAgIGZyb20gdGhlIHVy bCBmaWVsZCBvZiB0aGUgZmV0Y2hkYXRhIGlmIGl0IGV4aXN0cy4NCiAgICANCiAgICBTaWduZWQt b2ZmLWJ5OiBTemUgSG8gSGVsaW9zIFRzb2kgPGhldHNvaUBjaXNjby5jb20+DQoNCmRpZmYgLS1n aXQgYS9tZXRhL2NsYXNzZXMvY3JlYXRlLXNwZHgtMi4yLmJiY2xhc3MgYi9tZXRhL2NsYXNzZXMv Y3JlYXRlLXNwZHgtMi4yLmJiY2xhc3MNCmluZGV4IGFkZTFhMDRiZTMuLjZkNWNjNGM1OWIgMTAw NjQ0DQotLS0gYS9tZXRhL2NsYXNzZXMvY3JlYXRlLXNwZHgtMi4yLmJiY2xhc3MNCisrKyBiL21l dGEvY2xhc3Nlcy9jcmVhdGUtc3BkeC0yLjIuYmJjbGFzcw0KQEAgLTQ0NCw2ICs0NDQsMTAgQEAg ZGVmIGFkZF9kb3dubG9hZF9wYWNrYWdlcyhkLCBkb2MsIHJlY2lwZSk6DQogICAgICAgICAgICAg aWYgZi5tZXRob2Quc3VwcG9ydHNfc3JjcmV2KCk6DQogICAgICAgICAgICAgICAgIHVyaSA9IHVy aSArICJAIiArIGYucmV2aXNpb25zW25hbWVdDQogDQorICAgICAgICAgICAgIyBjcmF0ZTogaXMg bm90IGEgdmFsaWQgVVJMLiAgVXNlIHVybCBmaWVsZCBpbnN0ZWFkIGlmIGV4aXN0DQorICAgICAg ICAgICAgaWYgZi50eXBlID09ICJjcmF0ZSIgYW5kICd1cmwnIGluIGRpcihmKToNCisgICAgICAg ICAgICAgICAgdXJpID0gZi51cmwNCisNCiAgICAgICAgICAgICBpZiBmLm1ldGhvZC5zdXBwb3J0 c19jaGVja3N1bShmKToNCiAgICAgICAgICAgICAgICAgZm9yIGNoZWNrc3VtX2lkIGluIENIRUNL U1VNX0xJU1Q6DQogICAgICAgICAgICAgICAgICAgICBpZiBjaGVja3N1bV9pZC51cHBlcigpIG5v dCBpbiBvZS5zcGR4LlNQRFhQYWNrYWdlLkFMTE9XRURfQ0hFQ0tTVU1TOg0K