Bug 15012

See discussion on email list about how to implement this.

The email thread is:
https://lore.kernel.org/openembedded-core/CAO5Uq5RftGpqzF1=6NUhQChCqCyp2D-ud8KUB69u6+9=N1VkAg@mail.gmail.com/

There has been some work on the crate fetcher (thanks Alex, Frederick) but I think there's more to do so moving to 4.3.

Hello,

The checksum verification for crate fetcher no more rely solely on cargo job but is made by the parser itself, this verification is in mickledore already.

So adding

SRC_URI[binary-security-check-1.2.7.sha256sum] = "fc94808e1401ec3b4c5204cbee27dc83522518dc925c35c6271b7367a54ca143"

make the recipe valid.

Nevertheless, I try that out of curiosity and such a recipe (which have a crate fetched data as primary artifact) doesn't compile.
The fetched crate (here binary-security-check) is unpacked into ${CARGO_HOME}/bitbake (CARGO_VENDORING_DIRECTORY) and so the compile step doesn't find anything into ${B} for compiling:

DEBUG: Executing shell function do_compile
NOTE: Using rust targets from /home/jenkins/yocto-poky-master/poky/build/tmp/work/core2-64-poky-linux/binary-security-check-crates/1.2.7-r0/rust-targets/
NOTE: cargo = /home/jenkins/yocto-poky-master/poky/build/tmp/work/core2-64-poky-linux/binary-security-check-crates/1.2.7-r0/recipe-sysroot-native/usr/bin/cargo
NOTE: cargo build -v --offline --target x86_64-poky-linux-gnu --release --manifest-path=/home/jenkins/yocto-poky-master/poky/build/tmp/work/core2-64-poky-linux/binary-security-check-crates/1.2.7-r0/binary-security-check-crates-1.2.7//Cargo.toml 
error: manifest path `/home/jenkins/yocto-poky-master/poky/build/tmp/work/core2-64-poky-linux/binary-security-check-crates/1.2.7-r0/binary-security-check-crates-1.2.7//Cargo.toml` does not exist
WARNING: exit code 101 from a shell command



[ (12) ] jenkins@cibuilder2:~/yocto-poky-master$ ls -l /home/jenkins/yocto-poky-master/poky/build/tmp/work/core2-64-poky-linux/binary-security-check-crates/1.2.7-r0/cargo_home/bitbake/binary-security-check-1.2.7/
total 36
-rw-r--r-- 1 jenkins jenkins 11061 Jan  1  1970 Cargo.lock
-rw-r--r-- 1 jenkins jenkins  1832 Jan  1  1970 Cargo.toml
-rwxr-xr-x 1 jenkins jenkins  1651 Jul 24  2006 Cargo.toml.orig
-rwxr-xr-x 1 jenkins jenkins  1101 Jul 24  2006 LICENSE.txt
-rwxr-xr-x 1 jenkins jenkins  6202 Jul 24  2006 README.md
drwxr-xr-x 6 jenkins jenkins  4096 Apr 14 03:50 src


I don't know if we want to support such kind of recipe (instead of using git directly, in the example that would be:

SRC_URI = "git://github.com/koutheir/binary-security-check;protocol=https;"

If we want to support crate fetcher as primary artifact, there is indeed a problem, and if so, I suggest to at least rewrite the bug title for better clarity

Well look at this when we get time, hopefully in 4.3.

from comment4:
After making the recipe valid by adding the checksum showing in Error. while doing build with  bitbake  we will get compilation errors because the fetched crate (here binary-security-check) is unpacked into ${CARGO_HOME}/bitbake (CARGO_VENDORING_DIRECTORY) and so the compile step doesn't find anything into ${B} for compiling.
for this problem we must set S in the recipe so that the OpenEmbedded build system knows where to find the unpacked source.
ex: S="{WORKDIR}/cargo_home/bitbake/${BPN}-${PV} ,where ${BPN} is the base recipe name and ${PV} is the recipe version. In our case we have to use S="${WORKDIR}/cargo_home/bitbake/binary-security-check-1.2.7"

for checksum verfication we have tried by passing bogus checksum in recipe file we are getting checksum mismatch.
ERROR: binary-security-check-1.0-r0 do_fetch: Fetcher failure for URL: 'https://crates.io/api/v1/crates/binary-security-check/1.2.7/download'. Checksum mismatch!
File: '/ala-lpggp31/dhemraj/harish/lincd-13415/poky/build/downloads/binary-security-check-1.2.7.crate.tmp' has sha256 checksum 'fc94808e1401ec3b4c5204cbee27dc83522518dc925c35c6271b7367a54ca143' when 'fc94808e1401ec3b4c5204cbee27dc83522518dc925c35c6271b7367a54ca143abc' was expected
If this change is expected (e.g. you have upgraded to a new version without updating the checksums) then you can use these lines within the recipe:
SRC_URI[binary-security-check-1.2.7.sha256sum] = "fc94808e1401ec3b4c5204cbee27dc83522518dc925c35c6271b7367a54ca143"
Otherwise you should retry the download and/or check with upstream to determine if the file has become corrupted or otherwise unexpectedly modified.

and we have also checked the cheksum verification for downloaded crates by passing bogus checksum in the binary-security-check-crates.inc we will git following error:
ERROR: binary-security-check-1.0-r0 do_fetch: Checksum failure fetching crate://crates.io/aho-corasick/0.7.19
ERROR: binary-security-check-1.0-r0 do_fetch: Bitbake Fetcher Error: ChecksumError('Checksum mismatch!\nFile: \'/ala-lpggp31/dhemraj/harish/lincd-13415/poky/build/downloads/aho-corasick-0.7.19.crate\' has sha256 checksum \'b4f55bd91a0978cbfd91c457a164bab8b4001c833b7f323132c0a4e1922dd44e\' when \'b4f55bd91a0978cbfd91c457a164bab8b4001c833b7f323132c0a4e1922dd44eb\' was expected\nIf this change is expected (e.g. you have upgraded to a new version without updating the checksums) then you can use these lines within the recipe:\nSRC_URI[aho-corasick-0.7.19.sha256sum] = "b4f55bd91a0978cbfd91c457a164bab8b4001c833b7f323132c0a4e1922dd44e"\nOtherwise you should retry the download and/or check with upstream to determine if the file has become corrupted or otherwise unexpectedly modified.', 'https://crates.io/api/v1/crates/aho-corasick/0.7.19/download')

Harish, is this resolved in 5.1 ? If not move to 5.2 and remind me to bring it up in our meetings.

With the latest Poky source, if you don't include the checksum in the recipe, an error will be thrown as expected:

ERROR: example-0.1-r0 do_fetch: Missing SRC_URI checksum, please add those to the recipe:
SRC_URI[binary-security-check-1.2.7.sha256sum] = "fc94808e1401ec3b4c5204cbee27dc83522518dc925c35c6271b7367a54ca143"
ERROR: example-0.1-r0 do_fetch: Bitbake Fetcher Error: BBFetchException('There was some missing checksums in the recipe')

This issue is no longer a concern.

Please investigate writing an oeqa test to confirm that when the checksum is NOT correct, there is always an error. This will catch any regression on this bug.

Checksum support was implemented for the crate fetcher in BitBake with the following commit. https://git.openembedded.org/bitbake/commit/?id=4920686c13dd66f9bfa4f7dd38d6e955f153eeec
 
There is also a test case availble in this commit to check whether the checksum is correct/incorrect. For the missing checksum also the bb will throw an error.

Thanks for checking Harish and thanks for the patch Frederic !