Bug 14837

Summary: devtool modify doesnt fetch all the crates on rust based recipes
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Alejandro <alejandro>
Component: Scripts and ToolsAssignee: Eilís Ní Fhlannagáin <pidge>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium+ CC: randy.macleod, ripastri, ross.burton
Version: unspecified   
Target Milestone: 4.3 M4   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Don't know

Description Alejandro 2022-06-14 16:32:24 UTC
When using devtool modify <foo> (if <foo> uses the crate:// fetcher) bitbake fails to build <foo>.

From what I can see, this is caused because rust based recipes contain several crate:// lines and devtool only fetches the first main source code.

Then, when trying to build such recipe after having it in the workspace, the buildsystem (I believe cargo) tries to fetch the missing crates on do_compile.



Using python3-cryptography as an example since its rust based:

$ devtool modify python3-cryptography
INFO: Source tree extracted to builds/qemuarm64/workspace/sources/python3-cryptography                                                                                              
INFO: Recipe python3-cryptography now set up to build from builds/qemuarm64/wor
kspace/sources/python3-cryptography


$ bitbake python3-cryptography


|     Updating crates.io index                                                                       | warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: github.com); class=Net (12)                                                                
| warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name (Could not reso
lve host: github.com); class=Net (12)                                                                | error: failed to get `asn1` as a dependency of package `cryptography-rust v0.1.0 (builds/qemuarm64/workspace/sources/python3-cryptography/src/rust)`                              
|                                                 
| Caused by:                                                                                         |   failed to load source for dependency `asn1`                                                      |                                                                                                    
| Caused by:                                                                                         
|   Unable to update registry `crates-io`                                                            |                                                                                                    | Caused by:                                                                                         
|   failed to fetch `https://github.com/rust-lang/crates.io-index`                                   
|                                                                                                    | Caused by:                                                                                         |   network failure seems to have happened                                                           
|   if a proxy or similar is necessary `net.git-fetch-with-cli` may help here                        
|   https://doc.rust-lang.org/cargo/reference/config.html#netgit-fetch-with-cli                      |                                                                                                    | Caused by:                                                                                         
|   [6] Couldn't resolve host name (Could not resolve host: github.com); class=Net (12)

ERROR: Task (poky/meta/recipes-devtools/python/python3-cryptography_36.0.2.bb:do_compile) failed with exit code '1


This recipe builds just fine when its not in the devtool workspace.

I don't believe we were aware of this issue because we allowed tasks like do_compile to access the network, but since commit https://git.openembedded.org/openembedded-core-contrib/commit/meta/classes/base.bbclass?id=7ce1e88a3ad85bbb925bb9f7167dc0a5fd1c27f4 we don't allow do_compile to access the network (which is a good thing, I'm just clarifying that maybe that's why we hadn't seen this before)
Comment 1 Richard Pastrick 2022-07-19 19:09:19 UTC
This appears to have been fixed in master as of these two commits.

https://git.yoctoproject.org/poky/commit/?id=c31ee3cb121ada557f09f53ab3e92a91160b6e08 
https://git.yoctoproject.org/poky/commit/?id=19081924df2573ded966a56639e6eb2bf27d8f61

Additionally, once finding those commits, I sent those as a request to be backported to kirkstone.
Comment 2 Randy MacLeod 2024-01-18 16:26:45 UTC
Fixed as per previous comment.