Bug 16153

Summary: wget fetcher garble file if upstream is corrupted and mirror has a bigger file
Product: [Build System, Metadata & Runtime] BitBake Reporter: Yoann Congal <yoann.congal>
Component: bitbakeAssignee: Unassigned <unassigned>
Status: NEW --- QA Contact:
Severity: normal    
Priority: Medium CC: poky.bs.watcher, poky.watcher, randy.macleod, tgamblin, tim.orling
Version: unspecified   
Target Milestone: 6.1   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Don't know

Description Yoann Congal 2026-01-27 20:13:16 UTC
When:
* SRC_URI = "http://..." (the wget fetcher is used), 
* SRC_URI[sha256sum] does not match what is downloaded and
* the yp.org MIRRORS has a matching file that is bigger than the downloaded file
Then, the fetcher fail to correctly download the file.

What happen is:
* The fetcher downloads the corrupted file
* The file does not match the hash so this is considered a failure to download
* The mirror is tried *with the wget --continue* option[0].
* The mirrored file being bigger, the download is "continued" by wget: the end of the mirrored file is appended to the corrupted file.
* The resulting file does not match the recipe hash as well so the fetch task fails.

I guess we could:
* remove the .tmp file between upstream/mirror tries
* remove the --continue argument of wget: (see wget manpage[1]):
> However, if the file is bigger on the server because it’s been changed, as
> opposed to just appended to, *you’ll end up with a garbled file*. Wget has no
> way of verifying that the local file is really a valid prefix of the remote
> file. You need to be especially careful of this when using ‘-c’ in
> conjunction with ‘-r’, since every file will be considered as an "incomplete
> download" candidate.

I would remove --continue but I've yet to track down why it was added in the first place (it was before the fetch -> fetch2 migration, 14 years ago)

See: the meta-oe issue that triggered this bug: https://github.com/openembedded/meta-openembedded/issues/1023 (patch for it was sent https://lists.openembedded.org/g/openembedded-devel/message/123959)

[0]: https://github.com/openembedded/bitbake/blob/cdd79c1768ac396a9c6577e38098da4331507f24/lib/bb/fetch2/wget.py#L125
[1]: https://www.gnu.org/software/wget/manual/wget.html#index-continue-retrieval
Comment 1 Randy MacLeod 2026-01-29 15:47:24 UTC
This can still be useful for large downloads that get interrupted
but there may be too many cases where --continue is wrong.

Maybe we should change the point where the renaming happens.

It would be nice to add a test case as well.
Comment 2 Randy MacLeod 2026-05-28 13:16:30 UTC
Bulk move of 6.0 Medium bugs to 6.1.
Comment 3 Yoann Congal 2026-10-06 22:53:04 UTC
This merged patch sounds related:
fetch/wget: rename the partial download on checksum mismatch
https://git.openembedded.org/bitbake/commit/?id=37b9c56ae3b2294ad170859a638c879910c82c59

and its follow-up:
[PATCH] fetch/wget: rename the partial download on checksum mismatch
https://lore.kernel.org/bitbake-devel/20261006213334.3360919-1-ricardo.salveti@oss.qualcomm.com/T/#u