Bug 16503 - distro_check: create_distro_packages_list() crashes when a distro can't be fetched (Debian 404)
Summary: distro_check: create_distro_packages_list() crashes when a distro can't be fe...
Status: IN PROGRESS REVIEW
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: oe-core other (show other bugs)
Version: 6.99
Hardware: All Multiple
: Medium normal
Target Milestone: 6.2 M2
Assignee: Tim Orling
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2026-11-14 07:57 UTC by Tim Orling
Modified: 2026-11-15 00:26 UTC (History)
0 users

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Don't know


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Tim Orling 2026-11-14 07:57:11 UTC
create_distro_packages_list() in meta/lib/oe/distro_check.py crashes as soon as one distro's package list can't be fetched, so no distro package lists are written at all:

  WARNING: Cannot fetch packages for Debian: HTTP Error 404: Not Found
  File ".../meta/lib/oe/distro_check.py", line 173, in create_distro_packages_list
    bb.note("Distro: %s, Latest Release: %s, # src packages: %d" % (name, release, len(package_list)))
  UnboundLocalError: cannot access local variable 'release' where it is not associated with a value

(OE-Core master, f32bced7b4, 2026-11-13.)

Two problems:

1. When fetcher_func() raises, the except block only warns and then carries on using release and package_list, which were never set (or are left over from the previous distro). It should skip to the next distro ("continue") after the warning.

2. The Debian list currently always fails: get_latest_released_debian_source_package_list() fetches
   http://ftp.debian.org/debian/dists/stable-proposed-updates/main/source/Sources.gz
   which returns 404 (stable/main/source/Sources.gz still exists). The proposed-updates list should be optional, or fetched from where Debian now publishes it.

This breaks the Recipe Reporting System's rrs_distros.py on the layer index, which calls create_distro_packages_list() to show which distros package each recipe.
Comment 1 Tim Orling 2026-11-14 20:10:42 UTC
Patches sent to openembedded-core, under review:
https://lore.kernel.org/all/20261114200810.31078-1-tim.orling@konsulko.com/

1/3 lib/oe/distro_check: skip a distro whose package list can't be fetched
2/3 lib/oe/distro_check: fetch Debian updates from stable-updates
3/3 oeqa/selftest/oelib: add tests for oe.distro_check

Patchwork series 53375 (patches 104067-104069).
Comment 2 Tim Orling 2026-11-15 00:26:51 UTC
Tested with the three patches from series 53375 on a layer index test
instance (layers.moto-timo.net), running RRS's rrs_distros.py against
OE-Core master (f32bced7b4) plus the patches:

  NOTE: Distro: Debian, Latest Release: 13.7, # src packages: 37593
  NOTE: Distro: Ubuntu, Latest Release: xenial, # src packages: 3456
  NOTE: Distro: Fedora, Latest Release: 45, # src packages: 23237
  WARNING: Cannot fetch packages for openSUSE: HTTP Error 404: Not Found
  WARNING: Cannot fetch packages for Clear: <urlopen error [Errno -3] Temporary failure in name resolution>

The Debian list is fetched again, and the two distros that fail are now
skipped with a warning instead of stopping the run, so the package lists
for the other distros are written.

The run also shows three separate problems, not covered by these
patches:
- Ubuntu is detected as "xenial": find_latest_ubuntu_release() takes the
  most recently modified *-updates directory, which is not the latest
  release.
- The openSUSE Leap URLs return 404.
- Clear Linux has been discontinued, so download.clearlinux.org no
  longer resolves.