Bug 16410

Summary: toaster: test_run_lsupdates_command: KeyError
Product: [Build System, Metadata & Runtime] Toaster Reporter: Mathieu Dubois-Briand <mathieu.dubois-briand>
Component: toasterAssignee: David Reyna <david.reyna>
Status: ACCEPTED --- QA Contact: Toaster default assignee <toaster>
Severity: normal    
Priority: Undecided    
Version: unspecified   
Target Milestone: 0.0.0   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Mathieu Dubois-Briand 2026-08-28 06:34:35 UTC 
We had this error for the third day in a row. We have a lot of logs, but I have absolutely no clue about what might be the issue here. To the point the current title of this bug entry is probably useless.

First occurrence was logged in https://bugzilla.yoctoproject.org/show_bug.cgi?id=15659.

> =================================== FAILURES ===================================
> _______________ TestLayerIndexUpdater.test_run_lsupdates_command _______________
> 
> self = <toaster.tests.commands.test_lsupdates.TestLayerIndexUpdater > testMethod=test_run_lsupdates_command>
> 
>     def test_run_lsupdates_command(self):
>         # Load some release information for us to fetch from the layer index
>         management.call_command('loaddata', 'poky')
>     
>         old_layers_count = Layer_Version.objects.count()
>         old_recipes_count = Recipe.objects.count()
>         old_machines_count = Machine.objects.count()
>     
>         # Now fetch the metadata from the layer index
>       management.call_command('lsupdates')
> 
> ../layers/bitbake/lib/toaster/tests/commands/test_lsupdates.py:27: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> venv/lib/python3.10/site-packages/django/core/management/__init__.py:194: in > call_command
>     return command.execute(*args, **defaults)
> venv/lib/python3.10/site-packages/django/core/management/base.py:458: in execute
>     output = self.handle(*args, **options)
> ../layers/bitbake/lib/toaster/orm/management/commands/lsupdates.py:289: in handle
>     self.update()
> ...
>                   > pk=li_layer_branch_id_to_toaster_lv_id[index.distros[id].layerbranch_id]))
> E           KeyError: 2073009
>
> ../layers/bitbake/lib/toaster/orm/management/commands/lsupdates.py:232: KeyError


Later followed by:

> ../layers/bitbake/lib/toaster/tests/functional/test_project_page.py:148: in _navigate_to_config_nav
>     self.wait_until_visible(f'#{nav_id}')
> ../layers/bitbake/lib/toaster/tests/browser/selenium_helpers_base.py:225: in wait_until_visible
>     Wait(self.driver, timeout=timeout).until(is_visible, msg)
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _·
>·
> self = <tests.browser.selenium_helpers_base.Wait (session="eec9d1244ca5c9d33605904ac2b2cd69")>
> method = <function SeleniumTestCaseBase.wait_until_visible.<locals>.<lambda> at 0x7f4580c5d120>
> message = 'An element matching "#machinestable" should be visible'
> ... 
> >       raise TimeoutException(message)
> E       selenium.common.exceptions.TimeoutException: Message: An element matching "#machinestable" should be visible
> ... 
> ../layers/bitbake/lib/toaster/tests/functional/test_project_page.py:385:·
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _·
> ../layers/bitbake/lib/toaster/tests/functional/test_project_page.py:148: in _navigate_to_config_nav
>     self.wait_until_visible(f'#{nav_id}')
> ../layers/bitbake/lib/toaster/tests/browser/selenium_helpers_base.py:225: in wait_until_visible
>     Wait(self.driver, timeout=timeout).until(is_visible, msg)
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _·
>·
> self = <tests.browser.selenium_helpers_base.Wait (session="eec9d1244ca5c9d33605904ac2b2cd69")>
> method = <function SeleniumTestCaseBase.wait_until_visible.<locals>.<lambda> at 0x7f4580dc7880>
> message = 'An element matching "#softwarerecipestable" should be visible'
> ... 
> >       raise TimeoutException(message)
> E       selenium.common.exceptions.TimeoutException: Message: An element matching "#softwarerecipestable" should be visible
> ... 
> ../layers/bitbake/lib/toaster/tests/functional/test_project_page_tab_config.py:190:·
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _·
> ../layers/bitbake/lib/toaster/tests/browser/selenium_helpers_base.py:218: in wait_until_present
>     element = Wait(self.driver, timeout=timeout).until(is_present, msg)
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _·
>·
> self = <tests.browser.selenium_helpers_base.Wait (session="c140ab36742448844170700e56e28b44")>
> method = <function SeleniumTestCaseBase.wait_until_present.<locals>.<lambda> at 0x7f458ce5b370>
> message = 'An element matching "#imagerecipestable tbody tr" should be on the page'
> ... 
> >       raise TimeoutException(message)
> E       selenium.common.exceptions.TimeoutException: Message: An element matching "#imagerecipestable tbody tr" should be on the page
> ... 
> ../layers/bitbake/lib/toaster/tests/functional/test_project_page_tab_config.py:223:·
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _·
> ../layers/bitbake/lib/toaster/tests/browser/selenium_helpers_base.py:218: in wait_until_present
>     element = Wait(self.driver, timeout=timeout).until(is_present, msg)
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _·
>·
> self = <tests.browser.selenium_helpers_base.Wait (session="c140ab36742448844170700e56e28b44")>
> method = <function SeleniumTestCaseBase.wait_until_present.<locals>.<lambda> at 0x7f458c85d900>
> message = 'An element matching "#imagerecipestable tbody tr" should be on the page'
> ... 
> >       raise TimeoutException(message)
> E       selenium.common.exceptions.TimeoutException: Message: An element matching "#imagerecipestable tbody tr" should be on the page
> ... 
> ../layers/bitbake/lib/toaster/tests/browser/selenium_helpers_base.py:225: in wait_until_visible
>     Wait(self.driver, timeout=timeout).until(is_visible, msg)
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _·
>·
> self = <tests.browser.selenium_helpers_base.Wait (session="c140ab36742448844170700e56e28b44")>
> method = <function SeleniumTestCaseBase.wait_until_visible.<locals>.<lambda> at 0x7f458c85e290>
> message = 'An element matching "#imagerecipestable" should be visible'
> ... 
> E       selenium.common.exceptions.TimeoutException: Message: An element matching "#imagerecipestable" should be visible
> ... 
> ../layers/bitbake/lib/toaster/tests/browser/selenium_helpers_base.py:218: in wait_until_present
>     element = Wait(self.driver, timeout=timeout).until(is_present, msg)
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _·
>·
> self = <tests.browser.selenium_helpers_base.Wait (session="e43353a54247ad9c63afe9da8e7ba66f")>
> method = <function SeleniumTestCaseBase.wait_until_present.<locals>.<lambda> at 0x7f458ce1ff40>
> message = 'An element matching "#imagerecipestable tbody tr" should be on the page'
> ... 
> >       raise TimeoutException(message)
> E       selenium.common.exceptions.TimeoutException: Message: An element matching "#imagerecipestable tbody tr" should be on the page
> ...
> ../layers/bitbake/lib/toaster/tests/functional/test_project_page.py:730:·
> ...
> >           raise self.model.DoesNotExist(
>                 "%s matching query does not exist." % self.model._meta.object_name
>             )
> E           orm.models.Recipe.DoesNotExist: Recipe matching query does not exist.
>·

And finally:

> =========================== short test summary info ============================
> FAILED ../layers/bitbake/lib/toaster/tests/commands/test_lsupdates.py::TestLayerIndexUpdater::test_run_lsupdates_command
> FAILED ../layers/bitbake/lib/toaster/tests/functional/test_project_page.py::TestProjectPage::test_machines_page
> FAILED ../layers/bitbake/lib/toaster/tests/functional/test_project_page.py::TestProjectPage::test_softwareRecipe_page
> FAILED ../layers/bitbake/lib/toaster/tests/functional/test_project_page_tab_config.py::TestProjectConfigTab::test_image_recipe_editColumn
> FAILED ../layers/bitbake/lib/toaster/tests/functional/test_project_page_tab_config.py::TestProjectConfigTab::test_image_recipe_show_rows
> FAILED ../layers/bitbake/lib/toaster/tests/functional/test_project_page_tab_config.py::TestProjectConfigTab::test_project_page_image_recipe
> FAILED ../layers/bitbake/lib/toaster/tests/functional/test_project_page.py::TestProjectPageRecipes::test_image_recipe_editColumn
> FAILED ../layers/bitbake/lib/toaster/tests/functional/test_project_page.py::TestProjectPageRecipes::test_single_recipe_page
> ============ 8 failed, 148 passed, 12 warnings in 549.89s (0:09:09) ============
Comment 1 Mathieu Dubois-Briand 2026-08-28 06:48:09 UTC 
toaster ubuntu2204-vk-1 master completed at 2026-08-27 03:09:47+00:00
E           KeyError: 2073009
E       selenium.common.exceptions.TimeoutException: Message: An element matching "#machinestable" should be visible
E       selenium.common.exceptions.TimeoutException: Message: An element matching "#softwarerecipestable" should be visible
E       selenium.common.exceptions.TimeoutException: Message: An element matching "#imagerecipestable tbody tr" should be on the page
E       selenium.common.exceptions.TimeoutException: Message: An element matching "#imagerecipestable" should be visible
https://autobuilder.yoctoproject.org/valkyrie/#/builders/88/builds/810/steps/14/logs/stdio

toaster ubuntu2510-vk-1 master completed at 2026-08-28 03:10:04+00:00
E           KeyError: 2073009
E       selenium.common.exceptions.TimeoutException: Message: An element matching "#machinestable" should be visible
E       selenium.common.exceptions.TimeoutException: Message: An element matching "#softwarerecipestable" should be visible
E       selenium.common.exceptions.TimeoutException: Message: An element matching "#imagerecipestable tbody tr" should be on the page
E       selenium.common.exceptions.TimeoutException: Message: An element matching "#imagerecipestable" should be visible
https://autobuilder.yoctoproject.org/valkyrie/#/builders/88/builds/811/steps/14/logs/stdio
Comment 2 David Reyna 2026-08-28 17:40:49 UTC 
I tried to instantiate a new Toaster from fresh against master and I got this error:

-Failure while trying to setup toaster: Unable to fetch layerindex url https://layers.openembedded.org/layerindex/api/;branch=wrynose,HEAD,master,scarthgap: Unable to fetch layerindex url https://layers.openembedded.org/layerindex/api/: HTTP Error 403: Forbidden

When I opened the URL I was briefly blocked by CloudFlare, but the browser quickly resolved it. My theory is that we need some way to accommodate CloudFlare from the python script context.

As for the AB observed error, this may be the same root cause, or it may be that the Layer Index has a poisoned reference.