Bug 15460

Summary:	AB-INT: toaster test failures
Product:	[QA/Testing] Functional (self) Testing	Reporter:	Alexandre Belloni <alexandre.belloni>
Component:	general	Assignee:	Unassigned <unassigned>
Status:	RESOLVED FIXED	QA Contact:
Severity:	normal
Priority:	Medium	CC:	alexandre.belloni, mathieu.dubois-briand, randy.macleod, richard.purdie, tim.orling
Version:	unspecified
Target Milestone:	5.2
Hardware:	x86
OS:	Multiple
Whiteboard:	AB-INT
OS type for building Yocto:	---	Type of Regression:	---
Verified:		Documentation change:	No (bug/feature does not impact docs)

Description Alexandre Belloni 2024-03-27 20:41:01 UTC

Suddenly, we got two toaster test failures:

https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/237/steps/11/logs/stdio

fedora38-ty-2


FAILED ../bitbake/lib/toaster/tests/browser/test_all_builds_page.py::TestAllBuildsPage::test_builds_table_editColumn
FAILED ../bitbake/lib/toaster/tests/browser/test_all_builds_page.py::TestAllBuildsPage::test_builds_table_search_box
FAILED ../bitbake/lib/toaster/tests/browser/test_all_builds_page.py::TestAllBuildsPage::test_builds_table_show_rows
FAILED ../bitbake/lib/toaster/tests/browser/test_all_builds_page.py::TestAllBuildsPage::test_builds_time_links
FAILED ../bitbake/lib/toaster/tests/browser/test_all_builds_page.py::TestAllBuildsPage::test_filtering_on_completedOn_column
FAILED ../bitbake/lib/toaster/tests/browser/test_all_builds_page.py::TestAllBuildsPage::test_filtering_on_failure_tasks_column
FAILED ../bitbake/lib/toaster/tests/browser/test_all_builds_page.py::TestAllBuildsPage::test_rebuild_buttons
FAILED ../bitbake/lib/toaster/tests/browser/test_all_builds_page.py::TestAllBuildsPage::test_show_tasks_with_suffix
FAILED ../bitbake/lib/toaster/tests/browser/test_all_builds_page.py::TestAllBuildsPage::test_tooltips_on_project_name
FAILED ../bitbake/lib/toaster/tests/browser/test_project_config_page.py::TestProjectConfigsPage::test_checkbox_verification
FAILED ../bitbake/lib/toaster/tests/browser/test_project_config_page.py::TestProjectConfigsPage::test_no_underscore_iamgefs_type
FAILED ../bitbake/lib/toaster/tests/browser/test_project_config_page.py::TestProjectConfigsPage::test_set_download_dir
FAILED ../bitbake/lib/toaster/tests/browser/test_project_config_page.py::TestProjectConfigsPage::test_set_sstate_dir
FAILED ../bitbake/lib/toaster/tests/browser/test_project_config_page.py::TestProjectConfigsPage::test_textbox_with_checkbox_verification


https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/236/steps/11/logs/stdio

fedora38-ty-6


../bitbake/lib/toaster/tests/functional/test_project_page.py:393: IndexError

Comment 1 Randy MacLeod 2024-03-28 14:46:49 UTC

First occurrence.

Comment 2 Alexandre Belloni 2024-04-08 09:47:02 UTC

https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/185/steps/11/logs/stdio

Comment 3 Mathieu Dubois-Briand 2024-09-12 08:23:33 UTC

Several toaster failures:

selenium.common.exceptions.TimeoutException: Message: timeout: Timed out receiving message from renderer: 300.000
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/404/steps/12/logs/stdio

selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/401/steps/12/logs/stdio

selenium.common.exceptions.WebDriverException: Message: unknown error: unhandled inspector error: {"code":-32000,"message":"Node with given id does not belong to the document"}
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/383/steps/11/logs/stdio

selenium.common.exceptions.TimeoutException: Message: An element matching "#latest-builds" should be visible
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/382/steps/12/logs/stdio

Comment 4 Mathieu Dubois-Briand 2024-09-20 09:59:05 UTC

E       selenium.common.exceptions.WebDriverException: Message: unknown error: net::ERR_CONNECTION_REFUSED
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/179/steps/11/logs/stdio

E       selenium.common.exceptions.WebDriverException: Message: unknown error: session deleted because of page crash
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/180/steps/11/logs/stdio

E       selenium.common.exceptions.WebDriverException: Message: unknown error: net::ERR_CONNECTION_REFUSED
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/274/steps/11/logs/stdio

E       selenium.common.exceptions.WebDriverException: Message: tab crashed
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/290/steps/11/logs/stdio

E       selenium.common.exceptions.WebDriverException: Message: unknown error: unhandled inspector error: {"code":-32000,"message":"Node with given id does not belong to the document"}
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/301/steps/12/logs/stdio

E       selenium.common.exceptions.TimeoutException: Message: An element matching "#hint-error-project-name" should be on the page
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/349/steps/12/logs/stdio

E       selenium.common.exceptions.WebDriverException: Message: unknown error: net::ERR_CONNECTION_REFUSED
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/351/steps/11/logs/stdio

E       selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/354/steps/11/logs/stdio

E       selenium.common.exceptions.WebDriverException: Message: tab crashed
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/358/steps/11/logs/stdio

E       selenium.common.exceptions.TimeoutException: Message: An element matching "#hint-error-project-name" should be on the page
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/417/steps/12/logs/stdio

E       selenium.common.exceptions.WebDriverException: Message: unknown error: unhandled inspector error: {"code":-32000,"message":"Node with given id does not belong to the document"}
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/420/steps/11/logs/stdio

E   AssertionError: None is not true : The project release is not defined
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/421/steps/11/logs/stdio

self.assertTrue(
    f'You have removed 1 layer from your project' in str(change_notification.text)
            )
E       AssertionError: False is not true
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/422/steps/11/logs/stdio

self.assertTrue(
    f'You have removed 1 layer from your project' in str(change_notification.text)
            )
E       AssertionError: False is not true
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/423/steps/11/logs/stdio

Comment 5 Mathieu Dubois-Briand 2024-09-23 15:08:38 UTC

E       selenium.common.exceptions.TimeoutException: Message: timeout: Timed out receiving message from renderer: 600.000
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/424/steps/11/logs/stdio

Comment 6 Tim Orling 2024-09-26 15:17:19 UTC

Errors on valkyrie:
https://valkyrie.yoctoproject.org/#/builders/88/builds/6

FAILED ../bitbake/lib/toaster/tests/browser/test_landing_page.py::TestLandingPage::test_icon_info_visible_and_clickable
FAILED ../bitbake/lib/toaster/tests/functional/test_functional_basic.py::FuntionalTestBasic::test_review_configuration_information

Comment 7 Mathieu Dubois-Briand 2024-10-14 06:17:05 UTC

toaster ubuntu2204-vk-1
E       selenium.common.exceptions.WebDriverException: Message: unknown error: unhandled inspector error: {"code":-32000,"message":"Node with given id does not belong to the document"}
https://valkyrie.yoctoproject.org/#/builders/88/builds/12/steps/11/logs/stdio

toaster ubuntu2204-vk-2
E       selenium.common.exceptions.WebDriverException: Message: unknown error: unhandled inspector error: {"code":-32000,"message":"Node with given id does not belong to the document"}
https://valkyrie.yoctoproject.org/#/builders/88/builds/14/steps/11/logs/stdio

toaster ubuntu2004-vk-3
E       AttributeError: 'str' object has no attribute 'removeprefix'
https://valkyrie.yoctoproject.org/#/builders/88/builds/25/steps/11/logs/stdio

Comment 8 Mathieu Dubois-Briand 2024-10-16 11:54:26 UTC

toaster ubuntu2410-vk-1
E       selenium.common.exceptions.WebDriverException: Message: unknown error: unhandled inspector error: {"code":-32000,"message":"Node with given id does not belong to the document"}
https://valkyrie.yoctoproject.org/#/builders/88/builds/32/steps/11/logs/stdio

Comment 9 Mathieu Dubois-Briand 2024-10-16 15:44:26 UTC

toaster ubuntu2204-vk-2
E       selenium.common.exceptions.WebDriverException: Message: unknown error:
unhandled inspector error: {"code":-32000,"message":"Node with given id does not
belong to the document"}
https://valkyrie.yoctoproject.org/#/builders/88/builds/8/steps/11/logs/stdio

toaster ubuntu2204-vk-2
E       AssertionError: False is not true : Allert message not shown
https://valkyrie.yoctoproject.org/#/builders/88/builds/9/steps/11/logs/stdio

toaster ubuntu2204-vk-2
E       selenium.common.exceptions.WebDriverException: Message: unknown error:
unhandled inspector error: {"code":-32000,"message":"Node with given id does not
belong to the document"}
https://valkyrie.yoctoproject.org/#/builders/88/builds/13/steps/11/logs/stdio

toaster ubuntu2204-vk-1
E       AssertionError: False is not true
https://valkyrie.yoctoproject.org/#/builders/88/builds/16/steps/11/logs/stdio

toaster ubuntu2204-vk-1
selenium.common.exceptions.TimeoutException: Message: An element matching
"#select-machine-form" should be visible
https://valkyrie.yoctoproject.org/#/builders/88/builds/18/steps/11/logs/stdio

toaster ubuntu2004-vk-3
E       AttributeError: 'str' object has no attribute 'removeprefix'
https://valkyrie.yoctoproject.org/#/builders/88/builds/22/steps/11/logs/stdio

toaster ubuntu2004-vk-3
E       selenium.common.exceptions.TimeoutException: Message: An element
matching "#new-project-name" should be visible
https://valkyrie.yoctoproject.org/#/builders/88/builds/24/steps/11/logs/stdio

toaster fedora38-ty-6
E       AssertionError: False is not true
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/438/steps/11/logs/stdio

toaster ubuntu2304-ty-1
E       AssertionError: False is not true
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/439/steps/11/logs/stdio

toaster opensuse154-ty-2
E       AssertionError: False is not true : Allert message not shown
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/440/steps/12/logs/stdio

toaster ubuntu2304-ty-1
E       selenium.common.exceptions.NoSuchElementException: Message: no such
element: Unable to locate element: {"method":"css
selector","selector":"#projectversion"}
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/441/steps/11/logs/stdio

toaster opensuse154-ty-1
E       AssertionError: False is not true : Allert message not shown 
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/442/steps/12/logs/stdio

toaster fedora38-ty-6
E       AssertionError: False is not true : Allert message not shown
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/447/steps/11/logs/stdio

toaster opensuse154-ty-1
E       selenium.common.exceptions.TimeoutException: Message: An element
matching "#hint-error-project-name" should be on the page
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/448/steps/12/logs/stdio

toaster opensuse154-ty-1
E       AssertionError: False is not true : Allert message not shown
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/449/steps/12/logs/stdio

toaster ubuntu2304-ty-1
E       AssertionError: False is not true : Allert message not shown
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/451/steps/11/logs/stdio

Comment 10 Mathieu Dubois-Briand 2024-10-17 14:23:47 UTC

toaster ubuntu2204-vk-3
E       selenium.common.exceptions.WebDriverException: Message: unknown error: unhandled inspector error: {"code":-32000,"message":"Node with given id does not belong to the document"}
https://valkyrie.yoctoproject.org/#/builders/88/builds/33/steps/11/logs/stdio

Comment 11 Mathieu Dubois-Briand 2024-10-21 07:52:36 UTC

toaster ubuntu2404-vk-2
E       selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable
https://valkyrie.yoctoproject.org/#/builders/88/builds/40/steps/11/logs/stdio

Comment 12 Mathieu Dubois-Briand 2024-10-23 08:45:14 UTC

toaster ubuntu2404-vk-2
E       selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable
https://valkyrie.yoctoproject.org/#/builders/88/builds/107/steps/11/logs/stdio

Comment 13 Richard Purdie 2024-10-24 10:26:19 UTC

I've merged the toaster test suite improvements:

https://git.yoctoproject.org/poky/commit/?id=0fac8e96f73dc3cd885f2590615e0e52aab42786

I'd like to open a new bug for any subsequent failures as they will likely be different to the ones in this bug and we need to track whatever is left separately.