We have wayland-related recipes and images but there are some short comings: * automated QA: autobuilder does not test wayland * manual QA: We provide weston (with desktop-shell) but that's a little spartan: there's no way to start applications (apart from terminal) or configure things like a wireless connection * Testing relevancy: weston desktop-shell is essentially a validation device for wayland project. If we can test something that is closer to what real users of wayland-on-yocto are using, that would be great Tasks: 1. Discuss with wayland-on-yocto users, figure out what would be highest impact for them 2. Consider including/enabling additional components: - The ivi-shell in weston might be more relevant to yocto users than the desktop-shell is - Maynard would be a great Sato-like test shell, unfortunately it's not very actively maintained : https://github.com/raspberrypi/maynard/ - It might be possible to use matchbox-desktop on weston desktop-shell to achieve a roughly sato-like QA platform. Some assembly required though. 3. Based on 1 & 2, look at potential QA improvements.
I'm pushing this to 2.4 at this point: M3 freeze is next week. I have some experiments with both maynard and ivi-shell but nothing that's ready for merging, and more importantly failed on item #1 on the task list so far. I do intend to work on this during 2.3 M4 but the likely release is 2.4.
Admitting failure, moving milestone forward, assigning to Ross for further action. I don't have any good suggestions for this one... Properly useful upstream projects don't seem to have materialized and I have no practical testing tasks to suggest that would fulfill the "test something that is closer to what real users of wayland-on-yocto are using" requirement. So could be it's time to close as wontfix?
Rescoping 1) tell QA to verify it (pure weston, and x on weston) works 2) get autobuilder to build weston images
Sending to Ee Peng. The 'no-x11' build on the autobuilder does a core-image-weston build, but there's no QA that it even starts currently. *At least* we need to login and check that weston is running.
QA team will first creating a manual test case for this core-image-weston basic test inside existing oeqa/manual.
https://autobuilder.yoctoproject.org/typhoon/#/builders/40/builds/1773 RESULTS - weston.WestonTest.test_weston_running: PASSED (0.07s) Found that there are already testimage runtime weston automated test available. (In reply to comment #5) > Sending to Ee Peng. > > The 'no-x11' build on the autobuilder does a core-image-weston build, but > there's no QA that it even starts currently. *At least* we need to login > and check that weston is running. Hi Ross I found that Alex had enabled the weston automated test in testimage and it was running inside no-x11 build. https://autobuilder.yoctoproject.org/typhoon/#/builders/40/builds/1773 RESULTS - weston.WestonTest.test_weston_running: PASSED (0.07s) I hope that manual testing was not require anymore. What do you think?
Is there any manual QA such as 'verify HD video playback is accelerated' in the Sato images? These should be replicated to the Weston images too.
(In reply to comment #8) > Is there any manual QA such as 'verify HD video playback is accelerated' in > the Sato images? These should be replicated to the Weston images too. There were manual QA tests related to video and graphic for sato-sdk image: 1. Use media player inside sato-sdk to play video (bsps-hw.bsps-hw.media_player_-_play_video_(ogv)) 2. Run glxgears can run successfully (bsps-hw.bsps-hw.Graphics_-_ABAT) http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oeqa/manual/bsp-hw.json I believe weston image does not have media player. I could automated the glxgears runtime execution in testimage if that help testing weston.
I would like to propose 2 more automated sanity tests for westons and avoid manual testing: 1.Test#01 : Check weston interfaces are available using “weston-info” executable, expected that weston-info return 0. 2.Test#02: Check that it was possible to launch a new weston-compositor and weston-desktop-shell, expected that a new process with weston-desktop-shell being created. Thank you very much for your time and help! Please let me know you inputs and suggestions. Thanks, Ee Peng Test #01: weston-info root@qemux86-64:~# export XDG_RUNTIME_DIR=/run/user/0 root@qemux86-64:~# env SHELL=/bin/sh DISTCC_HOSTS=192.168.7.1 EDITOR=vi PWD=/home/root LOGNAME=root HOME=/home/root SSH_CONNECTION=10.0.2.2 57984 10.0.2.15 22 TERM=xterm USER=root SHLVL=1 XDG_RUNTIME_DIR=/run/user/0 PS1=\u@\h:\w\$ SSH_CLIENT=10.0.2.2 57984 22 PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin SSH_TTY=/dev/pts/0 _=/usr/bin/env root@qemux86-64:~# weston-info interface: 'wl_compositor', version: 4, name: 1 interface: 'wl_subcompositor', version: 1, name: 2 interface: 'wp_viewporter', version: 1, name: 3 interface: 'zxdg_output_manager_v1', version: 2, name: 4 xdg_output_v1 output: 13 name: 'Virtual-1' logical_x: 0, logical_y: 0 logical_width: 1024, logical_height: 768 interface: 'wp_presentation', version: 1, name: 5 presentation clock id: 1 (CLOCK_MONOTONIC) interface: 'zwp_relative_pointer_manager_v1', version: 1, name: 6 interface: 'zwp_pointer_constraints_v1', version: 1, name: 7 interface: 'zwp_input_timestamps_manager_v1', version: 1, name: 8 interface: 'wl_data_device_manager', version: 3, name: 9 interface: 'wl_shm', version: 1, name: 10 formats: 'YUYV'(0x56595559) 'NV12'(0x3231564e) 'YU12'(0x32315559) RGB565 XRGB8888 ARGB8888 interface: 'wl_seat', version: 7, name: 11 name: default capabilities: pointer keyboard keyboard repeat rate: 40 keyboard repeat delay: 400 interface: 'weston_content_protection', version: 1, name: 12 interface: 'wl_output', version: 3, name: 13 x: 0, y: 0, scale: 1, physical_width: 400 mm, physical_height: 300 mm, make: 'RHT', model: 'QEMU Monitor', subpixel_orientation: unknown, output_transform: normal, mode: width: 1024 px, height: 768 px, refresh: 74.993 Hz, flags: current preferred mode: width: 1920 px, height: 1080 px, refresh: 60.000 Hz, flags: mode: width: 1920 px, height: 1080 px, refresh: 50.000 Hz, flags: mode: width: 1600 px, height: 1200 px, refresh: 60.000 Hz, flags: mode: width: 1680 px, height: 1050 px, refresh: 59.954 Hz, flags: mode: width: 1400 px, height: 1050 px, refresh: 59.978 Hz, flags: mode: width: 1280 px, height: 1024 px, refresh: 60.020 Hz, flags: mode: width: 1440 px, height: 900 px, refresh: 59.887 Hz, flags: mode: width: 1280 px, height: 960 px, refresh: 60.000 Hz, flags: mode: width: 1360 px, height: 768 px, refresh: 60.015 Hz, flags: mode: width: 1280 px, height: 768 px, refresh: 59.870 Hz, flags: mode: width: 1024 px, height: 768 px, refresh: 60.004 Hz, flags: mode: width: 800 px, height: 600 px, refresh: 60.317 Hz, flags: mode: width: 640 px, height: 480 px, refresh: 60.000 Hz, flags: mode: width: 640 px, height: 480 px, refresh: 59.940 Hz, flags: interface: 'zwp_input_panel_v1', version: 1, name: 14 interface: 'zwp_input_method_v1', version: 1, name: 15 interface: 'zwp_text_input_manager_v1', version: 1, name: 16 interface: 'xdg_wm_base', version: 1, name: 17 interface: 'zxdg_shell_v6', version: 1, name: 18 interface: 'wl_shell', version: 1, name: 19 interface: 'weston_desktop_shell', version: 1, name: 20 interface: 'weston_screenshooter', version: 1, name: 21 Test #02: Launch a new weston-desktop-shell Before launch new weston root@qemux86-64:~# ps | grep weston 266 root 3700 S weston-launch -- --modules=xwayland.so --log=/var/log/weston.log 271 root 258m S /usr/bin/weston --modules=xwayland.so --log=/var/log/weston.log 302 root 17232 S /usr/libexec/weston-keyboard 303 root 16724 S /usr/libexec/weston-desktop-shell 396 root 21488 S /usr/bin/weston-terminal 426 root 4160 S grep weston After launch new weston root@qemux86-64:~# ps | grep weston 266 root 3700 S weston-launch -- --modules=xwayland.so --log=/var/log/weston.log 271 root 266m R /usr/bin/weston --modules=xwayland.so --log=/var/log/weston.log 302 root 17232 S /usr/libexec/weston-keyboard 303 root 16724 S /usr/libexec/weston-desktop-shell 396 root 21488 S /usr/bin/weston-terminal 427 root 246m R weston 428 root 17232 S /usr/libexec/weston-keyboard 429 root 16704 S /usr/libexec/weston-desktop-shell 433 root 4160 S grep weston root@qemux86-64:~# weston Date: 2020-04-10 UTC [12:01:19.022] weston 8.0.0 https://wayland.freedesktop.org Bug reports to: https://gitlab.freedesktop.org/wayland/weston/issues/ Build: 3.1_M3-97-gfa66d6b [12:01:19.031] Command line: weston [12:01:19.033] OS: Linux, 5.4.27-yocto-standard, #1 SMP PREEMPT Tue Mar 24 19:10:14 UTC 2020, x86_64 [12:01:19.040] Using config file '/etc/xdg/weston/weston.ini' [12:01:19.048] Output repaint window is 7 ms maximum. [12:01:19.052] Loading module '/usr/lib/libweston-8/x11-backend.so' [12:01:19.220] Loading module '/usr/lib/libweston-8/gl-renderer.so' [12:01:19.318] EGL client extensions: EGL_EXT_client_extensions EGL_EXT_device_base EGL_EXT_device_enumeration EGL_EXT_device_query EGL_EXT_platform_base EGL_KHR_client_get_all_proc_addresses EGL_KHR_debug EGL_EXT_platform_wayland EGL_EXT_platform_x11 EGL_MESA_platform_gbm EGL_MESA_platform_surfaceless EGL_EXT_platform_device [12:01:19.468] EGL version: 1.4 [12:01:19.471] EGL vendor: Mesa Project [12:01:19.474] EGL client APIs: OpenGL OpenGL_ES [12:01:19.474] EGL extensions: EGL_KHR_cl_event2 EGL_KHR_config_attribs EGL_KHR_context_flush_control EGL_KHR_create_context EGL_KHR_create_context_no_error EGL_KHR_fence_sync EGL_KHR_get_all_proc_addresses EGL_KHR_gl_colorspace EGL_KHR_gl_renderbuffer_image EGL_KHR_gl_texture_2D_image EGL_KHR_gl_texture_3D_image EGL_KHR_gl_texture_cubemap_image EGL_KHR_image_base EGL_KHR_no_config_context EGL_KHR_reusable_sync EGL_KHR_surfaceless_context EGL_EXT_pixel_format_float EGL_KHR_wait_sync EGL_MESA_configless_context EGL_MESA_drm_image EGL_MESA_query_driver [12:01:19.481] warning: Disabling render GPU timeline and explicit synchronization due to missing EGL_ANDROID_native_fence_sync extension [12:01:19.483] EGL_KHR_surfaceless_context available [12:01:20.460] GL version: OpenGL ES 3.1 Mesa 20.0.2 [12:01:20.462] GLSL version: OpenGL ES GLSL ES 3.10 [12:01:20.465] GL vendor: VMware, Inc. [12:01:20.466] GL renderer: softpipe [12:01:20.471] GL extensions: GL_EXT_blend_minmax GL_EXT_multi_draw_arrays GL_EXT_texture_filter_anisotropic GL_EXT_texture_compression_s3tc GL_EXT_texture_compression_dxt1 GL_EXT_texture_compression_rgtc GL_EXT_texture_format_BGRA8888 GL_OES_compressed_ETC1_RGB8_texture GL_OES_depth24 GL_OES_element_index_uint GL_OES_fbo_render_mipmap GL_OES_mapbuffer GL_OES_rgb8_rgba8 GL_OES_standard_derivatives GL_OES_stencil8 GL_OES_texture_3D GL_OES_texture_float GL_OES_texture_float_linear GL_OES_texture_half_float GL_OES_texture_half_float_linear GL_OES_texture_npot GL_OES_vertex_half_float GL_EXT_texture_sRGB_decode GL_OES_EGL_image GL_OES_depth_texture GL_OES_packed_depth_stencil GL_EXT_texture_type_2_10_10_10_REV GL_NV_conditional_render GL_OES_get_program_binary GL_APPLE_texture_max_level GL_EXT_discard_framebuffer GL_EXT_read_format_bgra GL_EXT_frag_depth GL_NV_fbo_color_attachments GL_OES_EGL_image_external GL_OES_EGL_sync GL_OES_vertex_array_object GL_OES_viewport_array GL_ANGLE_texture_compression_dxt3 GL_ANGLE_texture_compression_dxt5 GL_EXT_occlusion_query_boolean GL_EXT_texture_rg GL_EXT_unpack_subimage GL_NV_draw_buffers GL_NV_read_buffer GL_NV_read_depth GL_NV_read_depth_stencil GL_NV_read_stencil GL_EXT_draw_buffers GL_EXT_map_buffer_range GL_KHR_debug GL_KHR_texture_compression_astc_ldr GL_OES_depth_texture_cube_map GL_OES_required_internalformat GL_OES_surfaceless_context GL_EXT_color_buffer_float GL_EXT_sRGB_write_control GL_EXT_separate_shader_objects GL_EXT_shader_implicit_conversions GL_EXT_shader_integer_mix GL_EXT_base_instance GL_EXT_compressed_ETC1_RGB8_sub_texture GL_EXT_copy_image GL_EXT_draw_buffers_indexed GL_EXT_draw_elements_base_vertex GL_EXT_gpu_shader5 GL_EXT_primitive_bounding_box GL_EXT_render_snorm GL_EXT_shader_io_blocks GL_EXT_texture_border_clamp GL_EXT_texture_buffer GL_EXT_texture_cube_map_array GL_EXT_texture_norm16 GL_EXT_texture_view GL_KHR_context_flush_control GL_NV_image_formats GL_OES_copy_image GL_OES_draw_buffers_indexed GL_OES_draw_elements_base_vertex GL_OES_gpu_shader5 GL_OES_primitive_bounding_box GL_OES_shader_io_blocks GL_OES_texture_border_clamp GL_OES_texture_buffer GL_OES_texture_cube_map_array GL_OES_texture_stencil8 GL_OES_texture_storage_multisample_2d_array GL_OES_texture_view GL_EXT_blend_func_extended GL_EXT_float_blend GL_EXT_geometry_point_size GL_EXT_geometry_shader GL_KHR_no_error GL_KHR_texture_compression_astc_sliced_3d GL_OES_EGL_image_external_essl3 GL_OES_geometry_point_size GL_OES_geometry_shader GL_OES_shader_image_atomic GL_EXT_clip_cull_distance GL_EXT_disjoint_timer_query GL_EXT_texture_compression_s3tc_srgb GL_MESA_shader_integer_functions GL_EXT_clip_control GL_EXT_texture_compression_bptc GL_KHR_parallel_shader_compile GL_EXT_EGL_image_storage GL_EXT_texture_sRGB_R8 GL_MESA_framebuffer_flip_y GL_EXT_depth_clamp GL_EXT_texture_query_lod [12:01:20.503] GL ES 2 renderer features: read-back format: BGRA wl_shm sub-image to texture: yes EGL Wayland extension: no [12:01:20.506] Using gl renderer [12:01:20.616] Registered plugin API 'weston_windowed_output_api_v1' of size 16 [12:01:20.649] Chosen EGL config details: id: 6 rgba: 8 8 8 0 buf: 24 dep: 0 stcl: 0 int: 1-1 type: win|pix|pbf|swap_preserved vis_id: 0x23 [12:01:20.657] x11 output 1024x600, window id 4194309 [12:01:20.661] Output 'screen0' enabled with head(s) screen0 [12:01:20.666] Compositor capabilities: arbitrary surface rotation: yes screen capture uses y-flip: yes presentation clock: CLOCK_MONOTONIC_RAW, id 4 presentation clock resolution: 0.000000001 s [12:01:20.676] libwayland: unable to lock lockfile /run/user/0/wayland-0.lock, maybe another compositor is running [12:01:20.691] Loading module '/usr/lib/weston/desktop-shell.so' [12:01:20.720] launching '/usr/libexec/weston-keyboard' [12:01:20.776] launching '/usr/libexec/weston-desktop-shell' could not load cursor 'dnd-move' could not load cursor 'dnd-copy' could not load cursor 'dnd-move' could not load cursor 'dnd-copy' could not load cursor 'dnd-none' could not load cursor 'dnd-none' [12:01:44.890] unexpectedly large timestamp jump (from 1096685 to 1104482)
Patch sent to enable the proposed two new weston automated runtime tests. https://lists.openembedded.org/g/openembedded-core/topic/patch_oeqa_runtime_weston/72982654?p=,,,20,0,0,0::recentpostdate%2Fsticky,,,20,2,0,72982654
That's a good start, but the media player in the Sato images should be available in Weston images if it is build with the testapps image feature enabled.