| Summary: | Improve wayland QA | ||
|---|---|---|---|
| Product: | [Build System, Metadata & Runtime] OE-Core | Reporter: | Jussi Kukkonen <jku> |
| Component: | graphics | Assignee: | yeoh ee peng <ee.peng.yeoh> |
| Status: | RESOLVED FIXED | QA Contact: | |
| Severity: | enhancement | ||
| Priority: | Medium+ | CC: | meta.mr.watcher, meta.watcher, randy.macleod, ross.burton |
| Version: | 2.3 | ||
| Target Milestone: | 3.2 M1 | ||
| Hardware: | x86 | ||
| OS: | Multiple | ||
| Whiteboard: | |||
| OS type for building Yocto: | --- | Type of Regression: | --- |
| Verified: | Documentation change: | Don't know | |
I'm pushing this to 2.4 at this point: M3 freeze is next week. I have some experiments with both maynard and ivi-shell but nothing that's ready for merging, and more importantly failed on item #1 on the task list so far. I do intend to work on this during 2.3 M4 but the likely release is 2.4. Admitting failure, moving milestone forward, assigning to Ross for further action. I don't have any good suggestions for this one... Properly useful upstream projects don't seem to have materialized and I have no practical testing tasks to suggest that would fulfill the "test something that is closer to what real users of wayland-on-yocto are using" requirement. So could be it's time to close as wontfix? Rescoping 1) tell QA to verify it (pure weston, and x on weston) works 2) get autobuilder to build weston images Rescoping 1) tell QA to verify it (pure weston, and x on weston) works 2) get autobuilder to build weston images Sending to Ee Peng. The 'no-x11' build on the autobuilder does a core-image-weston build, but there's no QA that it even starts currently. *At least* we need to login and check that weston is running. QA team will first creating a manual test case for this core-image-weston basic test inside existing oeqa/manual. https://autobuilder.yoctoproject.org/typhoon/#/builders/40/builds/1773 RESULTS - weston.WestonTest.test_weston_running: PASSED (0.07s) Found that there are already testimage runtime weston automated test available. (In reply to comment #5) > Sending to Ee Peng. > > The 'no-x11' build on the autobuilder does a core-image-weston build, but > there's no QA that it even starts currently. *At least* we need to login > and check that weston is running. Hi Ross I found that Alex had enabled the weston automated test in testimage and it was running inside no-x11 build. https://autobuilder.yoctoproject.org/typhoon/#/builders/40/builds/1773 RESULTS - weston.WestonTest.test_weston_running: PASSED (0.07s) I hope that manual testing was not require anymore. What do you think? Is there any manual QA such as 'verify HD video playback is accelerated' in the Sato images? These should be replicated to the Weston images too. (In reply to comment #8) > Is there any manual QA such as 'verify HD video playback is accelerated' in > the Sato images? These should be replicated to the Weston images too. There were manual QA tests related to video and graphic for sato-sdk image: 1. Use media player inside sato-sdk to play video (bsps-hw.bsps-hw.media_player_-_play_video_(ogv)) 2. Run glxgears can run successfully (bsps-hw.bsps-hw.Graphics_-_ABAT) http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oeqa/manual/bsp-hw.json I believe weston image does not have media player. I could automated the glxgears runtime execution in testimage if that help testing weston. I would like to propose 2 more automated sanity tests for westons and avoid manual testing:
1.Test#01 : Check weston interfaces are available using “weston-info” executable, expected that weston-info return 0.
2.Test#02: Check that it was possible to launch a new weston-compositor and weston-desktop-shell, expected that a new process with weston-desktop-shell being created.
Thank you very much for your time and help! Please let me know you inputs and suggestions.
Thanks,
Ee Peng
Test #01: weston-info
root@qemux86-64:~# export XDG_RUNTIME_DIR=/run/user/0
root@qemux86-64:~# env
SHELL=/bin/sh
DISTCC_HOSTS=192.168.7.1
EDITOR=vi
PWD=/home/root
LOGNAME=root
HOME=/home/root
SSH_CONNECTION=10.0.2.2 57984 10.0.2.15 22
TERM=xterm
USER=root
SHLVL=1
XDG_RUNTIME_DIR=/run/user/0
PS1=\u@\h:\w\$
SSH_CLIENT=10.0.2.2 57984 22
PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
SSH_TTY=/dev/pts/0
_=/usr/bin/env
root@qemux86-64:~# weston-info
interface: 'wl_compositor', version: 4, name: 1
interface: 'wl_subcompositor', version: 1, name: 2
interface: 'wp_viewporter', version: 1, name: 3
interface: 'zxdg_output_manager_v1', version: 2, name: 4
xdg_output_v1
output: 13
name: 'Virtual-1'
logical_x: 0, logical_y: 0
logical_width: 1024, logical_height: 768
interface: 'wp_presentation', version: 1, name: 5
presentation clock id: 1 (CLOCK_MONOTONIC)
interface: 'zwp_relative_pointer_manager_v1', version: 1, name: 6
interface: 'zwp_pointer_constraints_v1', version: 1, name: 7
interface: 'zwp_input_timestamps_manager_v1', version: 1, name: 8
interface: 'wl_data_device_manager', version: 3, name: 9
interface: 'wl_shm', version: 1, name: 10
formats: 'YUYV'(0x56595559) 'NV12'(0x3231564e) 'YU12'(0x32315559) RGB565 XRGB8888 ARGB8888
interface: 'wl_seat', version: 7, name: 11
name: default
capabilities: pointer keyboard
keyboard repeat rate: 40
keyboard repeat delay: 400
interface: 'weston_content_protection', version: 1, name: 12
interface: 'wl_output', version: 3, name: 13
x: 0, y: 0, scale: 1,
physical_width: 400 mm, physical_height: 300 mm,
make: 'RHT', model: 'QEMU Monitor',
subpixel_orientation: unknown, output_transform: normal,
mode:
width: 1024 px, height: 768 px, refresh: 74.993 Hz,
flags: current preferred
mode:
width: 1920 px, height: 1080 px, refresh: 60.000 Hz,
flags:
mode:
width: 1920 px, height: 1080 px, refresh: 50.000 Hz,
flags:
mode:
width: 1600 px, height: 1200 px, refresh: 60.000 Hz,
flags:
mode:
width: 1680 px, height: 1050 px, refresh: 59.954 Hz,
flags:
mode:
width: 1400 px, height: 1050 px, refresh: 59.978 Hz,
flags:
mode:
width: 1280 px, height: 1024 px, refresh: 60.020 Hz,
flags:
mode:
width: 1440 px, height: 900 px, refresh: 59.887 Hz,
flags:
mode:
width: 1280 px, height: 960 px, refresh: 60.000 Hz,
flags:
mode:
width: 1360 px, height: 768 px, refresh: 60.015 Hz,
flags:
mode:
width: 1280 px, height: 768 px, refresh: 59.870 Hz,
flags:
mode:
width: 1024 px, height: 768 px, refresh: 60.004 Hz,
flags:
mode:
width: 800 px, height: 600 px, refresh: 60.317 Hz,
flags:
mode:
width: 640 px, height: 480 px, refresh: 60.000 Hz,
flags:
mode:
width: 640 px, height: 480 px, refresh: 59.940 Hz,
flags:
interface: 'zwp_input_panel_v1', version: 1, name: 14
interface: 'zwp_input_method_v1', version: 1, name: 15
interface: 'zwp_text_input_manager_v1', version: 1, name: 16
interface: 'xdg_wm_base', version: 1, name: 17
interface: 'zxdg_shell_v6', version: 1, name: 18
interface: 'wl_shell', version: 1, name: 19
interface: 'weston_desktop_shell', version: 1, name: 20
interface: 'weston_screenshooter', version: 1, name: 21
Test #02: Launch a new weston-desktop-shell
Before launch new weston
root@qemux86-64:~# ps | grep weston
266 root 3700 S weston-launch -- --modules=xwayland.so --log=/var/log/weston.log
271 root 258m S /usr/bin/weston --modules=xwayland.so --log=/var/log/weston.log
302 root 17232 S /usr/libexec/weston-keyboard
303 root 16724 S /usr/libexec/weston-desktop-shell
396 root 21488 S /usr/bin/weston-terminal
426 root 4160 S grep weston
After launch new weston
root@qemux86-64:~# ps | grep weston
266 root 3700 S weston-launch -- --modules=xwayland.so --log=/var/log/weston.log
271 root 266m R /usr/bin/weston --modules=xwayland.so --log=/var/log/weston.log
302 root 17232 S /usr/libexec/weston-keyboard
303 root 16724 S /usr/libexec/weston-desktop-shell
396 root 21488 S /usr/bin/weston-terminal
427 root 246m R weston
428 root 17232 S /usr/libexec/weston-keyboard
429 root 16704 S /usr/libexec/weston-desktop-shell
433 root 4160 S grep weston
root@qemux86-64:~# weston
Date: 2020-04-10 UTC
[12:01:19.022] weston 8.0.0
https://wayland.freedesktop.org
Bug reports to: https://gitlab.freedesktop.org/wayland/weston/issues/
Build: 3.1_M3-97-gfa66d6b
[12:01:19.031] Command line: weston
[12:01:19.033] OS: Linux, 5.4.27-yocto-standard, #1 SMP PREEMPT Tue Mar 24 19:10:14 UTC 2020, x86_64
[12:01:19.040] Using config file '/etc/xdg/weston/weston.ini'
[12:01:19.048] Output repaint window is 7 ms maximum.
[12:01:19.052] Loading module '/usr/lib/libweston-8/x11-backend.so'
[12:01:19.220] Loading module '/usr/lib/libweston-8/gl-renderer.so'
[12:01:19.318] EGL client extensions: EGL_EXT_client_extensions
EGL_EXT_device_base EGL_EXT_device_enumeration
EGL_EXT_device_query EGL_EXT_platform_base
EGL_KHR_client_get_all_proc_addresses EGL_KHR_debug
EGL_EXT_platform_wayland EGL_EXT_platform_x11
EGL_MESA_platform_gbm EGL_MESA_platform_surfaceless
EGL_EXT_platform_device
[12:01:19.468] EGL version: 1.4
[12:01:19.471] EGL vendor: Mesa Project
[12:01:19.474] EGL client APIs: OpenGL OpenGL_ES
[12:01:19.474] EGL extensions: EGL_KHR_cl_event2 EGL_KHR_config_attribs
EGL_KHR_context_flush_control EGL_KHR_create_context
EGL_KHR_create_context_no_error EGL_KHR_fence_sync
EGL_KHR_get_all_proc_addresses EGL_KHR_gl_colorspace
EGL_KHR_gl_renderbuffer_image EGL_KHR_gl_texture_2D_image
EGL_KHR_gl_texture_3D_image EGL_KHR_gl_texture_cubemap_image
EGL_KHR_image_base EGL_KHR_no_config_context
EGL_KHR_reusable_sync EGL_KHR_surfaceless_context
EGL_EXT_pixel_format_float EGL_KHR_wait_sync
EGL_MESA_configless_context EGL_MESA_drm_image
EGL_MESA_query_driver
[12:01:19.481] warning: Disabling render GPU timeline and explicit synchronization due to missing EGL_ANDROID_native_fence_sync extension
[12:01:19.483] EGL_KHR_surfaceless_context available
[12:01:20.460] GL version: OpenGL ES 3.1 Mesa 20.0.2
[12:01:20.462] GLSL version: OpenGL ES GLSL ES 3.10
[12:01:20.465] GL vendor: VMware, Inc.
[12:01:20.466] GL renderer: softpipe
[12:01:20.471] GL extensions: GL_EXT_blend_minmax GL_EXT_multi_draw_arrays
GL_EXT_texture_filter_anisotropic
GL_EXT_texture_compression_s3tc GL_EXT_texture_compression_dxt1
GL_EXT_texture_compression_rgtc GL_EXT_texture_format_BGRA8888
GL_OES_compressed_ETC1_RGB8_texture GL_OES_depth24
GL_OES_element_index_uint GL_OES_fbo_render_mipmap
GL_OES_mapbuffer GL_OES_rgb8_rgba8 GL_OES_standard_derivatives
GL_OES_stencil8 GL_OES_texture_3D GL_OES_texture_float
GL_OES_texture_float_linear GL_OES_texture_half_float
GL_OES_texture_half_float_linear GL_OES_texture_npot
GL_OES_vertex_half_float GL_EXT_texture_sRGB_decode
GL_OES_EGL_image GL_OES_depth_texture
GL_OES_packed_depth_stencil GL_EXT_texture_type_2_10_10_10_REV
GL_NV_conditional_render GL_OES_get_program_binary
GL_APPLE_texture_max_level GL_EXT_discard_framebuffer
GL_EXT_read_format_bgra GL_EXT_frag_depth
GL_NV_fbo_color_attachments GL_OES_EGL_image_external
GL_OES_EGL_sync GL_OES_vertex_array_object
GL_OES_viewport_array GL_ANGLE_texture_compression_dxt3
GL_ANGLE_texture_compression_dxt5
GL_EXT_occlusion_query_boolean GL_EXT_texture_rg
GL_EXT_unpack_subimage GL_NV_draw_buffers GL_NV_read_buffer
GL_NV_read_depth GL_NV_read_depth_stencil GL_NV_read_stencil
GL_EXT_draw_buffers GL_EXT_map_buffer_range GL_KHR_debug
GL_KHR_texture_compression_astc_ldr
GL_OES_depth_texture_cube_map GL_OES_required_internalformat
GL_OES_surfaceless_context GL_EXT_color_buffer_float
GL_EXT_sRGB_write_control GL_EXT_separate_shader_objects
GL_EXT_shader_implicit_conversions GL_EXT_shader_integer_mix
GL_EXT_base_instance GL_EXT_compressed_ETC1_RGB8_sub_texture
GL_EXT_copy_image GL_EXT_draw_buffers_indexed
GL_EXT_draw_elements_base_vertex GL_EXT_gpu_shader5
GL_EXT_primitive_bounding_box GL_EXT_render_snorm
GL_EXT_shader_io_blocks GL_EXT_texture_border_clamp
GL_EXT_texture_buffer GL_EXT_texture_cube_map_array
GL_EXT_texture_norm16 GL_EXT_texture_view
GL_KHR_context_flush_control GL_NV_image_formats
GL_OES_copy_image GL_OES_draw_buffers_indexed
GL_OES_draw_elements_base_vertex GL_OES_gpu_shader5
GL_OES_primitive_bounding_box GL_OES_shader_io_blocks
GL_OES_texture_border_clamp GL_OES_texture_buffer
GL_OES_texture_cube_map_array GL_OES_texture_stencil8
GL_OES_texture_storage_multisample_2d_array GL_OES_texture_view
GL_EXT_blend_func_extended GL_EXT_float_blend
GL_EXT_geometry_point_size GL_EXT_geometry_shader
GL_KHR_no_error GL_KHR_texture_compression_astc_sliced_3d
GL_OES_EGL_image_external_essl3 GL_OES_geometry_point_size
GL_OES_geometry_shader GL_OES_shader_image_atomic
GL_EXT_clip_cull_distance GL_EXT_disjoint_timer_query
GL_EXT_texture_compression_s3tc_srgb
GL_MESA_shader_integer_functions GL_EXT_clip_control
GL_EXT_texture_compression_bptc GL_KHR_parallel_shader_compile
GL_EXT_EGL_image_storage GL_EXT_texture_sRGB_R8
GL_MESA_framebuffer_flip_y GL_EXT_depth_clamp
GL_EXT_texture_query_lod
[12:01:20.503] GL ES 2 renderer features:
read-back format: BGRA
wl_shm sub-image to texture: yes
EGL Wayland extension: no
[12:01:20.506] Using gl renderer
[12:01:20.616] Registered plugin API 'weston_windowed_output_api_v1' of size 16
[12:01:20.649] Chosen EGL config details: id: 6 rgba: 8 8 8 0 buf: 24 dep: 0 stcl: 0 int: 1-1 type: win|pix|pbf|swap_preserved vis_id: 0x23
[12:01:20.657] x11 output 1024x600, window id 4194309
[12:01:20.661] Output 'screen0' enabled with head(s) screen0
[12:01:20.666] Compositor capabilities:
arbitrary surface rotation: yes
screen capture uses y-flip: yes
presentation clock: CLOCK_MONOTONIC_RAW, id 4
presentation clock resolution: 0.000000001 s
[12:01:20.676] libwayland: unable to lock lockfile /run/user/0/wayland-0.lock, maybe another compositor is running
[12:01:20.691] Loading module '/usr/lib/weston/desktop-shell.so'
[12:01:20.720] launching '/usr/libexec/weston-keyboard'
[12:01:20.776] launching '/usr/libexec/weston-desktop-shell'
could not load cursor 'dnd-move'
could not load cursor 'dnd-copy'
could not load cursor 'dnd-move'
could not load cursor 'dnd-copy'
could not load cursor 'dnd-none'
could not load cursor 'dnd-none'
[12:01:44.890] unexpectedly large timestamp jump (from 1096685 to 1104482)
Patch sent to enable the proposed two new weston automated runtime tests. https://lists.openembedded.org/g/openembedded-core/topic/patch_oeqa_runtime_weston/72982654?p=,,,20,0,0,0::recentpostdate%2Fsticky,,,20,2,0,72982654 That's a good start, but the media player in the Sato images should be available in Weston images if it is build with the testapps image feature enabled. |
We have wayland-related recipes and images but there are some short comings: * automated QA: autobuilder does not test wayland * manual QA: We provide weston (with desktop-shell) but that's a little spartan: there's no way to start applications (apart from terminal) or configure things like a wireless connection * Testing relevancy: weston desktop-shell is essentially a validation device for wayland project. If we can test something that is closer to what real users of wayland-on-yocto are using, that would be great Tasks: 1. Discuss with wayland-on-yocto users, figure out what would be highest impact for them 2. Consider including/enabling additional components: - The ivi-shell in weston might be more relevant to yocto users than the desktop-shell is - Maynard would be a great Sato-like test shell, unfortunately it's not very actively maintained : https://github.com/raspberrypi/maynard/ - It might be possible to use matchbox-desktop on weston desktop-shell to achieve a roughly sato-like QA platform. Some assembly required though. 3. Based on 1 & 2, look at potential QA improvements.