Bug 11418

Summary: Improve gstreamer testing
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Jussi Kukkonen <jku>
Component: multimediaAssignee: yeoh ee peng <ee.peng.yeoh>
Status: RESOLVED FIXED QA Contact:
Severity: enhancement    
Priority: Medium+ CC: alex.kanavin, apoorv.sangal, meta.mr.watcher, meta.watcher, robin.l.jordan, ross.burton, sgw
Version: 2.3   
Target Milestone: 4.99   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Jussi Kukkonen 2017-04-26 17:08:59 UTC 
We should not be finding GStreamer issues right around release time but that's what we seem to do. Let's see if there are low hanging fruit in automated testing that would allow us to catch at least some issues as they are merged.

Some potential tooling:
https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-validate/html/

I believe for core gstreamer there are test suites already, we'd just want to choose what to run and when (and then keep the expected results updated). 

Doing automated testing with e.g. gstreamer-vaapi would be best but testing hardware acceleration on VM can be a little tricky... Even then, a script that QA or developers could run that gave gst-validate like output would be useful.
Comment 1 Jussi Kukkonen 2017-04-28 07:51:36 UTC 
Forgot the repo links:

git://anongit.freedesktop.org/gstreamer/gst-devtools
git://anongit.freedesktop.org/gstreamer/gst-integration-testsuites
Comment 2 Maxin B. John 2018-02-21 07:57:47 UTC 
Moving gstreamer bugs to Anuj.
Comment 3 Anuj Mittal 2018-03-01 04:29:15 UTC 
Ee Peng, can you please evaluate the current tests that we have and see if we can use test-suites available here for runtime testing?

I think that we might not be able to test vaapi codecs for the bsp testing using this without changing some of the code to have default codec changed. gstreamer-vaapi does provides some tests which can be built & enabled but they need manual inspection, there's no image comparison to enable automated testing.
Comment 4 yeoh ee peng 2018-11-08 06:34:35 UTC 
We developed some gstreamer CLI tests back in the Intel Ref Kit. Need to try porting this tests to poky. 
https://github.com/intel/intel-iot-refkit/blob/master/meta-iotqa/lib/oeqa/runtime/multimedia/gstreamer/gstreamer_cli.py
Comment 5 yeoh ee peng 2018-11-08 08:19:19 UTC 
Porting tests from Intel Ref Kit. 
http://lists.openembedded.org/pipermail/openembedded-core/2018-November/157437.html