Bug 9299

Summary: Enable Tracing boot and process times in testing images
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Aníbal Limón <anibal.limon>
Component: coreAssignee: Aníbal Limón <anibal.limon>
Status: RESOLVED FIXED QA Contact: Francisco Pedraza <francisco.j.pedraza.gonzalez>
Severity: enhancement    
Priority: Medium+ CC: leonardo.sandoval.gonzalez, meta.mr.watcher, meta.watcher, richard.purdie, ross.burton
Version: 2.1   
Target Milestone: 2.2 M2   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)
Bug Depends on:    
Bug Blocks: 8142    

Description Aníbal Limón 2016-03-17 23:20:49 UTC
At this time we are experimenting some timeout issues in systemd/avahi, etc.

It will be good enable time tracing on testing images (core-image-) in order to provide better information for debugging purposes.

For make it possible changes needed,

- Qemu: Change clock to use host/rt one for provide a suitable clock for get times.
- Linux kernel: Evaluate the current tools like OProfile/LTTng for tracing process events.

The output of this task needs to be a log that testimage task could retrive and store for analysis. The performance needs to be evaluated to avoid slowly testing times.
Comment 1 Leonardo Sandoval Gonzalez 2016-04-28 17:02:09 UTC
Anibal,

The following script may be useful for this task:

http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/scripts/pybootchartgui
Comment 2 Aníbal Limón 2016-07-08 17:36:03 UTC
Patch for enable printk times on qemu machines sent to the ML,

http://lists.openembedded.org/pipermail/openembedded-core/2016-July/123664.html

I'll continue to review systemd and sysvinit.
Comment 3 Aníbal Limón 2016-07-12 21:30:38 UTC
After review the systemd and sysvinit options, i decide only to enable printk times as default on qemu machines and enable systemd log debug option on testimage it dumps to the qemuboot log.

The systemd log debug option provides all the information related to systemd boot process for debug.

Also when i was working on enable debug in systemd, i noticed if you use journal output for logs (disk I/O) it reaches the timeout see, [1]

Patches in review at, 

http://lists.openembedded.org/pipermail/openembedded-core/2016-July/123803.html
http://lists.openembedded.org/pipermail/openembedded-core/2016-July/123804.html
http://lists.openembedded.org/pipermail/openembedded-core/2016-July/123805.html

[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=8142#c19
Comment 4 Aníbal Limón 2016-07-12 22:01:03 UTC
I forget to mention that we already have a systemd test for boot times [1], Benjamin wrote it using journalctl.

[1] http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oeqa/runtime/systemd.py#n132
Comment 5 Ross Burton 2016-07-20 13:49:20 UTC
OE-Core dfa8be08aa6e7ad6dbc615df8875dc2ac1ac8946 added the debug/printk kernel feature, but enabling logging to console breaks the sanity tests as the journal isn't used anymore.
Comment 6 Aníbal Limón 2016-07-25 22:03:42 UTC
(In reply to comment #5)
> OE-Core dfa8be08aa6e7ad6dbc615df8875dc2ac1ac8946 added the debug/printk
> kernel feature, but enabling logging to console breaks the sanity tests as
> the journal isn't used anymore.

The main reason for this is because busybox-syslog isn't reciving journalctl forwarded syslog messages, the people from busybox remove the support of systemd [1].

I propose to get disabled of busybox-syslog on systemd images and use syslog-ng or make journalctl to write /var/log/messages?

What you think ross?

    alimon

[1] https://git.busybox.net/busybox/commit/?id=accd9eeb719916da974584b33b1aeced5f3bb346
Comment 7 Aníbal Limón 2016-07-25 22:18:35 UTC
(In reply to comment #6)
> (In reply to comment #5)
> > OE-Core dfa8be08aa6e7ad6dbc615df8875dc2ac1ac8946 added the debug/printk
> > kernel feature, but enabling logging to console breaks the sanity tests as
> > the journal isn't used anymore.
> 
> The main reason for this is because busybox-syslog isn't reciving journalctl
> forwarded syslog messages, the people from busybox remove the support of
> systemd [1].
> 
> I propose to get disabled of busybox-syslog on systemd images and use
> syslog-ng or make journalctl to write /var/log/messages?

Another easy option is to drop read /var/log/messages and use journalctl directly to found the message.

> 
> What you think ross?
> 
>     alimon
> 
> [1]
> https://git.busybox.net/busybox/commit/
> ?id=accd9eeb719916da974584b33b1aeced5f3bb346
Comment 8 Aníbal Limón 2016-07-27 22:42:40 UTC
Another set of patches for fix the issue with systemd and busybox syslog,

http://lists.openembedded.org/pipermail/openembedded-core/2016-July/124421.html
Comment 9 Aníbal Limón 2016-08-01 15:17:00 UTC
Remove dependency of #4528 because that bug is for get statistics about boot time [1].

[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=4528#c8