Bug 5300 - Modify buildstats implementation to use the bb.build.Task* timestamp
Summary: Modify buildstats implementation to use the bb.build.Task* timestamp
Status: VERIFIED FIXED
Alias: None
Product: Toaster
Classification: Build System, Metadata & Runtime
Component: toaster (show other bugs)
Version: unspecified
Hardware: All Multiple
: Medium normal
Target Milestone: 1.6 M5
Assignee: Marius Avram
QA Contact:
URL:
Whiteboard:
Depends on: 5485
Blocks:
  Show dependency tree
 
Reported: 2013-10-01 14:10 UTC by Belen Barros Pena
Modified: 2014-04-08 08:06 UTC (History)
6 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)

Attachments
sysvinit-inittab.do_package info from build stats (638 bytes, application/octet-stream)
2013-10-01 14:10 UTC, Belen Barros Pena 		no flags Details
sysvinit-inittab.do_package info from web hob (11.86 KB, image/png)
2013-10-01 14:10 UTC, Belen Barros Pena 		no flags Details
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Belen Barros Pena 2013-10-01 14:10:00 UTC 
Created attachment 1537 [details]
sysvinit-inittab.do_package info from build stats

We are now collecting tasks cpu usage and disk i/o from build stats. However, the task time reported by Web Hob doesn't seem to match the build stats information (see attached files).

How is the task time being obtained and why is it different from the information reported by build stats?

webhob-master-1_5-2609
Comment 1 Belen Barros Pena 2013-10-01 14:10:44 UTC 
Created attachment 1538 [details]
sysvinit-inittab.do_package info from web hob
Comment 2 Alexandru Damian 2013-10-01 15:54:07 UTC 
Tentative patch for both 5300 and 5299 in testing.

The root cause for both bugs seems that the native- and normal tasks had a naming collision in the state tracking structure, and the tasks may got overwritten.
Comment 3 Alexandru Damian 2013-10-01 15:56:38 UTC 
Fix available on adamian/webhob-minimal-1_5-3
Comment 4 Belen Barros Pena 2013-10-03 10:17:43 UTC 
Still not working for me in 

adamian/webhob-minimal-1_5-3

do_package.eglibc-initial

elapsed time in web hob: 0.018142
elapsed time in build stats: 0.57
Comment 5 Alexandru Damian 2013-10-03 15:18:27 UTC 
IMHO, this happens because on buildstats.class takes the completion time after creating the buildstats directory hierarchy and opening the destination file, which can take a significant amount of time on a loaded system (see write_task_data()).

webhob takes the completion time as soon as the event is received, so it's more accurate.
Comment 6 Belen Barros Pena 2013-10-11 10:11:21 UTC 
If this means that web hob times are the correct ones, should we do something about build stats?

I am concerned about our different tools reporting different information: it makes the lot look unreliable. But I am not sure if build stats can be made to work in any other way.
Comment 7 Alexandru Damian 2013-10-21 10:17:50 UTC 
Tentatively assigned to Beth, should we want to change the buildstats info.
Comment 8 Alexandru Damian 2013-12-30 12:38:20 UTC 
Moved to new, Beth, can you please let us know when you'll take a look at this ?
Comment 9 Marius Avram 2014-02-19 10:04:20 UTC 
This bug is very similar to:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=5485 , which is in process of solving.
Comment 10 Alexandru Damian 2014-02-19 18:46:29 UTC 
We are adding time-stamps to the bb.build.Task* events. This will allow time-based measurement of task duration synchronously during task execution, thereby eliminating measurement errors based on time of processing events.

Marius, can you please modify the buildstats code as well ?
Comment 11 Marius Avram 2014-03-25 13:27:32 UTC 
Solution merged to master:poky, commit: ba6caa021fea2edc5483bcde6d71122f2a2a871a
Comment 12 Yuan Sun 2014-04-08 08:06:23 UTC 
Verified in 3e528d3e4bc542ca2f654c54363c56af0fb9f61b.