It would be easier to consume buildstats if they were stored in some other format that per-task text files. I would propose using one (per-build) json-formatted file that would contain all tasks from all recipes. That would make buildstats a lot faster and easier to read. It would also be nicer on the filesystem level to have one file instead of thousands of them per build. Oe-build-perf-test script is already converting existing buildstats into json so we could use that format.
The format could look like this [ { "name": "bc-native", "epoch": null, "version": "1.06" "revision": "r3", "tasks": { "do_compile": { "child_rusage": { "ru_inblock": 0, "ru_majflt": 0, "ru_maxrss": 55212, "ru_minflt": 80968, "ru_nivcsw": 907, "ru_nvcsw": 320, "ru_oublock": 912, "ru_stime": 0.21015299999999998, "ru_utime": 1.9195280000000001 }, "elapsed_time": 2.120001, "iostat": { "cancelled_write_bytes": 4096, "rchar": 12604928, "read_bytes": 0, "syscr": 3553, "syscw": 707, "wchar": 885406, "write_bytes": 503808 }, "rusage": { "ru_inblock": 0, "ru_majflt": 0, "ru_maxrss": 58512, "ru_minflt": 4503, "ru_nivcsw": 19, "ru_nvcsw": 83, "ru_oublock": 72, "ru_stime": 0.011193, "ru_utime": 0.192331 }, "start_time": 1475630641.599999, "status": "PASSED" }, "do_configure": { ... }, ... }, { "name": "dbus-glib-native", ... }, ... ]
Moving to 2.4
One concern with single or small numbers of files is concurrency and locking. The existing multiple files were done individually to avoid locking issues.