8411 2015-09-28 18:58:29 +0000 Build name and log path don't match real log file path 2016-01-25 12:37:59 +0000 1 1 7 Build System, Metadata & Runtime BitBake bitbake 2.0 x86 Multiple VERIFIED FIXED Medium+ enhancement 2.1 M1 8373 1 elliot.smith richard.purdie alexandru.c.georgescu poky.bs.watcher poky.watcher costin.c.constantin --- --- Don't know oldest_to_newest 54756 0 elliot.smith 2015-09-28 18:58:29 +0000 This is a suggestion for improving the clarity of how builds are named, and how those names correspond to log files. The name for a build (as reported in the BuildStarted event) and the corresponding log file location (as retrieved via the BB_CONSOLELOG bitbake variable at the time when BuildStarted occurs) are based on the same timestamp. For example, for two consecutive builds, I got the following name/path pairs: build name (from BuildStarted event) | log file (BB_CONSOLELOG when BuildStarted was fired) 20150928123720 | /home/ell/toaster-run/build/tmp/log/cooker/qemux86/20150928123720.log 20150928123726 | /home/ell/toaster-run/build/tmp/log/cooker/qemux86/20150928123726.log However, the real log locations for those two builds had the following paths: /home/ell/toaster-run/build/tmp/log/cooker/qemux86/20150928123716.log /home/ell/toaster-run/build/tmp/log/cooker/qemux86/20150928123720.log (Note that the second path actually corresponds to the path which I expected the first build to write to. This makes the situation especially confusing.) While this may be expected behaviour, it makes it difficult to marry up a build with its log file (e.g. when developing tooling) and is also misleading, as log files appear to correspond to incorrect build names. I have also been unable to figure out the real log file path (e.g. from bitbake variables or events fired during a build) to remedy this. To see the log files and how they correspond to build names: 1. Add a line to bitbake/lib/bb/cooker.py, just before line 1397: logger.info('******* Build name: %s' % self.data.getVar("BUILDNAME", True)) 2. Run a build, e.g. "bitbake zlib". 3. During the build, the output on the command line will show the build name. 4. Once the build is finished, look in the build/tmp/log/cooker/qemux86 directory (or the equivalent directory which contains the log file for the build). The log file for the build should be there. Note that this file will also contain the line printed by the log.info() call added in step (1). I intuitively expected the log file path to have the build name's timestamp + ".log" as its base name; but the log file path has a different base name, corresponding to a different timestamp (earlier than the build timestamp) + ".log". 55140 1 richard.purdie 2015-10-11 07:05:41 +0000 The trouble is that bitbake.conf defines BB_CONSOLELOG as containing ${DATETIME} and this can get expanded to a different value each time the variable is read. The system does lock down DATE and TIME but only at BuildStarted time which is long after the UI comes up. One possible way of dealing with this may be to change the ?= definition of BB_CONSOLELOG as := however this means any user defined value won't work. Another may be for the UI to forcibly expand the value when it starts to use it, i.e. run a setVariable with the getVariable value. This is slightly more invasive with code changes but likely the correct thing to do. I would take a 2.0 patch to get this fixed properly rather than some of the workarounds being proposed. 56117 2 elliot.smith 2015-11-03 07:56:35 +0000 I think this is now resolved in master. I will close bug 8373 once this is closed. 56873 3 richard.purdie 2015-11-30 15:53:27 +0000 http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=5b19b712526729e19c787fa68e0e5d8191988609 57953 4 costin.c.constantin 2016-01-19 16:21:32 +0000 Hello, Used latest master: 839fb1893546ad58a1f5475a9e852b81afb056d6 on an Ubuntu 15.04 with latest updates. I reproduced steps from comment#1 and found some issues: Case 1) Using bitbake from command line to launch a target build (used same target zlib as on initial comment) Steps performed: a) add the line logger.info('******* Build name: %s' % self.data.getVar("BUILDNAME", True)) after line no. 1414. of bitbake/lib/bb/cooker.py b) Source the env. and build zlib: ~/poky$ source oe-init-build-env ~/poky/build$ bitbake zlib ---> NOTE: ******* Build name: 20160119152958 c) resulting logfile: ~/poky/build/tmp/log/cooker/genericx86/20160119152951.log looks fine until now. Case 2) Using toaster Steps performed: a) start toaster: ~/poky/build$ . ../bitbake/bin/toaster ... Successful start. b) Fire zlib build from ui (before that a "$ bitbake zlib -c cleansstate) c) resulting logfile: ~/poky/build/tmp/log/cooker/genericx86/build_20160119_174224.446.log Inside this log file the added logger.info(...) returned 20160119154223 my system time is: $ date Tue Jan 19 17:47:32 EET 2016 meaning "BUILDNAME" takes GMT time as value: $ TZ=GMT date Tue Jan 19 15:46:05 GMT 2016 d) run a new build in command line: $ bitbake zlib e) consult resulting log file: This time two log files resulted: ~/poky/build/tmp/log/cooker/genericx86$ ls -ltr total 96 ... -rw-rw-r-- 1 autobuilder autobuilder 24366 Jan 19 17:49 20160119154218.log -rw-rw-r-- 1 autobuilder autobuilder 22965 Jan 19 17:49 build_20160119_174854.653.log and though the BUILDNAME is the same value inside: 20160119154854 the file names differ badly. I'm not sure if i'm missing something, but i see no reason for two log files and two different naming for the same build (the difference between building in command line log naming and via toaster ui log naming) Attached are all log files following steps presented above. 57954 5 2935 costin.c.constantin 2016-01-19 16:24:46 +0000 Created attachment 2935 files inside preserve date of creation log files. files inside preserve date of creation 57955 6 elliot.smith 2016-01-19 16:30:42 +0000 The additional logfile build_20160119_174224.446.log is for use by Toaster and (I think) it is irrelevant to the bug. Note that the log files are not identical: bitbake doesn't produce a per-build log file, which is what was needed for Toaster; so our work-around was to add our own separate log file. This is why you don't get a "build_*" log file for builds initiated without toaster. There may be a case for renaming our log file "toaster_build*" to make it clearer why it's there, however. 57973 7 costin.c.constantin 2016-01-20 07:50:35 +0000 Hi Elliot, I noticed the existence of a separate log file for Toaster. What i wanted to point out is that when a build is done from command line, two log files are created: one the usual way, ${DATETIME}.log and the other one for Toaster, build_%y%m%d_%H%M%S.%miliseconds.log I noticed a 6 minutes difference in their naming between these two logs and the base time being different (one is GMT and the other one is my time), though they are the result of the same bitbake run. 58073 8 richard.purdie 2016-01-21 16:19:33 +0000 (In reply to comment #7) > I noticed the existence of a separate log file for Toaster. > What i wanted to point out is that when a build is done from command line, > two log files are created: one the usual way, ${DATETIME}.log and the other > one for Toaster, build_%y%m%d_%H%M%S.%miliseconds.log > I noticed a 6 minutes difference in their naming between these two logs and > the base time being different (one is GMT and the other one is my time), > though they are the result of the same bitbake run. I don't think there is any expectation that these stamps should match. The time toaster triggers the build is not going to be the same time that the cooker/server log is created. I therefore don't think there is an issue here and we're ok with the code as it stands. 58177 9 costin.c.constantin 2016-01-25 12:37:59 +0000 Hello, As per comment#8, marking this as verified. 2935 2016-01-19 16:24:46 +0000 2016-01-19 16:24:46 +0000 files inside preserve date of creation 8411.zip application/zip 4353 costin.c.constantin UEsDBBQDAAAAAI2UM0gAAAAAAAAAAAAAAAAFAAAAODQxMS9QSwMEFAMAAAgAxIszSAMh7AEYBAAA KloAABcAAAA4NDExLzIwMTYwMTE5MTUyOTUxLmxvZ+2cW8/SQBCG7/kVG248Aj1/hcSLD0UlUVQO Jl412+0A1Xa3tlsVf70tpcViDClCmpjhghD2mdmdd15CJ6TM3i0nIzKHRATffL4hlO9I6CdJ/lrS 5Av5mkIKxIMIuAec+ZB0OuPUDzzyXPC1v0ljKn3BR53x2Pk4mS+m72bk8HhGumpfs/tKtzNeTd+8 cBafFuS3xR+25VhGL/B5+qPbmd0vpx8nbxbjxXI+nb0qkJWbcpn2VLOvGN3O8n7+arKs0hSIb9lW LxJfdmWit/fPX09nkwIoqQ1wiH2WbdntvJhmW7wjdSDPUC5VhRRLWl95knAaJVshe5qiWoqqDrPT rGYT5+XkfrmaTxZlmlDXCBMxaNVpX75f/bZNtxOCpL2dYFKUb//2Vs9NojITTSTEI4Uy3aCqpxrq 2rszh9bQBpqVbDNKPZNR29A9825oFYmPJdWz2Ppw7ar2UDcNKwuzqbo2jTuTDsE2NddW6dpVTMvL snQKR7yPIaJx7oJ5yj/kHjgsTH4AS2W+sAC5YMCBLDOfJIflx8WDFBbhNIQRKSVTTW1o2v8jWIpS qYWiIIgggggiiCCCCCKIIIIIIogggggiiCCCCCKIIIIIIogggggiiCCCCCKIIIIIIogggggiiCCC CCKIIIIIIoggggi2Ac5TzqsbJHX9jog1MXSFPJy+GBHzKRlsRQgDmkrh5qkgHuR3Fw4cKfZ34zks EBySgbPxpePkz/39zX9RLD4Dk30Rb5w8wAn3+CAESQcxMD+CpMdEDIOfge/unxy1r/Xtvus+JZ5w 1iDZ9tHNykYQQQQRRBBBBBFEEEEEEfwnsJhrSD7M9PbDTC9WRvvJqppoRmQhaSzBaxSSMgbglUH1 mc1QrdrMprQ3s6U8ouzLo+vKViStdGsQc1a4+rCrtidcRLM+X1m3iDax2zHkrGp2TTWrPdXY4U+b oKlypxUN6z5o8RMUiSgNqAQn8NmV/VCp1cQT9RNdHlkzVZPDnnOjptR6d9de66ojO5GERDouTeBW LTxucezmpRnOS6zWJLbblDiM/OD6quZZj0I2CDqvnVbTbti6drc058kGFynayJh6TVy9PXF9nkga BFdW9JC1krFJ0FntjJp2Wuva3c6Yf25wkaJNjGnWLyiM9tTNr4Pp5l+vkLT6iNFmPeVFRbJLYiHk lb1ymr6BU45iV0EXbVmZq9Gm5z15Muy0Ou3sT+xRSa/dvmPmqglNA88KadeFbPE30sOpne+xL8GJ o/BfP+Yng1CL3wplaV9vZJGjZs2McjzXxXFHf1124L8blAv4AezQTF2pN7PFy6P9bk37WPzrbFZr GNJ4NyL3UkIYZXrva5L5Yl7g963PtsRQDeL5Hn8gCYcMkYK4kAkbp5xQ7hEaBCQpVet3fgFQSwME FAMAAAgAIo4zSIdum8h4BAAALl8AABcAAAA4NDExLzIwMTYwMTE5MTU0MjE4LmxvZ8RXXc+TMBS+ 369oduVX+Row9iZeOJ26RKfuw8SrppSzidso0mL2+uuldJgRF6MDK8kKo/Q5fc45fdqzeLee3aEl CH74lmY7RLN7dEyFUM+Sij36WkIJKIEcsgQyloIYDKZlekjQc55t011ZUJny7G4wnZKPs+Vq/m6B ztdTNHQtL7Kc4WC6mb95QVafVuii8xSFJPTxIc3K03CweLaef5y9WU1X6+V88Up/sonLTJbYDSzH Hw7Wz5avZmsFc4GShlGIc76/b4DePnv+er6YIXRpawcZFCmrTA4HL+aViXeo/YFCaLoaIucuz3Ie i4zm4jOX2HPc0HHdSTWbzWJGXs6erTfL2aqBOY48xHgB3s/Zvny/uTAzHBxBUnzPmeTN64tXOBZ5 g0SFhOLOoWzkUzdxfXebjINJOImAVpQjRmkSMBr5oyQYT0INfMGphRKNJtvYjSajwA+rYRF1t4E/ DugEosCLI5duYycIkwploDPifQE5LVQWLMvsg8qBc8fsBKyUqmMFcsUgA7SmYi90t/o6U50CpFCd OonCCPEtisbogf2ZH8GmpeSxSiIobOV4m0heT5SwA89A2GSXSkJUa9V+yQv+BZi0eLEjagDRvGzF 2S6ApTkIrPxufz+kcd2QKvesyIrjJyjh1aC8PFAJRNyLgnNJmgk+PE/8kb6Qzu2MHuEONbF2Az8K /N8znPx3hpTt6Q4SKmnf5MaOCXIJfJOcH4S9Y0z9cFFmMj0CCayRCZKuCZJ1BHdV8JhuiWd5ngl2 ngl2P0N4pHuoG+Jbrgl6IxP0RJnnvJB2LkFIlZ4ZFK0/xLUcE2x9E2z3UGRwsOtdFatcxZ+BVibE lVemwhyYIN4SoopldTOlQaFRfhU39dDcMSu4EMSzfCs0QXZsgqwWXJASTrK5E8dyJ2Y4mjv4wCmn UrdVCPsWIm0KXYodrmzgwrnTTK8bquBWkhYSkjaOOr7g+vhyK4LaXHClOreOrx2FPasDCS0NuJKG /h1x5bj6O5hfFbmLby4OX93YtYQF18JyK9R56WK1dDug1C5SZy6DASsZA0i6LYEGo/ty1Ei9hFxD dfOxxugebY3TUwJqsI6SoUG66FaD8Mdi/c8+vL67RWaLiit1oanyPjJSX1wtDvun2Nq+Oq/2K8Jx 2z7W1o7/tB32TbLB+312GannrpQ1/2YJtfKrh6TQQDc5vof8aoB6EhIjFd7flu7958APduzQCAAA hIHY/lsjMDhq6uKj4ED8D5OLrT+BfO/bvW8Q3+5dGwwIgiAIgiAIgiAIgiAIgiAITjt3jIIwEARQ tPcU02khgiaN6QQba08QswMJZDeyTlBvr7uSVrCwCb+aYh4Mc4EPBAKBQCAQCAQCgUAgEAgEAoFA IBAIBAKBQCAQCAQCgUAgEAgEAoHAWcMpURMGfWjzqaKURYo957E6HSvZFmv5Z6rmW9I6X/s5PZOz LHIeva/js5KDmfqrqcs/WVqmB+9t17RS7vbiOheWJkHfxAa5qESNY5A6OKn7Xm5TCWazeAFQSwME FAMAAAgAUY0zSC5vPvOAAwAAC14AACIAAAA4NDExL2J1aWxkXzIwMTYwMTE5XzE3NDIyNC40NDYu bG9n7Zxbz5NAEIbv/RVzVw8t59a2URM13hnPXpMtjAULu7gstmr87y6gxi8m9qsBW5O36QGYh+Xd 2dnZgYs+e/7myZreVqkwudwSH/K629g0eZFOaZ+bjL5Mur1YipIna5oEnr/wfH/lz6MgCCdTmmz0 pres/XY3UWVVsOE0VtIetm2zyUt2fmzcbFuYkm/fK/uZTykKphTY36UfhP5ySuZzLt+p+/fevnn8 4JZtsdLqPSfGNvZMSbYHUqtTq/gj6zpXsru2492ppajqTJnZD4mtmtoI/bdavPnvWkqRZLns+vuB y+awXPR9VjvWcaG2cSVM1lrdTJXsisaozn+s3UrtPrndjmvKyrWw25/nblmyzhPbWG+Pf/Qg9u9a L0dOFC0cy3fezs1G7PjXzvtOsHS8yU/HtMfai02+3uhHONEsDKckeU+ZRZS9mCioEJ9Y0/eGKFws T8NXJ+F3ve/4C82V0G2YvWrky4Yb/m54cuCk6eLvNZvXCUumN6Le1d/Nt/sXPWo9RG04rulqMAI8 D/hUbd/lBdM7pcnYEaOjsb9XemeDX3MwC4NZa5wVuWwObvdttzfJLGNhz6zdyPFn2nMN93PGSVVc iWQntpwKI+KaTd3GihOsosgfXtFnK8a1M8xZ/qZCVU0hDMf1p1orZa5ICRf/VErvkHivc6tHV+VV LctzaPkgroq4O7wIq8EoVcwSreraDZzIWVw7VEYJ3s02Sdy5EzretXUEw+vYsjF8MK7n+KsTPBIN r4QPlTB2YPwTHDIfXoYdlZluZLvinzY64WoEMW12sz4Jgmu7xBtexUXM3/8kof05z4+2rJ5n/UVG PR4cx1XMlyNk01Ls+IRyaOGPUA5VhmvTJlPJ2nrEu7aWEUblQlaXy6tZL6cA+M8nzvC5+ozggA79 Q0F1bM0Mvbk/hqKfVdVpUkaItsso8C4pUQ8ULueI9L4COV31Mhwjr1/SfcxleeZvp/9o2fY8wXqk Ejk+OHdHuM0bUlb/kPiXp8d4SAwQIECAAAECBAgQIECAAAECBAgQIECAAAECBAgQIECAAAECBAgQ IECAAAECBAgQIECAAAECBAgQIECAAAECBAgQIECAAAECBAgQIMB/AXb/dEKvm7IU+tOaHhrDZWU4 pSj0yHRG9Y72WZ5kFAUrSvNUTgxJtohRtGHSrBtJQqYkioLqJkmsjVPnxjdQSwMEFAMAAAgAIo4z SKMWz4NxAgAAtVkAACIAAAA4NDExL2J1aWxkXzIwMTYwMTE5XzE3NDg1NC42NTMubG9n7dxNi9sw EAbg+/6KuRUK/k42yd5a2Fvp1+590cqzjoktGWncTfrrq2xDSy8hLnbXhTeHQNAjeJFGY+FDPn66 v70h7VgJl2T4mba1F+tqrRpq1IEdfWPna2uoWGVXg3g+jBcn/tlxp1xtKvramy8993wauN2z7uU4 cMdyp9kw3Su/86fhtz8/9L6vm5KMavmG8jS7TrNsky0X6+UCcObwg62e6obpyTqSsLWUbG3LierF Ph7nsks6uzskLz8Sabvk2bpdoq3jPCry6DgYNbXp90kn7CVyvTFhUhankUsT4TCjsVVc2odO6Z2q uFSiHjyLP5ZTXKSbdTF+lu9N/Rgy5PH60hSrzfgpWrXjZBFnF69EPn4G3ndKkjwesiHr8WOE/ai0 TpZxMSDHcvwcZwrDdn2jhB/8wTtr5c/qmGRJwncU8uhoyyrM9INqZYITXGl9PL9Stzxwp9Lxw4SF EWubSDvrfSjgRXx9cZzr8eNULMJ7SdI42wxIspogybFkwoLk+cUpsvFTzOggzaPZz+nxN6uDPI96 nc/5nVljm8kFYfhdabI78SvCERd0Ljeu873o/PMizydpBb8a0qAoUyzO/3gHPL9dc634M6mL1TRd fj5h/rriJ2t0r1MfZw7bZZuzmeDiOGas0wu732/y8MIOEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQ EBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQ8F/Al386obu+bZU73NA7 EW474ZIWRUryMmif6Hlb6y0t8g2VdWneCBkORCw9Mjl2vSFlSlJNQ77XOoxxGV/9AFBLAQI/AxQD AAAAAI2UM0gAAAAAAAAAAAAAAAAFAAAAAAAAAAAAEID9QQAAAAA4NDExL1BLAQI/AxQDAAAIAMSL M0gDIewBGAQAACpaAAAXAAAAAAAAAAAAIIC0gSMAAAA4NDExLzIwMTYwMTE5MTUyOTUxLmxvZ1BL AQI/AxQDAAAIACKOM0iHbpvIeAQAAC5fAAAXAAAAAAAAAAAAIIC0gXAEAAA4NDExLzIwMTYwMTE5 MTU0MjE4LmxvZ1BLAQI/AxQDAAAIAFGNM0gubz7zgAMAAAteAAAiAAAAAAAAAAAAIIC0gR0JAAA4 NDExL2J1aWxkXzIwMTYwMTE5XzE3NDIyNC40NDYubG9nUEsBAj8DFAMAAAgAIo4zSKMWz4NxAgAA tVkAACIAAAAAAAAAAAAggLSB3QwAADg0MTEvYnVpbGRfMjAxNjAxMTlfMTc0ODU0LjY1My5sb2dQ SwUGAAAAAAUABQBdAQAAjg8AAAAA