| Summary: | Impossible to download cooker log from Toaster | ||
|---|---|---|---|
| Product: | [Build System, Metadata & Runtime] Toaster | Reporter: | Belen Barros Pena <belen.barros.pena> |
| Component: | toaster | Assignee: | Elliot Smith <elliot.smith> |
| Status: | VERIFIED FIXED | QA Contact: | Alexandru Roman <alexandru.costinx.roman> |
| Severity: | normal | ||
| Priority: | Medium | CC: | alexandru.costinx.roman, belen.barros.pena, jessica.zhang, stanciux.mihail |
| Version: | 1.8 | ||
| Target Milestone: | 2.0 | ||
| Hardware: | x86 | ||
| OS: | Multiple | ||
| Whiteboard: | |||
| OS type for building Yocto: | --- | Type of Regression: | --- |
| Verified: | Documentation change: | Yes (doc changes required) | |
|
Description
Belen Barros Pena
2015-08-26 09:48:52 UTC
The cooker log for a build is copied by the runbuilds command from the "real" build environment to a toaster artifacts directory, keyed by the ID of the build. e.g. artifacts for build 1 go into toaster_build_artifacts/1 (NB toaster_build_artifacts is a customisable directory name). runbuilds is run once per second (!) by bitbake/bin/toaster. runbuilds initiates the copying of build artifacts from bitbake to the toaster artifacts directory for the build. Only build requests with status BuildRequest.REQ_ARCHIVE are queued up to have their logs copied. One possibility is that build requests don't have or get this status (perhaps they used to), so runbuilds ignores them. The output for runbuilds goes into /tmp/toaster_N/, where N is the process ID of the original bitbake/bin/toaster command. runbuilds.py (in bitbake/lib/toaster/bldcontrol/management/commands) contains the command which does the copying, archive(). archive() constructs a new cooker log filename of the form toaster_build_artifacts/$build number$/cooker_log.txt, then writes the "real" log file's content into that file. Note that the build "knows" where its log file is (Build.cooker_log_path), so I'm not sure why we're not just using that. Perhaps because that would require the user to keep their bitbake logs. This might be a problem if a user cleared out any build detritus periodically (e.g. by deleting their old build directories), but we want toaster to be able to keep its logs. However, it would make toaster a lot simpler if it just pointed to the artifacts already in bitbake. This new log filename is the one which toaster tries to read (to send the log back to the user when they click the link in the UI). The view which handles requests for build artifacts is build_artifact() (in toastergui/views.py). The template used is unavailable_artifact.html (in toastergui/templates). When I tried to reproduce the bug, I modified the code so that the real error is shown by the view's template when the download fails. This reveals that the cooker_log.txt file is not being correctly created, so the expected file doesn't exist. This is why the view fails and shows the error message. I checked the filesystem and there was no sign of the cooker_log.txt file where it was supposed to be. The directory for the build (toaster_build_artifacts/1) was there, however. When a build request is successfully completed, its status is set to 3 (BuildRequest.REQ_COMPLETED). It looks like this means it won't be queued up to have its log copied to the toaster build artifacts directory, as its status never gets to BuildRequest.REQ_ARCHIVE (and runbuilds looks for BuildRequest objects with status BuildRequest.REQ_ARCHIVE). The only place I can find where the BuildRequest's status is set to REQ_ARCHIVE is in bitbake/lib/bb/ui/buildinfohelper.py, in the BuildInfoHelper._store_build_done() method. This method is invoked from BuildInfoHelper.close(), which appears to store done build requests, but only if the BuildInfoHelper has a brbe property set. This brbe property can be set from the TOASTER_BRBE environment variable, which in turn is set in the bitbake/bin/toaster script. However, it also appears to be set automatically in runbuilds if it is not in the environment. This variable is saved into the database. (It's not clear what brbe actually stands for; "build request build environment"? In runbuilds.py it's called the "build identification variable", and has the format "$build request id$:$build environment controller.build environment$".) After more digging: BuildInfoHelper._get_build_information() sets the cooker_log_path for a build from the BB_CONSOLELOG variable. The time stamped into the value for this variable seems to be about a minute outside the correct time. For example, the log file which is supposed to exist is called: /home/ell/dev/toaster/poky-contrib/build/tmp/log/cooker/qemux86/20150907181435.log But the nearest log file is actually: /home/ell/dev/toaster/poky-contrib/build/tmp/log/cooker/qemux86/20150907181339.log Note that the timestamp is 56 seconds before the expected timestamp. It seems as though the actual log file name is slightly different from the expected one produced by bitbake. >
> Note that the build "knows" where its log file is (Build.cooker_log_path),
> so I'm not sure why we're not just using that. Perhaps because that would
> require the user to keep their bitbake logs. This might be a problem if a
> user cleared out any build detritus periodically (e.g. by deleting their old
> build directories), but we want toaster to be able to keep its logs.
> However, it would make toaster a lot simpler if it just pointed to the
> artifacts already in bitbake.
This brings up the question of what happens to the task logs. Are they also stored in that artifacts directory? Or is Toaster just pointing to the logs created by BitBake?
In any case, I think it should be fine to just use what's created by BitBake instead of the black magic going on at the moment. Yes, you will lose the logs if you delete the build directory, but that is what currently happens with all other artifacts any way.
Submitted to toaster mailing list for review. Making modifications after review by Brian Avery: This fixed it for the managed build case but it failed for the analysis mode case. Environment: Request Method: GET Request URL: http://192.168.168.66:8000/toastergui/build/1/artifact/cookerlog/id/1 Django Version: 1.6 Python Version: 2.7.6 Installed Applications: ('django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.messages', 'django.contrib.sessions', 'django.contrib.admin', 'django.contrib.staticfiles', 'django.contrib.humanize', 'bldcollector', 'toastermain', 'south', 'toastergui', 'bldcontrol', 'orm') Installed Middleware: ('django.middleware.common.CommonMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware') Traceback: File "/big/src/intel/yocto-builds/TOASTER/myBugs/t-myBugs/local/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response 114. response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/big/src/intel/myBugs/poky/bitbake/lib/toaster/toastergui/views.py" in build_artifact 2745. build_artifact_storage_dir = os.path.join(ToasterSetting.objects.get(name="ARTIFACTS_STORAGE_DIR").value, "%d" % int(build_id)) File "/big/src/intel/yocto-builds/TOASTER/myBugs/t-myBugs/local/lib/python2.7/site-packages/django/db/models/manager.py" in get 151. return self.get_queryset().get(*args, **kwargs) File "/big/src/intel/yocto-builds/TOASTER/myBugs/t-myBugs/local/lib/python2.7/site-packages/django/db/models/query.py" in get 307. self.model._meta.object_name) Exception Type: DoesNotExist at /toastergui/build/1/artifact/cookerlog/id/1 Exception Value: ToasterSetting matching query does not exist. Changing the documentation flag to 'yes': we need to remove the layer checkout directory step from the set up process descriptions. Merged into bitbake master. Verified on master: fc45deac89ef63ca1c44e763c38ced7dfd72cbe1 |