Bug 8280

Created attachment 2725 [details]
cooker_log.txt

The build log has some errors that might be useful to debug this. Attaching it.

Created attachment 2740 [details]
Screenshot of directory structure

I am unable to reproduce this: on a clean toaster db with a fresh build of core-image-minimal, I can see the directory structure (see attachment).  I'm on Fedora 20 with poky master @ ea562de57590c966cd5a75fda8defecd397e6436, using the SQLite back-end.

I did a second run of the same build, and can also see the directory structure for that.

I did a third run, and can see the directory structure for that, too.

I started the first build by putting "core-image-minimal" into the build text entry and pressing the "Build" button; for the second build, I clicked on the "Run again" button next to the first successful build; for the third build, I used the build text entry field and selected "core-image-minimal" again.

Could you please provide a full set of steps, from a fresh checkout of master with a clean database? Include full information about which db backend you're using, how you start toaster, the project name you set, the release you're using for the project, how you're triggering the build etc. so I can follow your steps exactly.

Hi Elliot,

I've just tried this on Ubuntu 15.04 and i'm still missing the directory structure.
DB used: sqlite
Cloned new poky, checkout out on commit: ea562de57590c966cd5a75fda8defecd397e6436 (checkout being tested for M3)

Steps:
1. Start toaster
2. create new project (yocto project master) - project name testproj
3. build core-image-minimal by entering the text in the build text box.

No changes were made to the default settings, so machine is qemux86 and layers are openembedded-core, meta-yocto and meta-yocto-bsp.

I have also verified that this happens on Fedora 22, but with local yocto release and genericx86-64 machine.

Either this doesn't happen on Fedora 20, for some reason, or both myself and Cristina have something very wrong with our machines.

Mihai

Thanks Mihai, I think I have to upgrade my machine, as Fedora 20 isn't officially supported anyway. I will take this and get back to you.

I've done some investigation, and I think it boils down to the code trying to add files as parents of the root directory when that directory doesn't exist.

The code assumes that there will be an entry for the root directory in the list of directories. Looking in the log supplied in https://bugzilla.yoctoproject.org/attachment.cgi?id=2725, line 159, notice that there is no entry for the '/' directory. This means that when the first directory (./bin) is added into the data structure for the filesystem, the code does a lookup for the object representing the root '/' directory. This hasn't ever been created, so the lookup fails, and we see the DoesNotExist error.

I'm not sure why this doesn't happen on my setup, however. I'm also not 100% sure this is the problem, but my line of enquiry will continue from here.

Mihai, do you have one or more files in your bitbake build directories (on the machine where the directory listing is failing) called "files-in-image.txt"? If you do, could you please attach one of them, preferably for a build where you can't get the directory listing? Thanks.

This traceback can be related to the bug:
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 2033 tasks of which 2033 didn't need to be rerun and all succeeded.
ERROR: Target_File matching query does not exist.
Traceback (most recent call last):
  File "/home/ed/git/yocto/poky/bitbake/lib/bb/ui/toasterui.py", line 285, in main
    buildinfohelper.store_target_package_data(event)
  File "/home/ed/git/yocto/poky/bitbake/lib/bb/ui/buildinfohelper.py", line 1090, in store_target_package_data
    self.orm_wrapper.save_target_file_information(self.internal_state['build'], target, filedata)
  File "/home/ed/git/yocto/poky/bitbake/lib/bb/ui/buildinfohelper.py", line 364, in save_target_file_information
    parent_obj = self._cached_get(Target_File, target = target_obj, path = parent_path, inodetype = Target_File.ITYPE_DIRECTORY)
  File "/home/ed/git/yocto/poky/bitbake/lib/bb/ui/buildinfohelper.py", line 115, in _cached_get
    vars(self)[dictname][key] = clazz.objects.get(**kwargs)
  File "/usr/lib/python2.7/site-packages/django/db/models/manager.py", line 151, in get
    return self.get_queryset().get(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/django/db/models/query.py", line 310, in get
    self.model._meta.object_name)
DoesNotExist: Target_File matching query does not exist.

I can now reproduce this bug, even though I thought I was using the same environment as before, where I couldn't reproduce it. I will continue investigating.

This can be fixed in at least two ways; one I've figured out, the other I haven't:

1. (the quick work-around I've figured out) I can modify the save_target_file_information() method in buildinfohelper.py so that it always creates a mock root directory entry. This directory won't have proper file ownership, permissions or size data, because I'd be creating it out of thin air; but this isn't so important, because the root directory is never displayed anywhere.

Also note that it's not possible to create it from the current data in files-in-image.txt for a given build, as that file doesn't contain the root directory information. (see 2 below for a suggestion about fixing this)

Because the root directory would always exist, any files underneath that directory could be assigned a parent correctly.

2. (the proper fix I haven't figured out) Modify how files-in-image.txt is created so that I can ensure the root directory is included in the output (it currently isn't). The current code in buildinfohelper.py would then cope correctly, as the root directory would be available.

This would entail fixing meta/classes/toaster.bbclass, which creates the files-in-image.txt file.

Sent to toaster mailing list for review.

After discussion with Brian, we went for the work-around in (1): add a fake root directory.

In bitbake master.

This  issue can't be verified because the bug with 8558 id blocks it.

Verified on master: e1aa897beb33171bc902207b98c8f666b9e97ad3