8037 2015-07-21 09:34:25 +0000 External link to recipe file links to non-existing page when recipe information comes from a build 2016-08-24 10:02:41 +0000 1 1 7 Build System, Metadata & Runtime Toaster toaster 1.8 x86 Multiple RESOLVED FIXED Medium normal 2.2 1 belen.barros.pena david.reyna alexandru.damian belen.barros.pena cristiana.voicu jessica.zhang libertad.gonzalez.de.la.cruz --- --- No (bug/feature does not impact docs) oldest_to_newest 52501 0 belen.barros.pena 2015-07-21 09:34:25 +0000 Found on master:0f8d89ce66cfffccc9911d6c61470a585173b79d Steps to reproduce: 1. Create a Toaster project 2. Build a recipe (e.g. busybox) 3. Once the build finishes, go to the "all compatible recipes" page and search for the recipe you just built (e.g. busybox). It should have a commit SHA as its revision: this indicates the information for the recipe is coming from a build, and not from the layer index 4. In the "edit columns" menu, tick the checkbox for the "recipe file" column 5. Next to the recipe file information, there is a blue icon. Click it You will be brought to a URL like toastergui/project/[PROJECT_ID}/recipes/None and you'll get a 404 error. You should be brought to the recipe file in the layer web repository. In the case of busybox, something like http://cgit.openembedded.org/cgit.cgi/openembedded-core/tree/meta/recipes-core/busybox/busybox_1.23.1.bb?h=fido 54707 1 david.reyna 2015-09-27 07:19:22 +0000 I believe that the issue happens because/when there are multiple entries for same layer name in the table "orm_layer", and there is a race condition for which version of the layer a recipe attaches to which is why this defect is intermittent. I suspect that if these multiple entries did not exist there would not be false connections. 2) Here you can explicitly see that there are multiple entries for "openembedded-core" (the layer for "busybox") in the Toaster database. The broken URL happens when "vcs_web_file_base_url" is empty, and you can clearly see that one of the versions of that layer name has that problem. sqlite> select * from orm_layer where name="openembedded-core"; vcs_web_url = name = openembedded-core vcs_web_tree_base_url = vcs_web_file_base_url = layer_source_id = 1 layer_index_url = up_date = vcs_url = git://git.yoctoproject.org/poky summary = up_id = id = 1 description = vcs_web_url = http://cgit.openembedded.org/cgit.cgi/openembedded-core name = openembedded-core vcs_web_tree_base_url = http://cgit.openembedded.org/cgit.cgi/openembedded-core/tree/%path%?h=%branch% vcs_web_file_base_url = http://cgit.openembedded.org/cgit.cgi/openembedded-core/tree/%path%?h=%branch% layer_source_id = 2 layer_index_url = up_date = 2014-08-12 00:08:00 vcs_url = git://git.openembedded.org/openembedded-core summary = Core metadata up_id = 1 id = 4 description = OpenEmbedded-Core is a layer containing the core metadata for current versions of OpenEmbedded. It is distro-less (can build a functional image with DISTRO = "") and contains only emulated machine support. sqlite> 3) I believe that it is a race condition which instance of the layer name a recipe will attach to. If you for example you search on "alsa" you can see that for the recipes that use the "openembedded-core" layer, three succeed (alsa-plugins, alsa-tools, alsa-utils-scripts) and four fail (alsa-state, alsa-lib, alsa-lib-native, alsa-utils), at least in my database. - David 54708 2 david.reyna 2015-09-27 07:36:19 +0000 FYI, there are in fact three duplicate layer names: sqlite> .output a.txt sqlite> select name from orm_layer; sqlite> .exit [poky]$ cat a.txt | sort | uniq -d meta-yocto meta-yocto-bsp openembedded-core [poky]$ 54709 3 david.reyna 2015-09-27 07:48:12 +0000 Perhaps the "race condition" I mention is in fact the "build" action as per the defect title, where during a build the recipe gets reattached to the layer, but the bad layer entry wins. NOTE: I will also mention as an aside that it is weird that in the compatible recipe page that every single layer has the "Add Layer" button present, even thought they already part of the project, certainly "openembedded-core" is. 59614 4 david.reyna 2016-03-02 06:09:09 +0000 1) I can confirm that this is still a problem in Toaster-2.1 as of March 1 2016. 2) I observe that this only seems to be related to compatible recipes from "openembedded-core", even the ones that successfully build (My test was building "core-minimal-base", which built "busybox-1.24.1-r0"). This does not seem to affect any other layer, included/built (I tested meta-ros) or not. 3) This does not seem to be caused by the previous theory of duplicate layer entries in the database. I tested the newest database and found that the there is now one entry for "openembedded-core". There are still duplicated layer entries for "meta-poky" and "meta-yocto-bsp", but "meta-poky" has no recipes, and the single "meta-yocto-bsp" recipe "gma500-gfx-check" does not have this problem. 65174 5 david.reyna 2016-08-14 08:43:16 +0000 The reason the link is failing is because in the respective layer's table the "vcs_web_file_base_url" is not defined. The the affect layers, the "vcs_web_url" and "vcs_web_tree_base_url" are also not defined. These are the layers that have this deficit: Layer#: Name 1 : openembedded-core 2 : meta-poky 3 : meta-yocto-bsp 152 : meta-tizen 167 : meta-spec (has only "vcs_web_url") 206 : meta-phytec When this value in the database is manually set (e.g. "http://cgit.openembedded.org/openembedded-core/plain/" for the in "openembedded-core" layer) then the correct recipe files links appear. 65175 6 david.reyna 2016-08-14 09:15:31 +0000 The recipe links from layers "meta-yocto-bsp" and "meta-poky" work because these layers oddly have second corrected definitions: Layer#: Name 30 : meta-yocto-bsp 216 : meta-poky The recipe links from layer "openembedded-core" apparently fail because not only is it a default layer (e.g. 1..3) with incomplete handling, it did not get a second instantiation like the others to fill in the missing values. The values are indeed existing in the parent Layer Index, they are just not applied. The other layers like "meta-tizen" fail because the respective Web Repo values are not defined in the parent Layer Index. 65178 7 david.reyna 2016-08-15 01:21:55 +0000 1. Root Cause: It turns out that in "lsupdates.py", when scanning the updating the layer table from the Layer Index, the "openembedded-core" layer is treated specially and unfortunately the code to update the git web ULR's was not copied into that special case code. The fix is therefore trivial. 2. Note: it appears to me that all pre-set layers in the "toaster.conf" file will end up getting instantiated twice (except for the above hard-coded special case of "openembedded-core"), where their first is largely empty and their second entry has all of the Layer Index data. This would explain the above observations about "meta-poky" and "meta-yocto-bsp". There is no external issue since the second layer seems to always win and links are correctly populated (e.g. the recipes "tiny-init" and "gma500-gfx-check"), but this could be unstable and future updates of the code could reveal issues. 65478 8 belen.barros.pena 2016-08-24 10:02:41 +0000 merged to master http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=40b655db22f3ba27ff38e59acb657776e18e2db2