<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<!DOCTYPE bugzilla SYSTEM "https://bugzilla.yoctoproject.org/page.cgi?id=bugzilla.dtd">

<bugzilla version="5.0.6"
          urlbase="https://bugzilla.yoctoproject.org/"
          
          maintainer="it-coreprojects-helpdesk@linuxfoundation.org"
>

    <bug>
          <bug_id>13723</bug_id>
          
          <creation_ts>2019-12-23 02:27:34 +0000</creation_ts>
          <short_desc>Layer index parsing broken</short_desc>
          <delta_ts>2020-01-07 22:50:32 +0000</delta_ts>
          <reporter_accessible>1</reporter_accessible>
          <cclist_accessible>1</cclist_accessible>
          <classification_id>6</classification_id>
          <classification>Yocto Project Subprojects</classification>
          <product>Layer Index</product>
          <component>Layer Index</component>
          <version>unspecified</version>
          <rep_platform>x86</rep_platform>
          <op_sys>Multiple</op_sys>
          <bug_status>RESOLVED</bug_status>
          <resolution>FIXED</resolution>
          
          
          <bug_file_loc></bug_file_loc>
          <status_whiteboard></status_whiteboard>
          <keywords></keywords>
          <priority>High</priority>
          <bug_severity>major</bug_severity>
          <target_milestone>3.1 M2</target_milestone>
          
          
          <everconfirmed>1</everconfirmed>
          <reporter name="Paul Eggleton">bluelightning</reporter>
          <assigned_to name="Paul Eggleton">bluelightning</assigned_to>
          <cc>mark.hatle</cc>
          
          <qa_contact name="apoorv sangal">apoorvsangal</qa_contact>
          <cf_os>---</cf_os>
          <cf_regression_type>---</cf_regression_type>
          
          <cf_docchange>No (bug/feature does not impact docs)</cf_docchange>

      

      

      

          <comment_sort_order>oldest_to_newest</comment_sort_order>  
          <long_desc isprivate="0" >
    <commentid>85961</commentid>
    <comment_count>0</comment_count>
    <who name="Paul Eggleton">bluelightning</who>
    <bug_when>2019-12-23 02:27:34 +0000</bug_when>
    <thetext>Parsing OE-Core (and of course any other layer that uses it) is currently broken:

------------ snip ------------
$ layerindex/update.py -b master -l openembedded-core
INFO: Fetching remote repository git://git.openembedded.org/openembedded-core
INFO: Fetching remote repository git://git.openembedded.org/bitbake
INFO: Collecting data for layer openembedded-core on branch master
NOTE: Starting bitbake server...
ERROR: Unable to start bitbake server (None)
ERROR: Last 60 lines of server log for this session (/tmp/tmp_g7583ey/bitbake-cookerdaemon.log):
  File &quot;Var &lt;AVAILABLE_LICENSES[:=]&gt;&quot;, line 1, in &lt;module&gt;
  File &quot;/home/paul/poky/layerindex-web/layers/git___git_openembedded_org_openembedded-core/meta/classes/license.bbclass&quot;, line 275, in available_licenses
    licenses += os.listdir(lic_dir)
bb.data_smart.ExpansionError: Failure expanding variable AVAILABLE_LICENSES[:=], expression was ${@&apos; &apos;.join(available_licenses(d))} which triggered exception FileNotFoundError: [Errno 2] No such file or directory: &apos;${COREBASE}/meta/files/common-licenses&apos;
ERROR: Unable to parse Var &lt;AVAILABLE_LICENSES[:=]&gt;
Traceback (most recent call last):
  File &quot;/home/paul/poky/layerindex-web/layers/git___git_openembedded_org_bitbake/lib/bb/data_smart.py&quot;, line 411, in expandWithRefs
    s = __expand_python_regexp__.sub(varparse.python_sub, s)
  File &quot;/home/paul/poky/layerindex-web/layers/git___git_openembedded_org_bitbake/lib/bb/data_smart.py&quot;, line 136, in python_sub
    value = utils.better_eval(codeobj, DataContext(self.d), {&apos;d&apos; : self.d})
  File &quot;/home/paul/poky/layerindex-web/layers/git___git_openembedded_org_bitbake/lib/bb/utils.py&quot;, line 421, in better_eval
    return eval(source, ctx, locals)
  File &quot;Var &lt;AVAILABLE_LICENSES[:=]&gt;&quot;, line 1, in &lt;module&gt;
  File &quot;/home/paul/poky/layerindex-web/layers/git___git_openembedded_org_openembedded-core/meta/classes/license.bbclass&quot;, line 275, in available_licenses
    licenses += os.listdir(lic_dir)
FileNotFoundError: [Errno 2] No such file or directory: &apos;${COREBASE}/meta/files/common-licenses&apos;
------------ snip ------------

I haven&apos;t yet bisected to find out which commit caused this to break, but I have found that it is happening fairly early on - it&apos;s within the call to recipeparse.init_parser() in update_layer.py which in the layer index is too early for the layer.conf files to have been parsed, and thus COREBASE is not yet set.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>85964</commentid>
    <comment_count>1</comment_count>
    <who name="Mark Hatle">mark.hatle</who>
    <bug_when>2019-12-23 14:31:54 +0000</bug_when>
    <thetext>I ran into a similar issue with another layer.  The code was something like:

${@&apos;${VARIABLE}/something&apos;.function()}

And &apos;VARIABLE&apos; was not set within the context of the parsing engine (using tinfoil).  So an error was triggered that &apos;${...}&apos; didn&apos;t expand.  By simply defining &apos;VARIABLE&apos; this resolved the issue where I saw it.

But I do wonder if this is really the right behavior or if it the variable usage should be:

${@(&apos;%/something&apos; % d.getVar(&apos;VARIABLE&apos;)).function()}

The issue with the above is that the usage of &apos;VARIABLE&apos; is now lost and not tracked by the regular hash code that looks for ${...}</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>85987</commentid>
    <comment_count>2</comment_count>
    <who name="Paul Eggleton">bluelightning</who>
    <bug_when>2020-01-07 22:50:22 +0000</bug_when>
    <thetext>It turns out that this was directly caused by http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=2a263780814e3b07d48e3812579cfe4696db8413 (or more specifically, COREBASE not being set when the immediate assignment of AVAILABLE_LICENSES is set). I&apos;ve merged a patch that just sets a value for COREBASE to avoid the problem. Michael has cherry-picked the patch on top of the working branch used for layers.openembedded.org and the update is now running.</thetext>
  </long_desc>
      
      

    </bug>

</bugzilla>