Bug 13183 - bitbake-layers crashes with incorrect layer configuration data is given (expected proper error printing and exit with error)
Summary: bitbake-layers crashes with incorrect layer configuration data is given (expe...
Status: RESOLVED FIXED
Alias: None
Product: BitBake
Classification: Build System, Metadata & Runtime
Component: bitbake (show other bugs)
Version: 2.6.1
Hardware: x86 Multiple
: Medium+ normal
Target Milestone: 4.0 M1
Assignee: Steve Sakoman
QA Contact:
URL:
Whiteboard:
: 14053 (view as bug list)
Depends on:
Blocks:
 
Reported: 2019-02-15 09:31 UTC by Viktor Mitin
Modified: 2021-12-13 23:18 UTC (History)
8 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Don't know


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Viktor Mitin 2019-02-15 09:31:49 UTC
Hi,

After adding meta-updater layer I get the next bitbake-layers crash:
https://github.com/advancedtelematic/meta-updater.git


c@cd:~/w/poky_thud/build$ bitbake-layers
NOTE: Starting bitbake server...
Traceback (most recent call last):
  File "/home/c/w/poky_thud/bitbake/bin/bitbake-layers", line 103, in <module>
    ret = main()
  File "/home/c/w/poky_thud/bitbake/bin/bitbake-layers", line 71, in main
    tinfoil.prepare(True)
  File "/home/c/w/poky_thud/bitbake/lib/bb/tinfoil.py", line 418, in prepare
    self.run_command('parseConfiguration')
  File "/home/c/w/poky_thud/bitbake/lib/bb/tinfoil.py", line 476, in run_command
    raise TinfoilCommandFailed(result[1])
bb.tinfoil.TinfoilCommandFailed: Traceback (most recent call last):
  File "/home/c/w/poky_thud/bitbake/lib/bb/command.py", line 84, in runCommand
    result = command_method(self, commandline)
  File "/home/c/w/poky_thud/bitbake/lib/bb/command.py", line 285, in parseConfiguration
    command.cooker.parseConfiguration()
  File "/home/c/w/poky_thud/bitbake/lib/bb/cooker.py", line 437, in parseConfiguration
    self.handleCollections(self.data.getVar("BBFILE_COLLECTIONS"))
  File "/home/c/w/poky_thud/bitbake/lib/bb/cooker.py", line 1236, in handleCollections
    raise CollectionError("Errors during parsing layer configuration")
bb.cooker.CollectionError: Errors during parsing layer configuration
Comment 1 Viktor Mitin 2019-02-15 10:58:23 UTC
Build 02-15-2019 on Ubuntu 18.04.2
Comment 2 Viktor Mitin 2019-02-15 12:04:36 UTC
Checking out 'Thud' tag in meta-updater layer sources resolves bitbake-layers crash. However, bitbake-layers should no crash when incorrect layer configuration data was given. No?

Thanks
Comment 3 Viktor Mitin 2019-02-21 15:48:12 UTC
Please note that this issue has been reconstructed with 'older' version of Yocto as well (do not remember which one, seems Rocko). So it looks like it is relevant for the previous versions of Yocto as well.

Thanks
Comment 4 sungwon.pino 2019-02-25 13:18:01 UTC
Hello

bitbake-layer work well, before I type `apt-get upgrade` 
both ubuntu 16.04 and ubuntu 18.04
So share the apt-get upgrade log.

pino@pino-dev:~$ sudo apt-get upgrade 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
The following packages were automatically installed and are no longer required:
  gconf-service gconf-service-backend gconf2-common libgconf-2-4
  x11proto-dri2-dev x11proto-gl-dev
Use 'sudo apt autoremove' to remove them.
The following packages will be upgraded:
  bind9-host dnsutils ghostscript ghostscript-x libbind9-160 libdns-export1100
  libdns1100 libgs9 libgs9-common libirs160 libisc-export169 libisc169
  libisccc160 libisccfg160 liblwres160
15 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/9,912 kB of archives.
After this operation, 11.3 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
(Reading database ... 195919 files and directories currently installed.)
Preparing to unpack .../00-libisc-export169_1%3a9.11.3+dfsg-1ubuntu1.5_amd64.deb ...
Unpacking libisc-export169:amd64 (1:9.11.3+dfsg-1ubuntu1.5) over (1:9.11.3+dfsg-1ubuntu1.3) ...
Preparing to unpack .../01-libdns-export1100_1%3a9.11.3+dfsg-1ubuntu1.5_amd64.deb ...
Unpacking libdns-export1100 (1:9.11.3+dfsg-1ubuntu1.5) over (1:9.11.3+dfsg-1ubuntu1.3) ...
Preparing to unpack .../02-libirs160_1%3a9.11.3+dfsg-1ubuntu1.5_amd64.deb ...
Unpacking libirs160:amd64 (1:9.11.3+dfsg-1ubuntu1.5) over (1:9.11.3+dfsg-1ubuntu1.3) ...
Preparing to unpack .../03-bind9-host_1%3a9.11.3+dfsg-1ubuntu1.5_amd64.deb ...
Unpacking bind9-host (1:9.11.3+dfsg-1ubuntu1.5) over (1:9.11.3+dfsg-1ubuntu1.3) ...
Preparing to unpack .../04-dnsutils_1%3a9.11.3+dfsg-1ubuntu1.5_amd64.deb ...
Unpacking dnsutils (1:9.11.3+dfsg-1ubuntu1.5) over (1:9.11.3+dfsg-1ubuntu1.3) ...
Preparing to unpack .../05-libbind9-160_1%3a9.11.3+dfsg-1ubuntu1.5_amd64.deb ...
Unpacking libbind9-160:amd64 (1:9.11.3+dfsg-1ubuntu1.5) over (1:9.11.3+dfsg-1ubuntu1.3) ...
Preparing to unpack .../06-libisccfg160_1%3a9.11.3+dfsg-1ubuntu1.5_amd64.deb ...
Unpacking libisccfg160:amd64 (1:9.11.3+dfsg-1ubuntu1.5) over (1:9.11.3+dfsg-1ubuntu1.3) ...
Preparing to unpack .../07-libisccc160_1%3a9.11.3+dfsg-1ubuntu1.5_amd64.deb ...
Unpacking libisccc160:amd64 (1:9.11.3+dfsg-1ubuntu1.5) over (1:9.11.3+dfsg-1ubuntu1.3) ...
Preparing to unpack .../08-libdns1100_1%3a9.11.3+dfsg-1ubuntu1.5_amd64.deb ...
Unpacking libdns1100:amd64 (1:9.11.3+dfsg-1ubuntu1.5) over (1:9.11.3+dfsg-1ubuntu1.3) ...
Preparing to unpack .../09-libisc169_1%3a9.11.3+dfsg-1ubuntu1.5_amd64.deb ...
Unpacking libisc169:amd64 (1:9.11.3+dfsg-1ubuntu1.5) over (1:9.11.3+dfsg-1ubuntu1.3) ...
Preparing to unpack .../10-liblwres160_1%3a9.11.3+dfsg-1ubuntu1.5_amd64.deb ...
Unpacking liblwres160:amd64 (1:9.11.3+dfsg-1ubuntu1.5) over (1:9.11.3+dfsg-1ubuntu1.3) ...
Preparing to unpack .../11-ghostscript-x_9.26~dfsg+0-0ubuntu0.18.04.6_amd64.deb ...
Unpacking ghostscript-x (9.26~dfsg+0-0ubuntu0.18.04.6) over (9.26~dfsg+0-0ubuntu0.18.04.4) ...
Preparing to unpack .../12-ghostscript_9.26~dfsg+0-0ubuntu0.18.04.6_amd64.deb ...
Unpacking ghostscript (9.26~dfsg+0-0ubuntu0.18.04.6) over (9.26~dfsg+0-0ubuntu0.18.04.4) ...
Preparing to unpack .../13-libgs9_9.26~dfsg+0-0ubuntu0.18.04.6_amd64.deb ...
Unpacking libgs9:amd64 (9.26~dfsg+0-0ubuntu0.18.04.6) over (9.26~dfsg+0-0ubuntu0.18.04.4) ...
Preparing to unpack .../14-libgs9-common_9.26~dfsg+0-0ubuntu0.18.04.6_all.deb ...
Unpacking libgs9-common (9.26~dfsg+0-0ubuntu0.18.04.6) over (9.26~dfsg+0-0ubuntu0.18.04.4) ...
Setting up libgs9-common (9.26~dfsg+0-0ubuntu0.18.04.6) ...
Setting up libisc169:amd64 (1:9.11.3+dfsg-1ubuntu1.5) ...
Setting up libisccc160:amd64 (1:9.11.3+dfsg-1ubuntu1.5) ...
Setting up libisc-export169:amd64 (1:9.11.3+dfsg-1ubuntu1.5) ...
Processing triggers for libc-bin (2.27-3ubuntu1) ...
Setting up libdns-export1100 (1:9.11.3+dfsg-1ubuntu1.5) ...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
Setting up libdns1100:amd64 (1:9.11.3+dfsg-1ubuntu1.5) ...
Setting up libgs9:amd64 (9.26~dfsg+0-0ubuntu0.18.04.6) ...
Setting up liblwres160:amd64 (1:9.11.3+dfsg-1ubuntu1.5) ...
Setting up ghostscript (9.26~dfsg+0-0ubuntu0.18.04.6) ...
Setting up libisccfg160:amd64 (1:9.11.3+dfsg-1ubuntu1.5) ...
Setting up ghostscript-x (9.26~dfsg+0-0ubuntu0.18.04.6) ...
Setting up libirs160:amd64 (1:9.11.3+dfsg-1ubuntu1.5) ...
Setting up libbind9-160:amd64 (1:9.11.3+dfsg-1ubuntu1.5) ...
Setting up bind9-host (1:9.11.3+dfsg-1ubuntu1.5) ...
Setting up dnsutils (1:9.11.3+dfsg-1ubuntu1.5) ...
Processing triggers for libc-bin (2.27-3ubuntu1) ...
Comment 5 Viktor Mitin 2019-02-25 14:25:56 UTC
(In reply to comment #4)
> Hello
> 
> bitbake-layer work well, before I type `apt-get upgrade` 
> both ubuntu 16.04 and ubuntu 18.04
> So share the apt-get upgrade log.
> 
> pino@pino-dev:~$ sudo apt-get upgrade 
...
> Setting up bind9-host (1:9.11.3+dfsg-1ubuntu1.5) ...
> Setting up dnsutils (1:9.11.3+dfsg-1ubuntu1.5) ...
> Processing triggers for libc-bin (2.27-3ubuntu1) ...

Hello,

I'm using latest updates for Ubuntu. Please try the next scenario to reconstruct this issue:

1. Clone Yocto Thud to some directory, for example ./myproject
2. Clone https://github.com/advancedtelematic/meta-updater.git layer to the same directory and change directory to it. Do not use checkout to 'thud' branch (it will work in case of thud branch). Use master branch for this check (default after clone).
3. Add next line `BBLAYERS += "${METADIR}/meta-updater"` to bblayers.conf
4. Run bitbake or bitbake-layers to see the crash mentioned in the bug description (the crash not expected to appear in this case).

-----------------
5. Go to meta-updater directory and checkout to 'thud' tag. Restart bitbake-layers - it will not crash in this case (expected)

Thanks
Comment 6 sungwon.pino 2019-02-25 16:55:07 UTC
(In reply to comment #5)
> (In reply to comment #4)
> > Hello
> > 
> > bitbake-layer work well, before I type `apt-get upgrade` 
> > both ubuntu 16.04 and ubuntu 18.04
> > So share the apt-get upgrade log.
> > 
> > pino@pino-dev:~$ sudo apt-get upgrade 
> ...
> > Setting up bind9-host (1:9.11.3+dfsg-1ubuntu1.5) ...
> > Setting up dnsutils (1:9.11.3+dfsg-1ubuntu1.5) ...
> > Processing triggers for libc-bin (2.27-3ubuntu1) ...
> 
> Hello,
> 
> I'm using latest updates for Ubuntu. Please try the next scenario to
> reconstruct this issue:
> 
> 1. Clone Yocto Thud to some directory, for example ./myproject
> 2. Clone https://github.com/advancedtelematic/meta-updater.git layer to the
> same directory and change directory to it. Do not use checkout to 'thud'
> branch (it will work in case of thud branch). Use master branch for this
> check (default after clone).
> 3. Add next line `BBLAYERS += "${METADIR}/meta-updater"` to bblayers.conf
> 4. Run bitbake or bitbake-layers to see the crash mentioned in the bug
> description (the crash not expected to appear in this case).
> 
> -----------------
> 5. Go to meta-updater directory and checkout to 'thud' tag. Restart
> bitbake-layers - it will not crash in this case (expected)
> 
> Thanks

you right.
but in my case master and thud branch
are return same error log.
you mentioned.

<branch : master & thud>
1. git clone poky
2. git clone meta-updater in poky directory.
3. source oe-init-build-env to set bitbake env.
4. bitbake-layer add-layer ../meta-updater (work fine, it's ok)
5. check bblayer.conf added meta-updater.
6. bitbake-layer add-layer ../meta-updater again.
6. issue occur.
Comment 7 Chris Laplante 2019-10-05 22:46:56 UTC
I hit this today. The problem is that the actual errors are getting logged to the "BitBake.Parsing" logging namespace, which the bitbake-layers isn't exposing. The errors for me at least were still printed to bitbake-cookerdaemon.log.
Comment 8 Ross Burton 2020-09-21 02:56:51 UTC
*** Bug 14053 has been marked as a duplicate of this bug. ***
Comment 9 Andrei Gherzan 2021-11-21 13:36:51 UTC
This issue was fixed in master. To be more specific, the fixes firstly landed in  `2020-10-gatesgarth`. There are three patches that fix two sides of this issue:
1. fixes surfacing the server logs on parsing error
2. avoids `traceback` on `bb.cooker.CollectionError` - fixed logs above will be clear enough

On dunfell (bitbake 1.46)  three patches are needed:

86b24016 command: Ensure exceptions inheriting from BBHandledException are visible
e40fc16a tinfoil: When sending commands we need to process events
579f6896 process/knotty: Improve early exception handling

This report, as far as I can see, was raised against `thud` but I haven't checked how these patches apply against that release.
Comment 10 Richard Purdie 2021-11-21 13:41:12 UTC
Thanks Andrei, reassigning to Steve so we can fix this in dunfell and then close the bug as it was already fixed in master.
Comment 11 Andrei Gherzan 2021-11-21 13:48:59 UTC
(In reply to comment #9)
> This issue was fixed in master. To be more specific, the fixes firstly
> landed in  `2020-10-gatesgarth`. There are three patches that fix two sides
> of this issue:
> 1. fixes surfacing the server logs on parsing error
> 2. avoids `traceback` on `bb.cooker.CollectionError` - fixed logs above will
> be clear enough
> 
> On dunfell (bitbake 1.46)  three patches are needed:
> 
> 86b24016 command: Ensure exceptions inheriting from BBHandledException are
> visible
> e40fc16a tinfoil: When sending commands we need to process events
> 579f6896 process/knotty: Improve early exception handling
> 
> This report, as far as I can see, was raised against `thud` but I haven't
> checked how these patches apply against that release.

The git hashes I've posted above were wrong (as they were from a branch with cherry-picked commits). Here are the right SHAs in the bitbake repository:

ef762d92 command: Ensure exceptions inheriting from BBHandledException are visible
64ae9d7e tinfoil: When sending commands we need to process events
6059d0e7 process/knotty: Improve early exception handling
Comment 12 Steve Sakoman 2021-12-13 23:18:18 UTC
Fixed in dunfell/1.46

7fdd43c5 process/knotty: Improve early exception handling
f20da524 tinfoil: When sending commands we need to process events
80348b68 command: Ensure exceptions inheriting from BBHandledException are visible