Bug 7076

Summary:	missing python modules in the 1.7 buildtools tarball
Product:	[Build System, Metadata & Runtime] OE-Core	Reporter:	Nitin Kamble <nitin.a.kamble>
Component:	devtools / tool chain	Assignee:	Saul Wold <sgw>
Status:	RESOLVED FIXED	QA Contact:
Severity:	normal
Priority:	Medium	CC:	bluelightning, meta.mr.watcher, meta.watcher, richard.purdie
Version:	1.6.3
Target Milestone:	1.7.1
Hardware:	All
OS:	Multiple
Whiteboard:
OS type for building Yocto:	---	Type of Regression:	Regression (Used to work)
Verified:		Documentation change:	Done (doc changes complete)

Description Nitin Kamble 2014-12-11 16:24:09 UTC

The 1.7 buildtools are installed on the system from here: 
http://autobuilder.yoctoproject.org/pub/releases/yocto-1.7.rc4/buildtools/poky-glibc-x86_64-buildtools-tarball-core2-64-buildtools-nativesdk-standalone-1.7.sh

And this build environment script is sourced being building.
. /opt/poky/1.7/environment-setup-x86_64-pokysdk-linux

Current head of poky master: 8657c2fd57600e0b0eaf2a14692de7453c78e1bb

With this setup bitbake throws errors like this

ImportError: No module named htmlentitydefs



. oe-init-build-env build-dir
bitbake core-image-sato

Traceback (most recent call last):
  File "/home/nitin/build-test-bsps/poky.git/bitbake/bin/bitbake", line 34, in <module>
    import bb
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/__init__.py", line 77, in <module>
    from bb import fetch2 as fetch
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/fetch2/__init__.py", line 1565, in <module>
    from . import wget
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/fetch2/wget.py", line 40, in <module>
    from   bs4 import BeautifulSoup
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bs4/__init__.py", line 30, in <module>
    from .builder import builder_registry, ParserRejectedMarkup
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bs4/builder/__init__.py", line 4, in <module>
    from bs4.element import (
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bs4/element.py", line 5, in <module>
    from bs4.dammit import EntitySubstitution
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bs4/dammit.py", line 11, in <module>
    from htmlentitydefs import codepoint2name
ImportError: No module named htmlentitydefs
/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/event.py:111: RuntimeWarning: Parent module 'bb' not found while handling absolute import
  from bb.msg import BBLogFormatter
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/opt/poky/1.7/sysroots/x86_64-pokysdk-linux/usr/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/event.py", line 111, in print_ui_queue
    from bb.msg import BBLogFormatter
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/__init__.py", line 77, in <module>
    from bb import fetch2 as fetch
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/fetch2/__init__.py", line 39, in <module>
    from bb import data
ImportError: cannot import name data
Error in sys.exitfunc:
Traceback (most recent call last):
  File "/opt/poky/1.7/sysroots/x86_64-pokysdk-linux/usr/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/event.py", line 111, in print_ui_queue
    from bb.msg import BBLogFormatter
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/__init__.py", line 77, in <module>
    from bb import fetch2 as fetch
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/fetch2/__init__.py", line 39, in <module>
    from bb import data
ImportError: cannot import name data
Traceback (most recent call last):
  File "/home/nitin/build-test-bsps/poky.git/bitbake/bin/bitbake", line 34, in <module>
    import bb
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/__init__.py", line 77, in <module>
    from bb import fetch2 as fetch
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/fetch2/__init__.py", line 1565, in <module>
    from . import wget
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/fetch2/wget.py", line 40, in <module>
    from   bs4 import BeautifulSoup
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bs4/__init__.py", line 30, in <module>
    from .builder import builder_registry, ParserRejectedMarkup
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bs4/builder/__init__.py", line 4, in <module>
    from bs4.element import (
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bs4/element.py", line 5, in <module>
    from bs4.dammit import EntitySubstitution
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bs4/dammit.py", line 11, in <module>
    from htmlentitydefs import codepoint2name
ImportError: No module named htmlentitydefs
/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/event.py:111: RuntimeWarning: Parent module 'bb' not found while handling absolute import
  from bb.msg import BBLogFormatter
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/opt/poky/1.7/sysroots/x86_64-pokysdk-linux/usr/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/event.py", line 111, in print_ui_queue
    from bb.msg import BBLogFormatter
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/__init__.py", line 77, in <module>
    from bb import fetch2 as fetch
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/fetch2/__init__.py", line 39, in <module>
    from bb import data
ImportError: cannot import name data
Error in sys.exitfunc:
Traceback (most recent call last):
  File "/opt/poky/1.7/sysroots/x86_64-pokysdk-linux/usr/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/event.py", line 111, in print_ui_queue
    from bb.msg import BBLogFormatter
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/__init__.py", line 77, in <module>
    from bb import fetch2 as fetch
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/fetch2/__init__.py", line 39, in <module>
    from bb import data
ImportError: cannot import name data

Comment 1 Richard Purdie 2014-12-11 16:35:21 UTC

Fix backported into the 1.7 branch for 1.7.1:

http://git.yoctoproject.org/cgit.cgi/poky/commit/?h=dizzy&id=02627ad3d93e8221daf5d178c096ab48a9d1fd68

Comment 2 Nitin Kamble 2014-12-11 16:46:35 UTC

Looks like there is an opportunity for enhancing autobuilder setup here. If AB can use the current buildtools-tarball, then this kind of issues will get resolved before the release.

Comment 3 Paul Eggleton 2014-12-15 09:49:29 UTC

We don't want all of the autobuilders using the buildtools, only those that need them installed, or we won't actually be testing the environment that most users will be using our build system under. We have a CentOS 6 autobuilder, and I would have to assume that buildtools is running on there (it wouldn't work otherwise).

Note that the change that caused the htmlentitydefs module to be required came in _after_ 1.7.