Bug 7076 - missing python modules in the 1.7 buildtools tarball
Summary: missing python modules in the 1.7 buildtools tarball
Status: RESOLVED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: devtools / tool chain (show other bugs)
Version: 1.6.3
Hardware: All Multiple
: Medium normal
Target Milestone: 1.7.1
Assignee: Saul Wold
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2014-12-11 16:24 UTC by Nitin Kamble
Modified: 2014-12-15 09:49 UTC (History)
4 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: Regression (Used to work)
Verified:
Documentation change: Done (doc changes complete)

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Nitin Kamble 2014-12-11 16:24:09 UTC 
The 1.7 buildtools are installed on the system from here: 
http://autobuilder.yoctoproject.org/pub/releases/yocto-1.7.rc4/buildtools/poky-glibc-x86_64-buildtools-tarball-core2-64-buildtools-nativesdk-standalone-1.7.sh

And this build environment script is sourced being building.
. /opt/poky/1.7/environment-setup-x86_64-pokysdk-linux

Current head of poky master: 8657c2fd57600e0b0eaf2a14692de7453c78e1bb

With this setup bitbake throws errors like this

ImportError: No module named htmlentitydefs



. oe-init-build-env build-dir
bitbake core-image-sato

Traceback (most recent call last):
  File "/home/nitin/build-test-bsps/poky.git/bitbake/bin/bitbake", line 34, in <module>
    import bb
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/__init__.py", line 77, in <module>
    from bb import fetch2 as fetch
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/fetch2/__init__.py", line 1565, in <module>
    from . import wget
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/fetch2/wget.py", line 40, in <module>
    from   bs4 import BeautifulSoup
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bs4/__init__.py", line 30, in <module>
    from .builder import builder_registry, ParserRejectedMarkup
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bs4/builder/__init__.py", line 4, in <module>
    from bs4.element import (
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bs4/element.py", line 5, in <module>
    from bs4.dammit import EntitySubstitution
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bs4/dammit.py", line 11, in <module>
    from htmlentitydefs import codepoint2name
ImportError: No module named htmlentitydefs
/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/event.py:111: RuntimeWarning: Parent module 'bb' not found while handling absolute import
  from bb.msg import BBLogFormatter
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/opt/poky/1.7/sysroots/x86_64-pokysdk-linux/usr/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/event.py", line 111, in print_ui_queue
    from bb.msg import BBLogFormatter
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/__init__.py", line 77, in <module>
    from bb import fetch2 as fetch
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/fetch2/__init__.py", line 39, in <module>
    from bb import data
ImportError: cannot import name data
Error in sys.exitfunc:
Traceback (most recent call last):
  File "/opt/poky/1.7/sysroots/x86_64-pokysdk-linux/usr/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/event.py", line 111, in print_ui_queue
    from bb.msg import BBLogFormatter
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/__init__.py", line 77, in <module>
    from bb import fetch2 as fetch
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/fetch2/__init__.py", line 39, in <module>
    from bb import data
ImportError: cannot import name data
Traceback (most recent call last):
  File "/home/nitin/build-test-bsps/poky.git/bitbake/bin/bitbake", line 34, in <module>
    import bb
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/__init__.py", line 77, in <module>
    from bb import fetch2 as fetch
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/fetch2/__init__.py", line 1565, in <module>
    from . import wget
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/fetch2/wget.py", line 40, in <module>
    from   bs4 import BeautifulSoup
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bs4/__init__.py", line 30, in <module>
    from .builder import builder_registry, ParserRejectedMarkup
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bs4/builder/__init__.py", line 4, in <module>
    from bs4.element import (
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bs4/element.py", line 5, in <module>
    from bs4.dammit import EntitySubstitution
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bs4/dammit.py", line 11, in <module>
    from htmlentitydefs import codepoint2name
ImportError: No module named htmlentitydefs
/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/event.py:111: RuntimeWarning: Parent module 'bb' not found while handling absolute import
  from bb.msg import BBLogFormatter
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/opt/poky/1.7/sysroots/x86_64-pokysdk-linux/usr/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/event.py", line 111, in print_ui_queue
    from bb.msg import BBLogFormatter
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/__init__.py", line 77, in <module>
    from bb import fetch2 as fetch
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/fetch2/__init__.py", line 39, in <module>
    from bb import data
ImportError: cannot import name data
Error in sys.exitfunc:
Traceback (most recent call last):
  File "/opt/poky/1.7/sysroots/x86_64-pokysdk-linux/usr/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/event.py", line 111, in print_ui_queue
    from bb.msg import BBLogFormatter
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/__init__.py", line 77, in <module>
    from bb import fetch2 as fetch
  File "/home/nitin/build-test-bsps/poky.git/bitbake/lib/bb/fetch2/__init__.py", line 39, in <module>
    from bb import data
ImportError: cannot import name data
Comment 1 Richard Purdie 2014-12-11 16:35:21 UTC 
Fix backported into the 1.7 branch for 1.7.1:

http://git.yoctoproject.org/cgit.cgi/poky/commit/?h=dizzy&id=02627ad3d93e8221daf5d178c096ab48a9d1fd68
Comment 2 Nitin Kamble 2014-12-11 16:46:35 UTC 
Looks like there is an opportunity for enhancing autobuilder setup here. If AB can use the current buildtools-tarball, then this kind of issues will get resolved before the release.
Comment 3 Paul Eggleton 2014-12-15 09:49:29 UTC 
We don't want all of the autobuilders using the buildtools, only those that need them installed, or we won't actually be testing the environment that most users will be using our build system under. We have a CentOS 6 autobuilder, and I would have to assume that buildtools is running on there (it wouldn't work otherwise).

Note that the change that caused the htmlentitydefs module to be required came in _after_ 1.7.