| Summary: | pseudo segfault in the 2nd build from hob | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [Yocto Project Subprojects] Pseudo | Reporter: | Jessica <jessica.zhang> | ||||
| Component: | pseudo | Assignee: | Paul Eggleton <bluelightning> | ||||
| Status: | RESOLVED FIXED | QA Contact: | |||||
| Severity: | critical | ||||||
| Priority: | High | CC: | josh, richard.purdie, yp.pseudo.watcher, yp.watcher | ||||
| Version: | unspecified | ||||||
| Target Milestone: | 1.1 | ||||||
| Hardware: | x86 | ||||||
| OS: | Multiple | ||||||
| Whiteboard: | Fix posted for review (5 sep) | ||||||
| OS type for building Yocto: | --- | Type of Regression: | --- | ||||
| Verified: | Documentation change: | --- | |||||
| Bug Depends on: | |||||||
| Bug Blocks: | 1285 | ||||||
| Attachments: |
|
||||||
|
Description
Jessica
2011-08-04 16:40:12 UTC
seems the core file is too big to be attached.
Also, make sure you have following patch in the code:
$ git diff
diff --git a/lib/bb/ui/hob.py b/lib/bb/ui/hob.py
index b5c2342..8af7740 100644
--- a/lib/bb/ui/hob.py
+++ b/lib/bb/ui/hob.py
@@ -446,6 +446,7 @@ class MainWindow (gtk.Window):
self.nb.set_current_page(0)
def build_complete_cb(self, running_build):
+ self.handler.building = None
self.stopping = False
self.back.connect("clicked", self.back_button_clicked_cb)
self.back.set_sensitive(True)
This is not looking like a pseudo problems, at least not directly.
The cooker.log from the first "good" run:
OE Build Configuration:
BB_VERSION = "1.13.3"
TARGET_ARCH = "arm"
TARGET_OS = "linux-gnueabi"
MACHINE = "qemuarm"
DISTRO = ""
DISTRO_VERSION = ""
TUNE_FEATURES = "armv5 dsp thumb arm926ejs"
TARGET_FPU = "soft"
meta = "mhatle/fix_1352:68661582ede47f7b70999764ea80fe3e5e6aa78d"
The log from the subsequent run:
pseudo: You must set the PSEUDO_PREFIX environment variable to run pseudo.
pseudo: You must set the PSEUDO_PREFIX environment variable to run pseudo.
pseudo: You must set the PSEUDO_PREFIX environment variable to run pseudo.
OE Build Configuration:
BB_VERSION = "1.13.3"
TARGET_ARCH = "arm"
TARGET_OS = "linux-gnueabi"
MACHINE = "qemuarm"
DISTRO = ""
DISTRO_VERSION = ""
TUNE_FEATURES = "armv5 dsp thumb arm926ejs"
TARGET_FPU = "soft"
meta = "<unknown>:<unknown>"
The three pseudo messages indicate that the environment was cleared and pseudo was trying to be run way too early in the build. The following messages indicate that the meta data couldn't be found correctly..
The core indicates the failure occurred in: (note pseudo should be in memory, but inactive at this point)
python /home/mhatle/git/oss/oe-core/bitbake/bin/bitbake -R conf/hob.local.conf
The backtrace:
#0 0x0000003c534966f8 in tupleitem (a=0x3689890, i=0) at Objects/tupleobject.c:337
#1 0x0000003c534970e0 in tuplesubscript (self=0x3689890, item=<value optimized out>)
at Objects/tupleobject.c:658
#2 0x0000003c534db99b in PyEval_EvalFrameEx (f=<value optimized out>,
throwflag=<value optimized out>) at Python/ceval.c:1294
#3 0x0000003c534df4e9 in PyEval_EvalCodeEx (co=0x1936990,
globals=<value optimized out>, locals=<value optimized out>,
args=<value optimized out>, argcount=2, kws=0x3725248, kwcount=0, defs=
0x19fe1e8, defcount=1, closure=0x0) at Python/ceval.c:3026
#4 0x0000003c534dd897 in fast_function (f=<value optimized out>,
throwflag=<value optimized out>) at Python/ceval.c:3860
#5 call_function (f=<value optimized out>, throwflag=<value optimized out>)
at Python/ceval.c:3785
#6 PyEval_EvalFrameEx (f=<value optimized out>, throwflag=<value optimized out>)
at Python/ceval.c:2445
#7 0x0000003c534de312 in fast_function (f=<value optimized out>,
throwflag=<value optimized out>) at Python/ceval.c:3850
#8 call_function (f=<value optimized out>, throwflag=<value optimized out>)
at Python/ceval.c:3785
#9 PyEval_EvalFrameEx (f=<value optimized out>, throwflag=<value optimized out>)
at Python/ceval.c:2445
#10 0x0000003c534df4e9 in PyEval_EvalCodeEx (co=0x190e558,
globals=<value optimized out>, locals=<value optimized out>,
args=<value optimized out>, argcount=4, kws=0x0, kwcount=0, defs=0x0,
defcount=0, closure=0x0) at Python/ceval.c:3026
#11 0x0000003c5346df70 in function_call (func=<function at remote 0x191a0c8>, arg=
(<HobHandler(building='image', generating=False, image_dir='/tmp/hob-images', current_command=None, server=<ServerProxy(_ServerProxy__transport=<Transport(_use_datetime=0, verbose=0) at remote 0x365e7a0>, _ServerProxy__handler='/', _ServerProxy__host='127.0.0.1:50989', _ServerProxy__verbose=0, _ServerProxy__allow_none=True, _ServerProxy__encoding=None) at remote 0x365ef38>, build_toolchain_headers=False, build_toolchain=False, image_output_types=['tar.bz2', 'ext3'], model=<TaskListModel(packages=<unknown at remote 0x7fdf74268690>, images=<unknown at remote 0x7fdf74268730>, tasks=<unknown at remote 0x7fdf74268820>, contents=<unknown at remote 0x7fdf74268780>, selected_image='core-image-sato') at remote 0x3059500>, build_queue=[], current_phase='building') at remote 0x30595a0>, <BBUIEventQueue(EventHandle=1, eventQueueNotify=<_Event(_Verbose__verbose=False, _Event__flag=False, _Event__cond=<_Condition(_Verbose__verbose=False, _Condition__lock=<thread.lock at remote 0x1844b10>, acquire=<built-in method acquire o...(truncated), kw=0x0) at Objects/funcobject.c:524
#12 0x0000003c53443db3 in PyObject_Call (func=<function at remote 0x191a0c8>,
arg=<value optimized out>, kw=<value optimized out>) at Objects/abstract.c:2492
#13 0x0000003c53458cff in instancemethod_call (func=<function at remote 0x191a0c8>,
arg=
(<HobHandler(building='image', generating=False, image_dir='/tmp/hob-images', current_command=None, server=<ServerProxy(_ServerProxy__transport=<Transport(_use_datetime=0, verbose=0) at remote 0x365e7a0>, _ServerProxy__handler='/', _ServerProxy__host='127.0.0.1:50989', _ServerProxy__verbose=0, _ServerProxy__allow_none=True, _ServerProxy__encoding=None) at remote 0x365ef38>, build_toolchain_headers=False, build_toolchain=False, image_output_types=['tar.bz2', 'ext3'], model=<TaskListModel(packages=<unknown at remote 0x7fdf74268690>, images=<unknown at remote 0x7fdf74268730>, tasks=<unknown at remote 0x7fdf74268820>, contents=<unknown at remote 0x7fdf74268780>, selected_image='core-image-sato') at remote 0x3059500>, build_queue=[], current_phase='building') at remote 0x30595a0>, <BBUIEventQueue(EventHandle=1, eventQueueNotify=<_Event(_Verbose__verbose=False, _Event__flag=False, _Event__cond=<_Condition(_Verbose__verbose=False, _Condition__lock=<thread.lock at remote 0x1844b10>, acquire=<built-in method acquire o...(truncated), kw=0x0) at Objects/classobject.c:2579
#14 0x0000003c53443db3 in PyObject_Call (func=<instancemethod at remote 0x368b9b0>,
arg=<value optimized out>, kw=<value optimized out>) at Objects/abstract.c:2492
#15 0x0000003c534d7ba3 in PyEval_CallObjectWithKeywords (func=
<instancemethod at remote 0x368b9b0>, arg=
(<BBUIEventQueue(EventHandle=1, eventQueueNotify=<_Event(_Verbose__verbose=False, _Event__flag=False, _Event__cond=<_Condition(_Verbose__verbose=False, _Condition__lock=<thread.lock at remote 0x1844b10>, acquire=<built-in method acquire of thread.lock object at remote 0x1844b10>, _Condition__waiters=[], release=<built-in method release of thread.lock object at remote 0x1844b10>) at remote 0x1c81110>) at remote 0x1b85650>, eventQueueLock=<thread.lock at remote 0x1844af8>, port=32778, host='127.0.0.1', t=<Thread(_Thread__ident=140597806331648, run=<instancemethod at remote 0x30591e0>, _Thread__block=<_Condition(_Verbose__verbose=False, _Condition__lock=<thread.lock at remote 0x1844b40>, acquire=<built-in method acquire of thread.lock object at remote 0x1844b40>, _Condition__waiters=[], release=<built-in method release of thread.lock object at remote 0x1844b40>) at remote 0x32d3d10>, _Thread__name='Thread-1', _Thread__daemonic=True, _Thread__started=<_Event(_Verbose__verbose=False, _Event__flag=True, _Event__con...(truncated), kw=<value optimized out>) at Python/ceval.c:3633
#16 0x00007fdf7cdd51bb in ?? ()
#17 0x0000000001f38190 in ?? ()
#18 0x0000000000000000 in ?? ()
The code where the failure occured:
332 {
333 if (i < 0 || i >= Py_SIZE(a)) {
334 PyErr_SetString(PyExc_IndexError, "tuple index out of range");
335 return NULL;
336 }
337 Py_INCREF(a->ob_item[i]);
338 return a->ob_item[i];
339 }
a = 0x3689890, a->ob_item = 0x0, i = 0
Reassigning to Joshua, he indicated on IRC he would investigate the hob refresh issue. If this persists after that has been resolved, please reassign to me. This is confusing to me because the only code we * explicitly* call from hob which affects the environment is the reparse command, however this is run at least once before the first build and doesn't have an adverse effect. I wonder if there's some tear down done on build completion that unsets parts of the environment?
The sequence of commands for a typical build will consist of
i) buildTargetsTree
ii) resetCooker & reparseFiles (triggered at least once before the build when the custom recipe of the image contents is written to ensure it's included, may have been triggered more times by editing preferences etc)
iii) buildTargets
Commands are executed by the run_next_command() method of bb/ui/crumbs/hobeventhandler.py usually triggered by an explicit user action or a CommandCompleted event from the Cooker.
Once the build has completed no new command is run before the build again, which should treigger a reset & reparse before the next build - it's at this point that we see complaints from pseudo about the unset environment.
Debugging of this issue can be sped up with a full sstate cache and by turning do_rootfs into a noop for your test target, i.e:
diff --git a/meta/recipes-core/images/core-image-minimal.bb b/meta/recipes-core/images/core-image-minimal.bb
index 743e121..c339483 100644
--- a/meta/recipes-core/images/core-image-minimal.bb
+++ b/meta/recipes-core/images/core-image-minimal.bb
@@ -13,3 +13,7 @@ IMAGE_ROOTFS_SIZE = "8192"
# remove not needed ipkg informations
ROOTFS_POSTPROCESS_COMMAND += "remove_packaging_data_files ; "
+
+do_rootfs () {
+ exit 0
+}
Further a patch similar to the following will ensure that any errors are visible:
diff --git a/lib/bb/ui/hob.py b/lib/bb/ui/hob.py
index b0a98cb..999203f 100644
--- a/lib/bb/ui/hob.py
+++ b/lib/bb/ui/hob.py
@@ -134,6 +134,9 @@ class MainWindow (gtk.Window):
self.set_busy_cursor(False)
gtk.main_quit()
+ def command_failed_cb(self, handler, errormsg):
+ print(errormsg)
+
def scroll_tv_cb(self, model, path, it, view):
view.scroll_to_cell(path)
@@ -1003,6 +1007,7 @@ def main (server, eventHandler):
configurator.connect("layers-changed", handler.reload_data)
handler.connect("config-found", configurator.configFound)
handler.connect("fatal-error", window.fatal_error_cb)
+ handler.connect("command-failed", window.command_failed_cb)
try:
# kick the while thing off
Further debugging hints gleaned through discussing the bug with various folk. RP suggested the following may provide more insight: * Dumping the environment (os.environ) before each build * Blowing away the pseudo data dirs between builds * Killing the pseudo process between builds (PID can be found in the pseudo.pid file) Paul asked about how to get more debugging output from hob. attachment 201 [details] from bug 1285 includes some hints on how to print more debugging output. Created attachment 217 [details] Patch to aid debugging in hob Based on attachment 201 [details] from bug 1285 an untested patch against BitBake master to add more debugging output. So far I have been able to at least determine that pseudo isn't the cause of the crash - with pseudo hacked out of the bitbake script and the sanity.bbclass check commented out it still crashes. This would appear to suggest that we're somehow crashing the python interpreter, which is certainly unusual. Found the issue - we weren't clearing out some internal lists between builds. I have posted a couple of patches to the bitbake-devel list for review. Fixed in Poky commit a9325ecad889556b0043244509ed767eea363e1e. |