3328 2012-10-23 17:43:14 +0000 sstate: show some progress when downloading lots of packages from SSTATE_MIRROR 2013-03-26 09:02:03 +0000 1 1 7 Build System, Metadata & Runtime OE-Core core 1.4 x86 Multiple VERIFIED FIXED Medium enhancement 1.4 1 Martin.Jansa richard.purdie corneliux.stoicescu meta.mr.watcher meta.watcher richard.purdie --- --- --- oldest_to_newest 26847 0 Martin.Jansa 2012-10-23 17:43:14 +0000 19:31:34 < JaMa> bluelightning: is this worth enhancement bug report: with SSTATE_MIRROR fully populated, bitbake hangs on "NOTE: Preparing runqueue" for really long time without showing any output - downloading lots of sstate packages on background 19:32:18 < kergoth> interesting. ideally we'd have some sort of progress indicator like the parsing, probably.. 19:33:04 < bluelightning> JaMa: yes, I think it should show something to let you know what's happening in that case 19:38:33 < JaMa> kergoth: maybe progress bar is not enough, as some packages are very quick, but e.g. 2GB webkit should be probably shown alone 19:38:49 < JaMa> kergoth: but on otherside another Download queue is probably too much 19:39:15 < JaMa> maybe something like knotty2 does with showing current task to show current package being downloaded In my case (slower network, big sstate-cache - 25GB) it will show "NOTE: Preparing runqueue" for few hours and only indicator that it's still working is wget showing up in top. 26854 1 Martin.Jansa 2012-10-24 08:17:55 +0000 It is still running (showing "NOTE: Preparing runqueue") now :). So I've noticed that you cannot even stop it with Ctrl+C (tried 5 times). 27360 2 richard.purdie 2012-11-12 15:56:45 +0000 Did http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=44c5d3dc845988be9a11df6e0c8aef51377aef49 mean this at least shows there is a task running now? 27458 3 Martin.Jansa 2012-11-15 00:12:53 +0000 (In reply to comment #2) > Did > http://git.yoctoproject.org/cgit.cgi/poky/commit/ > ?id=44c5d3dc845988be9a11df6e0c8aef51377aef49 mean this at least shows there > is a task running now? No it hangs the same and still cannot be killed by Ctrl+C. 28296 4 richard.purdie 2012-12-14 14:40:00 +0000 I'm pretty sure this was actually caused by http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=a0cf759537260f00311450e0dac41bf45b474454 and this issue should be resolved now. The download will happen in the actually running task, not during the runqueue initilisation. 28316 5 Martin.Jansa 2012-12-16 21:19:20 +0000 (In reply to comment #4) > I'm pretty sure this was actually caused by > http://git.yoctoproject.org/cgit.cgi/poky/commit/ > ?id=a0cf759537260f00311450e0dac41bf45b474454 and this issue should be > resolved now. The download will happen in the actually running task, not > during the runqueue initilisation. Yes it is much better now, but still hangs on ~ 10 mins without any output according to strace doing mostly "stat sstate-cache/foo" and also in only one thread. I've tried it on big build (world with many layers, cca 6500 setscene tasks, 23000 normal tasks) but also SSTATE_MIRRORS points to the same machine (over http:// for test). I don't think that every "stat" should be shown on screen, but is there way to see what's going on at least with -D or -v? Maybe something like: Looking for sstate-cache files: .... and add dot for every archive tried (or every 10th, 100th) 30897 6 corneliux.stoicescu 2013-03-20 15:07:19 +0000 The download is no longer done during "NOTE: Preparing runqueue" phase. The apparent "hang" is gone. Verified in master:e89f812a0af56beda951c1dce6113a61d6db8cda 31164 7 corneliux.stoicescu 2013-03-26 09:02:03 +0000 Because of the way this was implemented, we will not be making a test case to test this enhancement at this time.