Happens in Ostro CI builders sometimes: (1st column is time elapsed from build start) 00:04:11.609 NOTE: Running task 1237 of 6419 (/srv/jenkins/ostro-bld-07-slot-3-ugeKN/ostro-os/meta-java/recipes-core/openjdk/openjdk-8-native_102b14.bb:do_fetch) 00:04:11.832 ERROR: local variable 'value' referenced before assignment 00:04:11.840 ERROR: Task (/srv/jenkins/ostro-bld-07-slot-3-ugeKN/ostro-os/meta-java/recipes-core/openjdk/openjdk-8-native_102b14.bb:do_fetch) failed with exit code '1' Full log: https://ostroproject.org/jenkins/view/Build/job/build_beaglebone/2724/console Don't know how to reproduce in local build. Frequency: 20 times in recorded 2000 builds. First time seen: Jul-22-2016
Is it always during openjdk-8-native_102b14.bb:do_fetch ?
Olev, would you add the full log to the attachments of this bug?
(In reply to comment #1) > Is it always during openjdk-8-native_102b14.bb:do_fetch ? yes, based on 20 existing logs, all cases are about openjdk-8-native. Distribution by target machines (Ostro CI builds for 4 machines): 7 x beaglebone 9 x edison 2 x intel-corei7 2 x intel-quark
Created attachment 3474 [details] console log from failing build
observation: I cannot reproduce the problem in the same workspace on a builder after such failure. Tried with SSTATE, also cleaned SSTATE.
Looking to logs from failed builds: Below WARNING is reported earlier in build log in case of failure. In good build, there is no such warning reported. WARNING: Setscene task .../meta-java/recipes-core/openjdk/openjdk-8-native_72b05.bb:do_populate_sysroot (.../meta-java/recipes-core/openjdk/openjdk-8-native_72b05.bb:do_populate_sysroot_setscene) failed with exit code '1' - real task will be run instead
one of (perhaps key) questions in this puzzle is, why isn't trace shown. One strange thing can be seen from logs: Seems logging of this ERROR happens differently than errors usually. CI runs bitbake build stage with "bitbake TARGETS | tee -a logfile" (which is actually risky as it does not put stderr into logfile) However we dont totally lose stderr because Jenkins captures it in its log. In most cases, logfile and Jenkins job log show same content, i.e. trace of build, plus error and backtrace. In the case of this bug failures, its different: two ERROR lines do not appear in logfile by tee, but are present in Jenkins job log. That is sign that these 2 ERROR lines were only sent to stderr, which is different behavior from usual (everything sent to stdout). That does not directly explain missing backtrace yet, but is there something not as planned with logging?
This was a bit annoying bug because of the missing backtrace. I sent patches for review: https://patchwork.openembedded.org/series/3696/ Olev: there's a problem in the openjdk-8-native recipe in ostro. d.getVar("CFLAGS") fails in the bitbake worker context and the recipe should be fixed.
Thanks! You mean actual root cause of failure is use of CFLAGS in openjdk-8-native_102b14.bb ? And more specifically, thats about --with-extra-cflags='${CFLAGS}' in meta-java/recipes-core/openjdk/openjdk-8-native.inc, right? But what about other variables used same way: CXXFLAGS, LDFLAGS ? The block using --with-extra-cflags for CFLAGS, CXXFLAGS, LDFLAGS was added 2016-06-22 (see commit d0282c48870790d0c87d2c6b67b15ac59f566adc) which matches start of failure in Ostro builds
(In reply to comment #9) > Thanks! > You mean actual root cause of failure is use of CFLAGS > in openjdk-8-native_102b14.bb ? Basically, yes. > And more specifically, thats about > --with-extra-cflags='${CFLAGS}' > in meta-java/recipes-core/openjdk/openjdk-8-native.inc, right? > > But what about other variables used same way: CXXFLAGS, LDFLAGS ? If you apply my patches you will get a warning from all the problematic variables. In my environment CXXFLAGS fails similarly to CFLAGS but LDFLAGS doesn't raise a warning.
Fix merged in: http://git.openembedded.org/bitbake/commit/?id=f639f06cfa280adcc25438387567966271b9b2c3
Ostro CI does not run (and can't be run) any more, so it's not possible to verify in original setup on which this report was based. Refkit CI (successor of Ostro CI) does not build openjdk, cant check there either. I think we can close this issue.