Bug 13869

Summary: icu_66.1.bb:do_make_icudata Segmentation Fault in Parallel Builds
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Hongxu Jia <hongxu.jia>
Component: oe-core otherAssignee: Ross Burton <ross.burton>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium+ CC: randy.macleod, richard.purdie, ross.burton, steve
Version: unspecified   
Target Milestone: 3.2 M2   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Hongxu Jia 2020-04-15 00:12:25 UTC 
Since commit [icu: Add knobs to generate a subset of ICU data][1]applied,
It caused Segmentation fault while executing genrb to generate *.res files

1473 LD_LIBRARY_PATH=../lib:../stubdata:../tools/ctestfw:$LD_LIBRARY_PATH  ../bin/genrb -s ./misc -d ./out/build/icudt66l -i ./out/build/icudt66l -k -q pluralRanges.txt
...
1577 make[1]: *** [../data/rules.mk:732: out/build/icudt66l/pluralRanges.res] Segmentation fault (core dumped)
1578 make[1]: *** Waiting for unfinished jobs....


[1]https://git.openembedded.org/openembedded-core/commit/?id=5e5be67744d7ddf5a9ac433ecba02f697a84a325
Comment 2 Khem Raj 2020-04-16 15:53:10 UTC 
(In reply to comment #1)
> With a simple google, similar issue of icu
> 
> https://unicode-org.atlassian.net/browse/ICU-
> 13595?attachmentOrder=asc&oldIssueView=true
> 
> https://bugzilla.mozilla.org/show_bug.cgi?id=869406
> 
> https://sourceforge.net/p/icu/mailman/message/12529195/

does disabling parallel make help ?
Comment 3 Richard Purdie 2020-04-20 08:38:43 UTC 
Also seen here: https://autobuilder.yoctoproject.org/typhoon/#/builders/44/builds/1816 (step 3b)
Comment 4 Hongxu Jia 2020-04-20 19:07:34 UTC 
(In reply to comment #2)
> (In reply to comment #1)
> > With a simple google, similar issue of icu
> > 
> > https://unicode-org.atlassian.net/browse/ICU-
> > 13595?attachmentOrder=asc&oldIssueView=true
> > 
> > https://bugzilla.mozilla.org/show_bug.cgi?id=869406
> > 
> > https://sourceforge.net/p/icu/mailman/message/12529195/
> 
> does disabling parallel make help ?

I was trying it locally on wind river linux, if it works, I will send
the fix to community

--- a/meta/recipes-support/icu/icu_66.1.bb
+++ b/meta/recipes-support/icu/icu_66.1.bb
@@ -55,7 +55,7 @@ do_make_icudata_class-target () {
     LDFLAGS='${BUILD_LDFLAGS}' \
     ICU_DATA_FILTER_FILE=${WORKDIR}/filter.json \
     ./runConfigureICU Linux --with-data-packaging=archive
-    oe_runmake ${PARALLEL_MAKE}
+    oe_runmake
     install -Dm644 ${S}/data/out/icudt${ICU_MAJOR_VER}l.dat ${S}/data/in/icudt${ICU_MAJOR_VER}l.dat
 }
Comment 5 Steve Sakoman 2020-05-06 07:36:24 UTC 
I get intermittent failures on dunfell autobuilder builds of icu, likely related:

https://errors.yoctoproject.org/Errors/Details/410500/
Comment 8 Ross Burton 2020-06-17 03:50:38 UTC 
Hongxu: did disabling parallel make help?
Comment 9 Hongxu Jia 2020-06-17 04:04:40 UTC 
(In reply to comment #8)
> Hongxu: did disabling parallel make help?

Yes, the issue was not found any more
Comment 10 Richard Purdie 2020-06-18 07:20:20 UTC 
We can add PARALLEL_MAKE_task-make-icudata = "" 
to the recipe to disable parallel make in this task.

Execution time for the task goes from 31s to 328s which is my view is unacceptable though, we need to fix this properly.
Comment 11 Randy MacLeod 2020-06-18 08:17:26 UTC 
Hongxu, Could you (and Robert?) take a look at the build to try to find and fix the race condition?
Comment 12 Randy MacLeod 2020-06-18 08:20:10 UTC 
Actually Ross is going to work on it.
Comment 13 Ross Burton 2020-06-18 11:07:40 UTC 
Got a patch that removes the parallel building but also changes a lot of the build so the performance hit isn't as bad.
Comment 14 Richard Purdie 2020-07-06 15:29:07 UTC 
Dropping the AB-INT tag since this is now disabled from the autobuilder perspective (but is still an issue we need to fix).
Comment 15 Ross Burton 2020-08-04 02:10:44 UTC 
oe-core df89e8d1136fd406ba35ae573e2cb0cfc88c6aad mostly fixed this.