Bug 13869 - icu_66.1.bb:do_make_icudata Segmentation Fault in Parallel Builds
Summary: icu_66.1.bb:do_make_icudata Segmentation Fault in Parallel Builds
Status: RESOLVED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: oe-core other (show other bugs)
Version: unspecified
Hardware: x86 Multiple
: Medium+ normal
Target Milestone: 3.2 M2
Assignee: Ross Burton
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2020-04-15 00:12 UTC by Hongxu Jia
Modified: 2020-08-04 02:10 UTC (History)
4 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Hongxu Jia 2020-04-15 00:12:25 UTC
Since commit [icu: Add knobs to generate a subset of ICU data][1]applied,
It caused Segmentation fault while executing genrb to generate *.res files

1473 LD_LIBRARY_PATH=../lib:../stubdata:../tools/ctestfw:$LD_LIBRARY_PATH  ../bin/genrb -s ./misc -d ./out/build/icudt66l -i ./out/build/icudt66l -k -q pluralRanges.txt
...
1577 make[1]: *** [../data/rules.mk:732: out/build/icudt66l/pluralRanges.res] Segmentation fault (core dumped)
1578 make[1]: *** Waiting for unfinished jobs....


[1]https://git.openembedded.org/openembedded-core/commit/?id=5e5be67744d7ddf5a9ac433ecba02f697a84a325
Comment 2 Khem Raj 2020-04-16 15:53:10 UTC
(In reply to comment #1)
> With a simple google, similar issue of icu
> 
> https://unicode-org.atlassian.net/browse/ICU-
> 13595?attachmentOrder=asc&oldIssueView=true
> 
> https://bugzilla.mozilla.org/show_bug.cgi?id=869406
> 
> https://sourceforge.net/p/icu/mailman/message/12529195/

does disabling parallel make help ?
Comment 3 Richard Purdie 2020-04-20 08:38:43 UTC
Also seen here: https://autobuilder.yoctoproject.org/typhoon/#/builders/44/builds/1816 (step 3b)
Comment 4 Hongxu Jia 2020-04-20 19:07:34 UTC
(In reply to comment #2)
> (In reply to comment #1)
> > With a simple google, similar issue of icu
> > 
> > https://unicode-org.atlassian.net/browse/ICU-
> > 13595?attachmentOrder=asc&oldIssueView=true
> > 
> > https://bugzilla.mozilla.org/show_bug.cgi?id=869406
> > 
> > https://sourceforge.net/p/icu/mailman/message/12529195/
> 
> does disabling parallel make help ?

I was trying it locally on wind river linux, if it works, I will send
the fix to community

--- a/meta/recipes-support/icu/icu_66.1.bb
+++ b/meta/recipes-support/icu/icu_66.1.bb
@@ -55,7 +55,7 @@ do_make_icudata_class-target () {
     LDFLAGS='${BUILD_LDFLAGS}' \
     ICU_DATA_FILTER_FILE=${WORKDIR}/filter.json \
     ./runConfigureICU Linux --with-data-packaging=archive
-    oe_runmake ${PARALLEL_MAKE}
+    oe_runmake
     install -Dm644 ${S}/data/out/icudt${ICU_MAJOR_VER}l.dat ${S}/data/in/icudt${ICU_MAJOR_VER}l.dat
 }
Comment 5 Steve Sakoman 2020-05-06 07:36:24 UTC
I get intermittent failures on dunfell autobuilder builds of icu, likely related:

https://errors.yoctoproject.org/Errors/Details/410500/
Comment 8 Ross Burton 2020-06-17 03:50:38 UTC
Hongxu: did disabling parallel make help?
Comment 9 Hongxu Jia 2020-06-17 04:04:40 UTC
(In reply to comment #8)
> Hongxu: did disabling parallel make help?

Yes, the issue was not found any more
Comment 10 Richard Purdie 2020-06-18 07:20:20 UTC
We can add PARALLEL_MAKE_task-make-icudata = "" 
to the recipe to disable parallel make in this task.

Execution time for the task goes from 31s to 328s which is my view is unacceptable though, we need to fix this properly.
Comment 11 Randy MacLeod 2020-06-18 08:17:26 UTC
Hongxu, Could you (and Robert?) take a look at the build to try to find and fix the race condition?
Comment 12 Randy MacLeod 2020-06-18 08:20:10 UTC
Actually Ross is going to work on it.
Comment 13 Ross Burton 2020-06-18 11:07:40 UTC
Got a patch that removes the parallel building but also changes a lot of the build so the performance hit isn't as bad.
Comment 14 Richard Purdie 2020-07-06 15:29:07 UTC
Dropping the AB-INT tag since this is now disabled from the autobuilder perspective (but is still an issue we need to fix).
Comment 15 Ross Burton 2020-08-04 02:10:44 UTC
oe-core df89e8d1136fd406ba35ae573e2cb0cfc88c6aad mostly fixed this.