Bug 5239 - [autobuilder] u-boot does not build correctly
Summary: [autobuilder] u-boot does not build correctly
Status: RESOLVED FIXED
Alias: None
Product: BSPs
Classification: Build System, Metadata & Runtime
Component: bsps-meta-fsl-ppc (show other bugs)
Version: unspecified
Hardware: x86 Multiple
: Medium+ normal
Target Milestone: 1.5 M5
Assignee: Zhenhua Luo
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2013-09-19 17:53 UTC by Saul Wold
Modified: 2013-09-24 06:15 UTC (History)
4 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments
Log file of uboot failure (6.14 MB, application/octet-stream)
2013-09-19 17:53 UTC, Saul Wold
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Saul Wold 2013-09-19 17:53:30 UTC
Created attachment 1527 [details]
Log file of uboot failure

Failure on autobuilder see attachement
Comment 1 Otavio Salvador 2013-09-19 18:01:50 UTC
Zhenhua,

I don't have a full environment to reproduce the issue. Could you take a look at this?

Regards,
Comment 2 Chunrong Guo 2013-09-22 02:48:31 UTC
poky: commit 853bc53cd58a621918f0e5ce662dba263d1befb4
meta-fsl-ppc :commit 1ddf5e66c635e086864ce15931628ce1a2d6ff0a
branch master


I built u-boot for p1022ds and the issue cannot reproduce.

Build Configuration:
BB_VERSION        = "1.19.1"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "Fedora-13"
TARGET_SYS        = "powerpc-poky-linux-gnuspe"
MACHINE           = "p1022ds"
DISTRO            = "poky"
DISTRO_VERSION    = "1.4+snapshot-20130922"
TUNE_FEATURES     = "m32 spe ppce500v2"
TARGET_FPU        = "ppc-efd"
meta              
meta-yocto        
meta-yocto-bsp    = "master:853bc53cd58a621918f0e5ce662dba263d1befb4"
meta-oe           = "master:1d536390dcafe4d539335dec2173aa9ddc3d8b51"
meta-fsl-ppc      = "master:1ddf5e66c635e086864ce15931628ce1a2d6ff0a"
Comment 3 Zhenhua Luo 2013-09-22 07:08:15 UTC
The build also passed in recent Yocto AB report, please confirm. 
http://autobuilder.yoctoproject.org:8011/builders/nightly-fsl-ppc. 

Please provide more details on how to reproduce if the issue persists.
Comment 4 Richard Purdie 2013-09-22 15:50:34 UTC
It failed on the autobuilder again:

http://autobuilder.yoctoproject.org:8011/builders/nightly-fsl-ppc/builds/270/steps/Building%20Images/logs/stdio

In the logs it is clearly running:

"cat /dev/null /srv/home/pokybuild/yocto-autobuilder-new/yocto-slave/nightly-fsl-ppc/build/build/tmp/work/p1022ds-poky-linux-gnuspe/u-boot/git-r30/git/P1022DS_NAND/spl/arch/powerpc/cpu/mpc85xx/.depend.start /srv/home/pokybuild/yocto-autobuilder-new/yocto-slave/nightly-fsl-ppc/build/build/tmp/work/p1022ds-poky-linux-gnuspe/u-boot/git-r30/git/P1022DS_NAND/spl/arch/powerpc/cpu/mpc85xx/.depend.resetvec /srv/home/pokybuild/yocto-autobuilder-new/yocto-slave/nightly-fsl-ppc/build/build/tmp/work/p1022ds-poky-linux-gnuspe/u-boot/git-r30/git/P1022DS_NAND/spl/arch/powerpc/cpu/mpc85xx/.depend.cpu_init_early /srv/home/pokybuild/yocto-autobuilder-new/yocto-slave/nightly-fsl-ppc/build/build/tmp/work/p1022ds-poky-linux-gnuspe/u-boot/git-r30/git/P1022DS_NAND/spl/arch/powerpc/cpu/mpc85xx/.depend.tlb /srv/home/pokybuild/yocto-autobuilder-new/yocto-slave/nightly-fsl-ppc/build/build/tmp/work/p1022ds-poky-linux-gnuspe/u-boot/git-r30/git/P1022DS_NAND/spl/arch/powerpc/cpu/mpc85xx/.depend.spl_minimal >/srv/home/pokybuild/yocto-autobuilder-new/yocto-slave/nightly-fsl-ppc/build/build/tmp/work/p1022ds-poky-linux-gnuspe/u-boot/git-r30/git/P1022DS_NAND/spl/arch/powerpc/cpu/mpc85xx/.depend"

twice at the same time. This looks like a race/dependency problem in the makefile.
Comment 5 Richard Purdie 2013-09-22 15:59:01 UTC
In the top level makefile there is the comment:
# Explicitly make _depend in subdirs containing multiple targets to prevent
# parallel sub-makes creating .depend files simultaneously.

and I think this describes the problem. I'd guess something is missing a _depend dependency or similar.
Comment 6 Richard Purdie 2013-09-23 13:27:03 UTC
You can make it fail with parallel make by adding this:

Index: git/rules.mk
===================================================================
--- git.orig/rules.mk	2013-09-23 13:25:23.797831358 +0000
+++ git/rules.mk	2013-09-23 13:22:57.000000000 +0000
@@ -40,6 +40,11 @@
 #	3. Add in the HOSTSRCS
 $(obj).depend:	$(src)Makefile $(TOPDIR)/config.mk $(DEPS) $(OTHER_SRCS) \
 		$(HOSTSRCS)
+	if [ -f $@ ]; then \
+		echo "Found $@ already exists. Parallel make race."; \
+		exit 1; \
+	fi
+		
 	cat /dev/null $(DEPS) >$@
 	@for f in $(OTHER_SRCS); do \
 		g=`basename $$f | sed -e 's/\(.*\)\.[[:alnum:]_]/\1.o/'`; \

since writing a depend file more than once in a clean build is a race.
Comment 7 Richard Purdie 2013-09-23 14:12:43 UTC
I think the fix is:
Index: git/spl/Makefile
===================================================================
--- git.orig/spl/Makefile	2013-09-23 13:25:23.797831358 +0000
+++ git/spl/Makefile	2013-09-23 14:12:19.809766137 +0000
@@ -185,7 +185,11 @@
 $(obj)u-boot-spl.lds: $(LDSCRIPT) $(obj)u-boot.lst depend
 	$(CPP) $(CPPFLAGS) $(LDPPFLAGS) -I$(obj). -ansi -D__ASSEMBLY__ -P - < $< > $@
 
-depend:	$(obj).depend
+# Explicitly make _depend in subdirs containing multiple targets to prevent
+# parallel sub-makes creating .depend files simultaneously.
+depend dep:	$(obj).depend
+		for dir in $(SUBDIRS) $(CPUDIR) $(LDSCRIPT_MAKEFILE_DIR) ; do \
+			$(MAKE) -C $(SRCTREE)/$$dir _depend ; done
 .PHONY: depend
 
 # defines $(obj).depend target

Basically copy the dependency code from the top level makefile into the spl one.
Comment 8 Zhenhua Luo 2013-09-24 02:37:08 UTC
(In reply to comment #7)
> I think the fix is:
> Index: git/spl/Makefile
> ===================================================================
> --- git.orig/spl/Makefile	2013-09-23 13:25:23.797831358 +0000
> +++ git/spl/Makefile	2013-09-23 14:12:19.809766137 +0000
> @@ -185,7 +185,11 @@
>  $(obj)u-boot-spl.lds: $(LDSCRIPT) $(obj)u-boot.lst depend
>  	$(CPP) $(CPPFLAGS) $(LDPPFLAGS) -I$(obj). -ansi -D__ASSEMBLY__ -P - < $< >
> $@
>  
> -depend:	$(obj).depend
> +# Explicitly make _depend in subdirs containing multiple targets to prevent
> +# parallel sub-makes creating .depend files simultaneously.
> +depend dep:	$(obj).depend
> +		for dir in $(SUBDIRS) $(CPUDIR) $(LDSCRIPT_MAKEFILE_DIR) ; do \
> +			$(MAKE) -C $(SRCTREE)/$$dir _depend ; done
>  .PHONY: depend
>  
>  # defines $(obj).depend target
> 
> Basically copy the dependency code from the top level makefile into the spl
> one.

Richard, thanks a lot for your suggestion. 

We will try as your comments.
Comment 9 Zhenhua Luo 2013-09-24 06:15:03 UTC
I tried -j128 -j256 -j1000 to build u-boot for p1022ds, but the issue is still not reproduce, maybe the issue happens on hosts with specific Linux distros.

Anyway, I submit a patch according to Richard's comment, let's see whether the issue will appear again with the patch.

http://patches.openembedded.org/patch/58631/. 

Please reopen if the issue is reproduced again.