Bug 2908 - gcc git repo is huge and takes a long time to fetch
Summary: gcc git repo is huge and takes a long time to fetch
Status: VERIFIED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: devtools / tool chain (show other bugs)
Version: 1.3
Hardware: All Multiple
: Medium normal
Target Milestone: 1.3 M4
Assignee: Khem Raj
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2012-08-07 09:24 UTC by Paul Eggleton
Modified: 2012-10-05 13:11 UTC (History)
5 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Paul Eggleton 2012-08-07 09:24:33 UTC
The git repo we now fetch gcc from (changed from svn in OE-Core revision 3f3b74b3fd6ba813dd69804706edcea145876e09) is very large - 1.7GB on disk. This takes a very long time to download. Is there a workable alternative to having the entire history here that would make things less painful for users?
Comment 1 Luke A. Guest 2012-08-07 15:06:36 UTC
Even after downloading the massive repository, it fails at the end of it anyway.

----
$ bitbake -c fetchall core-image-minimal
Loading cache: 100% |##############################################################################################################################################################################| ETA:  00:00:00
Loaded 1195 entries from dependency cache.
WARNING: Host distribution "Debian GNU/Linux testing (wheezy)" has not been validated with this version of the build system; you may possibly experience unexpected failures. It is recommended that you use a tested distribution.

Build Configuration:
BB_VERSION        = "1.15.3"
TARGET_ARCH       = "arm"
TARGET_OS         = "linux-gnueabi"
MACHINE           = "pandaboard"
DISTRO            = "poky"
DISTRO_VERSION    = "1.2+snapshot-20120807"
TUNE_FEATURES     = "armv7a vfp neon cortexa9"
TARGET_FPU        = "vfp-neon"
meta              
meta-yocto        = "master:c86d26cb976e665b1516e72153f3f686f62dedf9"
meta-ti           = "remotes/origin/HEAD:9bc77dff5f84578e259f8225bfa0656d94a2a60a"

NOTE: Resolving any missing task queue dependencies
NOTE: Preparing runqueue
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Running task 229 of 286 (ID: 247, /home/laguest/src/others/poky/poky/meta/recipes-devtools/gcc/gcc-cross-initial_4.7.bb, do_fetch)
NOTE: package gcc-cross-initial-4.7.1.0+git1+d07e24f4ab59f264d68d21838795349faab5dede-r9: task do_fetch: Started
WARNING: Failed to fetch URL git://github.com/mirrors/gcc.git;branch=gcc-4_7-branch;protocol=git, attempting MIRRORS if available
ERROR: Fetcher failure: Fetch command export HOME="/home/laguest"; export SSH_AGENT_PID="4489"; export SSH_AUTH_SOCK="/home/laguest/.cache/keyring-xBrsPv/ssh"; export GIT_CONFIG="/home/laguest/src/others/poky/build/tmp/sysroots/x86_64-linux/etc/gitconfig"; export PATH="/home/laguest/src/others/poky/poky/scripts:/home/laguest/src/others/poky/build/tmp/sysroots/x86_64-linux/usr/bin/armv7a-vfp-neon-poky-linux-gnueabi:/home/laguest/src/others/poky/build/tmp/sysroots/x86_64-linux/home/laguest/src/others/poky/build/tmp/sysroots/x86_64-linux/usr/bin/armv7a-vfp-neon-poky-linux-gnueabi.gcc-cross-initial/crossscripts:/home/laguest/src/others/poky/build/tmp/sysroots/x86_64-linux/usr/sbin:/home/laguest/src/others/poky/build/tmp/sysroots/x86_64-linux/usr/bin:/home/laguest/src/others/poky/build/tmp/sysroots/x86_64-linux/sbin:/home/laguest/src/others/poky/build/tmp/sysroots/x86_64-linux//bin:/home/laguest/src/others/poky/poky/scripts:/home/laguest/src/others/poky/poky/bitbake/bin/:/home/laguest/opt/wxWidgets/bin:/home/laguest/opt/android-sdk-linux_x86/tools:/home/laguest/opt/android-sdk-linux_x86/platform-tools:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/usr/brlcad/bin:/opt/LPCXpresso/bin:/opt/LPCXpresso/tools/bin"; tar -xzf /home/laguest/src/others/poky/build/downloads/git2_github.com.mirrors.gcc.git.tar.gz failed with exit code 2, output:
STDOUT: 
STDERR: 
gzip: stdin: unexpected end of file
tar: Unexpected EOF in archive
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now

ERROR: Function failed: Fetcher failure for URL: 'git://github.com/mirrors/gcc.git;branch=gcc-4_7-branch;protocol=git'. Unable to fetch URL from any source.
ERROR: Logfile of failure stored in: /home/laguest/src/others/poky/build/tmp/work/armv7a-vfp-neon-poky-linux-gnueabi/gcc-cross-initial-4.7.1.0+git1+d07e24f4ab59f264d68d21838795349faab5dede-r9/temp/log.do_fetch.5589
NOTE: package gcc-cross-initial-4.7.1.0+git1+d07e24f4ab59f264d68d21838795349faab5dede-r9: task do_fetch: Failed
ERROR: Task 247 (/home/laguest/src/others/poky/poky/meta/recipes-devtools/gcc/gcc-cross-initial_4.7.bb, do_fetch) failed with exit code '1'
NOTE: Tasks Summary: Attempted 229 tasks of which 228 didn't need to be rerun and 1 failed.

Summary: 1 task failed:
  /home/laguest/src/others/poky/poky/meta/recipes-devtools/gcc/gcc-cross-initial_4.7.bb, do_fetch
Summary: There were 2 WARNING messages shown.
Summary: There were 2 ERROR messages shown, returning a non-zero exit code.
Comment 2 Khem Raj 2012-08-08 05:35:29 UTC
I am testing a patch to move back to svn. This will let us keep the old size which is around 96M and then when 4.7.2 comes out we should switch to using tar balls + patches method may be.
Comment 3 Paul Eggleton 2012-09-24 08:46:37 UTC
Fixed in master thanks to Khem (using the release tarball now that 4.7.2 is out):

http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=df74f8d9a8b1500da3fffd77f12ec8d1900d0bf6
Comment 4 Stoicescu Cornel 2012-10-05 13:11:07 UTC
Verified in master, commit: ef5dcad3e3e3c004112cfc00843727a7fda75cd2

gcc package is now downloaded via tarball:

"
-SRC_URI = "git://github.com/mirrors/gcc.git;branch=${BRANCH};protocol=git +SRC_URI = "${GNU_MIRROR}/gcc/gcc-${PV}/gcc-${PV}.tar.bz2 
"

"bitbake gcc" was successfull in master, commit: ef5dcad3e3e3c004112cfc00843727a7fda75cd2