I wanted to set up an HTTP-accessible mirror of poky and a few meta-xxx layers so I can sync with them from behind my company's firewall. I was planning to use a PC at home to pull the latest upstream 'git:://' sources every 15 minutes and then push them to a (HTTP-accessible) mirror on github. I wanted to use a Windows box for this, because I happen to have one sitting around at home doing nothing most of the time. Alas, this isn't possible due to some fiendishly clever (ab)use of file-naming conventions. When I try to clone poky onto an NTFS filesystem using msysgit, I get the errors shown appended below. I'm pretty sure this is expected behavior, but I found it intriguing enough that I thought it worthwhile to document in a place where google can find it. Also, I found a workaround that allows me to use a Windows box for mirror-wrangling, anyway. Basically, I did the following: > $ git clone --mirror git://git.yoctoproject.org/poky.git > $ cd poky.git > $ git remote add github git@github.com:evadeflow/poky.git > $ git push --mirror github The 'clone --mirror' command creates a bare repo with no working copy, so I avoid the errors due to invalid filenames. Then, I just run a script every 15 minutes that changes to poky.git folder and does: > $ git fetch > $ git push --mirror github So far, it seems to work fine. I think this is the 'right' way to handle mirroring, even from Linux. (When all you want to do is fetch/push without injecting any customizations, a working copy isn't actually needed...) -------------------- [Errors seen with a naive attempt to clone poky on Windows] > $ git clone git://git.yoctoproject.org/poky.git > Cloning into 'poky'... > remote: Counting objects: 162905, done. > remote: Compressing objects: 100% (42924/42924), done. > remote: Total 162905 (delta 116084), reused 162788 (delta 115967) > Receiving objects: 100% (162905/162905), 81.67 MiB | 350 KiB/s, done. > Resolving deltas: 100% (116084/116084), done. > error: unable to create file scripts/lib/bsp/substrate/target/arch/arm/recipes-graphics/xorg-xserver/xserver-xf86-config/{{=machine}}/{{ if xserver == "y": }} xorg.conf (Invalid argument) > error: unable to create file scripts/lib/bsp/substrate/target/arch/arm/recipes-graphics/xorg-xserver/{{ if xserver == "y": }} xserver-xf86-config_0.1.bbappend (Invalid argument) > error: unable to create file scripts/lib/bsp/substrate/target/arch/arm/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto-rt_3.2": }} linux-yocto-rt_3.2.bbappend (Invalid argument) > error: unable to create file scripts/lib/bsp/substrate/target/arch/arm/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto-rt_3.4": }} linux-yocto-rt_3.4.bbappend (Invalid argument) > error: unable to create file scripts/lib/bsp/substrate/target/arch/arm/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto_3.2": }} linux-yocto_3.2.bbappend (Invalid argument) > error: unable to create file scripts/lib/bsp/substrate/target/arch/arm/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto_3.4": }} linux-yocto_3.4.bbappend (Invalid argument) > error: unable to create file scripts/lib/bsp/substrate/target/arch/i386/recipes-graphics/xorg-xserver/xserver-xf86-config/{{=machine}}/{{ if xserver_choice == "xserver_emgd": }} xorg.conf (Invalid argument) > error: unable to create file scripts/lib/bsp/substrate/target/arch/i386/recipes-graphics/xorg-xserver/xserver-xf86-config/{{=machine}}/{{ if xserver_choice == "xserver_i915": }} xorg.conf (Invalid argument) > error: unable to create file scripts/lib/bsp/substrate/target/arch/i386/recipes-graphics/xorg-xserver/xserver-xf86-config/{{=machine}}/{{ if xserver_choice == "xserver_vesa": }} xorg.conf (Invalid argument) > error: unable to create file scripts/lib/bsp/substrate/target/arch/i386/recipes-graphics/xorg-xserver/{{ if xserver == "y": }} xserver-xf86-config_0.1.bbappend (Invalid argument) > error: unable to create file scripts/lib/bsp/substrate/target/arch/i386/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto-rt_3.2": }} linux-yocto-rt_3.2.bbappend (Invalid argument) > error: unable to create file scripts/lib/bsp/substrate/target/arch/i386/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto-rt_3.4": }} linux-yocto-rt_3.4.bbappend (Invalid argument) > error: unable to create file scripts/lib/bsp/substrate/target/arch/i386/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto_3.2": }} linux-yocto_3.2.bbappend (Invalid argument) > error: unable to create file scripts/lib/bsp/substrate/target/arch/i386/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto_3.4": }} linux-yocto_3.4.bbappend (Invalid argument) > error: unable to create file scripts/lib/bsp/substrate/target/arch/mips/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto-rt_3.2": }} linux-yocto-rt_3.2.bbappend (Invalid argument) > error: unable to create file scripts/lib/bsp/substrate/target/arch/mips/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto-rt_3.4": }} linux-yocto-rt_3.4.bbappend (Invalid argument) > error: unable to create file scripts/lib/bsp/substrate/target/arch/mips/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto_3.2": }} linux-yocto_3.2.bbappend (Invalid argument) > error: unable to create file scripts/lib/bsp/substrate/target/arch/mips/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto_3.4": }} linux-yocto_3.4.bbappend (Invalid argument) > error: unable to create file scripts/lib/bsp/substrate/target/arch/powerpc/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto-rt_3.2": }} linux-yocto-rt_3.2.bbappend (Invalid argument) > error: unable to create file scripts/lib/bsp/substrate/target/arch/powerpc/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto-rt_3.4": }} linux-yocto-rt_3.4.bbappend (Invalid argument) > error: unable to create file scripts/lib/bsp/substrate/target/arch/powerpc/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto_3.2": }} linux-yocto_3.2.bbappend (Invalid argument) > error: unable to create file scripts/lib/bsp/substrate/target/arch/powerpc/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto_3.4": }} linux-yocto_3.4.bbappend (Invalid argument) > error: unable to create file scripts/lib/bsp/substrate/target/arch/qemu/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto-rt_3.2": }} linux-yocto-rt_3.2.bbappend (Invalid argument) > error: unable to create file scripts/lib/bsp/substrate/target/arch/qemu/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto-rt_3.4": }} linux-yocto-rt_3.4.bbappend (Invalid argument) > error: unable to create file scripts/lib/bsp/substrate/target/arch/qemu/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto_3.2": }} linux-yocto_3.2.bbappend (Invalid argument) > error: unable to create file scripts/lib/bsp/substrate/target/arch/qemu/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto_3.4": }} linux-yocto_3.4.bbappend (Invalid argument) > error: unable to create file scripts/lib/bsp/substrate/target/arch/x86_64/recipes-graphics/xorg-xserver/xserver-xf86-config/{{=machine}}/{{ if xserver_choice =="xserver_i915": }} xorg.conf (Invalid argument) > error: unable to create file scripts/lib/bsp/substrate/target/arch/x86_64/recipes-graphics/xorg-xserver/xserver-xf86-config/{{=machine}}/{{ if xserver_choice =="xserver_vesa": }} xorg.conf (Invalid argument) > error: unable to create file scripts/lib/bsp/substrate/target/arch/x86_64/recipes-graphics/xorg-xserver/{{ if xserver == "y": }} xserver-xf86-config_0.1.bbappend (Invalid argument) > error: unable to create file scripts/lib/bsp/substrate/target/arch/x86_64/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto-rt_3.2": }} linux-yocto-rt_3.2.bbappend (Invalid argument) > error: unable to create file scripts/lib/bsp/substrate/target/arch/x86_64/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto-rt_3.4": }} linux-yocto-rt_3.4.bbappend (Invalid argument) > error: unable to create file scripts/lib/bsp/substrate/target/arch/x86_64/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto_3.2": }} linux-yocto_3.2.bbappend (Invalid argument) > error: unable to create file scripts/lib/bsp/substrate/target/arch/x86_64/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto_3.4": }} linux-yocto_3.4.bbappend (Invalid argument) > Checking out files: 100% (3845/3845), done.
I'm closing this since I really only created it as a 'Note To Self'. Please let me know if doing this messes with bug tracking metrics or something and I'll desist...
I set up mirrors of poky and the layers I'm interested in at: - http://github.com/yoctoproject-mirrors I'm syncing them with the upstream sources every 15 minutes using a Windows box at home. I have a (git) bash script containing the following: > #!/bin/sh > > # File: sync_mirrors.sh > > for repo in `find . -name "*.git" -a -type d`; do > (cd $repo; git fetch; git push --mirror github) > done I run it from the Windows Task Scheduler such that it gets invoked as: > "c:\Program Files (x86)\Git\bin\sh.exe" --login sync_mirrors.sh Note: It's important to include '--login' in the git bash invocation, or else the environment won't be set up correctly. Also, as mentioned previously, the repos on the Windows box *must* be bare repos, because it's not possible to create a working copy on Windows (due to invalid characters in the file names). The script is simple because all the repos have a consistent config. Here's the one for poky: > $ cat poky.git/config > [core] > repositoryformatversion = 0 > filemode = false > bare = true > symlinks = false > ignorecase = true > hideDotFiles = dotGitOnly > [remote "origin"] > fetch = +refs/*:refs/* > mirror = true > url = git://git.yoctoproject.org/poky.git > [remote "github"] > url = git@github.com:yoctoproject-mirrors/poky.git > fetch = +refs/heads/*:refs/remotes/github/* Thus, they all have a remote named 'github' pointing at 'github.com/yoctoproject-mirrors'. Oh, yeah---I also needed to set up a an SSH key with an empty passphrase for this, and upload it to the yoctoproject-mirrors account on github. Otherwise, the push(es) fail with authentication errors. It's the weekend, so there haven't yet been any commits I can use to confirm whether this setup is working. But I *think* it is...