Bug 1089

Summary: bitbake git fetcher leaves working trees in horrible states
Product: [Build System, Metadata & Runtime] BitBake Reporter: Richard Purdie <richard.purdie>
Component: bitbakeAssignee: Ke Yu <ke.yu>
Status: RESOLVED FIXED QA Contact:
Severity: major    
Priority: High CC: bruce.ashfield, dvhart, poky.bs.watcher, poky.watcher, sgw
Version: unspecified   
Target Milestone: 1.1 M2   
Hardware: All   
OS: Multiple   
Whiteboard: Patch sent out. May 27
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: ---

Description Richard Purdie 2011-05-19 09:17:36 UTC 
The git fetcher is providing the correct code but if you do something like:

SRC_URI = "git://git.denx.de/u-boot.git;branch=next;protocol=git;tag=b8339e2b9f32663411dba0f48e25b23f542d53bc"

then change into the workdir, git branch will show "master". git log shows the head of master. The user then thinks "What?!"

If you run "git status", you get a scary report which is the differences between master and the specified tag b8339e2b9f32663411dba0f48e25b23f542d53bc.

The reason is the fetcher after doing a clone does:

git read-tree b8339e2b9f32663411dba0f48e25b23f542d53bc
git checkout-index -q -f -a

which means it checkout out the right revision in the index but does not make the branch/heads point at the right places.

We can do better than this...
Comment 1 Richard Purdie 2011-05-19 15:21:29 UTC 
Adding Bruce as he might have some advice on the better thing to do here...
Comment 2 Darren Hart 2011-05-20 11:14:06 UTC 
Why the use of read-tree in the first place? What is preventing us from using the more lay commands, like checkout?
Comment 3 Bruce Ashfield 2011-05-20 18:35:01 UTC 
My comments are very similar to Darren's. I don't know the
background on why read-tree is being used here at all ?

Since a tag, and not a branch is being specified, what's 
wrong with auto creating a branch when a tag is specified ?
Or if not auto creating a branch, leave it on master, but
reset to the tag, or some similarly common action that
will keep the index, working tree and staged changes all
in a consistent state ?

I haven't looked through the code, but it strikes me to
ask why this is even processed differently from a clone
with a specific SRCREV specified ?
Comment 4 Darren Hart 2011-05-20 22:33:07 UTC 
I sent an email to the poky list on this, but it seems to me the only reason to do the read-tree/checkout-index approach is for specifying a subdir, but it seems that can be managed more directly and explicitly in the recipe. It seems to me just doing a checkout of the specified hash and leaving the working dir in a detached head state would be the way to go.
Comment 5 Darren Hart 2011-05-23 11:15:30 UTC 
From what I can tell the read-tree usage is all about the subdir concept - checking out only part of the tree. We could do something like:

# just guessing here, commands are inaccurate, pseudo-code only:
if (subdir)
	git read-tree branch:subdir
	git checkout-index
else
	git checkout branch

This would fix the issue for most recipes. Ultimately I think it would make sense to deprecate subdir for git repositories and push the necessary build logic up into the recipe itself.

For now, the fetcher2/git.py is generating the correct source tree, so this isn't particularly urgent. I'd suggest we drop the Importance a bit.
Comment 6 Ke Yu 2011-05-24 00:41:28 UTC 
(In reply to comment #5)
> From what I can tell the read-tree usage is all about the subdir concept -
> checking out only part of the tree. We could do something like:
> 
> # just guessing here, commands are inaccurate, pseudo-code only:
> if (subdir)
>     git read-tree branch:subdir
>     git checkout-index
> else
>     git checkout branch
> 
> This would fix the issue for most recipes. Ultimately I think it would make
> sense to deprecate subdir for git repositories and push the necessary build
> logic up into the recipe itself.
> 
> For now, the fetcher2/git.py is generating the correct source tree, so this
> isn't particularly urgent. I'd suggest we drop the Importance a bit.

yeah, this looks reasonable to me. I can work out this patch.

And currently poky a.k.a oe-core meta data already get rid of the subdir usage, but  there are still bunch of subdir usage in OE, so it is still necessary to keep subdir.