Bug 9924

Summary: Prevent users from using HEAD as Git revision when importing layers
Product: [Build System, Metadata & Runtime] Toaster Reporter: Belen Barros Pena <belen.barros.pena>
Component: toasterAssignee: David Reyna <david.reyna>
Status: RESOLVED FIXED QA Contact: Libertad <libertad.gonzalez.de.la.cruz>
Severity: enhancement    
Priority: Medium+ CC: belen.barros.pena, brian.avery, david.reyna, jessica.zhang, joshuagloe, stanciux.mihail
Version: 2.3   
Target Milestone: 2.4 M4   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Belen Barros Pena 2016-07-13 11:22:41 UTC
According to this email to the mailing list 

https://lists.yoctoproject.org/pipermail/toaster/2016-July/004957.html

importing a layer with Git revision "HEAD" causes builds to fail with this error:

"Existing git repository at /var/www/toaster/poky, but with different 
remotes ('origin git://git.yoctoproject.org/poky (fetch), origin 
git://git.yoctoproject.org/poky (push), ', expected 
'git://git.yoctoproject.org/meta-raspberrypi'). Toaster will not 
continue out of fear of damaging something."

We should consider stopping users from entering HEAD as the Git revision for imported layers.
Comment 1 Belen Barros Pena 2016-08-09 10:03:45 UTC
Users keep hitting this: we really must fix in 2.3. I am going to promote it to medium+ so that we don't forget about it.
Comment 2 David Reyna 2017-05-04 04:49:29 UTC
I have replicated the issue. When you specify the branch "HEAD", Toaster attempts to clone it into the main "poky" directory instead in the clone directory.

As the solution, either we block the choice of "HEAD" (as suggested), or we make sure all clones (even HEAD) go into the the clone directory. I cannot believe that cloning into "poky" is ever correct, so perhaps there is obsolete code.

NOTE: With Toaster-2.3, I did not get the relatively clean error message as reported, but a hidden bitbake crash and a spinning build that I had to cancel.

(a) Here is the log for the clone attempt (which quietly failed):

2017-05-03 21:12:54,271 DEBUG localhostbecontroller: giturl git://git.yoctoproject.org/meta-raspberrypi:HEAD checking out in current directory /opt/dreyna/toaster_newer/poky
2017-05-03 21:12:54,271 DEBUG lbc_shellcmmd: (/opt/dreyna/toaster_newer/poky) git remote -v
2017-05-03 21:12:54,278 DEBUG localhostbecontroller: shellcmd success
2017-05-03 21:12:54,278 DEBUG localhostbecontroller: localdirpath expected '/opt/dreyna/toaster_newer/poky/'

(b) Here is the list of layers:
...
 '/opt/dreyna/toaster_newer/poky',
...

(c) Here is the bitbake crash:

... [Errno 2] file /opt/dreyna/toaster_newer/poky/conf/layer.conf not found
Comment 3 Joshua Lock 2017-05-04 09:44:39 UTC
(In reply to comment #2)
> I have replicated the issue. When you specify the branch "HEAD", Toaster
> attempts to clone it into the main "poky" directory instead in the clone
> directory.

This appears to be an intentional decision made in localhostbecontroller.py's getGitCloneDirectory(). Unfortunately why that decision was made isn't apparent from reading the code and my git archaeology isn't sufficient enough to easily trace the lineage of this decision.
Comment 4 David Reyna 2017-05-04 20:57:17 UTC
1) I now observe in the code the "HEAD" branch is reserved exclusively for the "Local Yocto Project" layers, and specifically the default layers set up in the respective fixture. That is why the path is set to the "poky" directory itself, with the layer's "Repository subdirectory" value pointing into the layer directory under "poky". 

It seems best to leave that code as it is. Other layers can use the "master" branch instead, because most layers have the mapping "origin/HEAD -> origin/master", so that is the advice we can provide when we give an error message for trying to use "HEAD".

2) There is a second problem where when I create a new imported layer the repo path is lost and the branch name is reset to "N/A", forcing me to re-edit to re-assert the layer parameters.
Comment 5 David Reyna 2017-09-15 04:37:02 UTC
The submitted patch solves 4 problems. three discovered while solving the original problem.

1) It now notifies and prevents users from selecting HEAD as the branch, as per the original intent of this defect.

2) In solving #1 I found that it was dropping the entered values for git repos. There was as error in the receiving XHR logic that resulted in git repo selection being treated as local layers (and thus skipping the entered values).

3) In solving #2 I found that there were typos in reading the received git repo object, which triggered a fatal (and silent) error.

4) In solving #3 I realized that exceptions notices were sent back via AJAX to the web page but they were only passed to the console, leaving the user in the dark as to why nothing happened. Hopefully the above fixes address the errors, but if any occur in the future the user will at least have a clue.
Comment 6 David Reyna 2017-09-18 17:35:14 UTC
commit d5a109daeaa7a83bfa2c34c45dc01a1579d24893
(Bitbake rev: 04bd9c9341e8390a7923fa0122cd4cb0befa569f)