10405 2016-10-10 17:10:56 +0000 [bitbake] Missing SRC_URI "path_spec" option for SVN fetcher 2017-03-16 18:35:38 +0000 1 1 9 Documentation BitBake User Manual bitbake-manual unspecified All Multiple RESOLVED FIXED 16 Mar 2017: RESOLVED Medium normal 2.3 M4 1 karim_atiki srifenbark bluelightning markus.lehtonen srifenbark --- --- Done (doc changes complete) oldest_to_newest 67095 0 karim_atiki 2016-10-10 17:10:56 +0000 Hello, I wrote a recipe to fetch code from the SVN repositories of our team. The repo is structured in a wa, that each project has its own trunk, branches and tags subdirectory: + proj1 ++ trunk + proj2 ++ trunk ... In the recipe file, I put the SRC_URI that way: SRC_URI = "svn://myrepos/proj1;module=trunk;protocol=http" But the content of the "trunk" was put directly into the directory: i.e. ${SVNDIR}/myrepos/proj1/trunk/myfile.h ${SVNDIR}/myrepos/proj1/trunk/myfile.cpp ${SVNDIR}/myrepos/proj1/trunk/proj1.mak ... But I was looking for the ability to have the following checkout: ${SVNDIR}/myrepos/proj1/myfile.h ${SVNDIR}/myrepos/proj1/myfile.cpp ${SVNDIR}/myrepos/proj1/proj1.mak In the BitBake and Yocto MegaManual reference, I didn't find a keyword or option that I could use within SRC_URI. Until I take a look to BitBake source-code and I found out in the svn.py that it as using the option "path_spec" to determine the output checkout directory. But this option is not documented at all in the BitBake manual. But changing my original SRC_URI made the things work correctly: SRC_URI = "svn://myrepos/proj1;module=trunk;protocol=http;path_spec=${ANOTHER_DIR}/proj1" Is it normal that this option is not documented ? If so, would it possible to add it in the documentation ? I guess I'm not the only to fetch source code from SVN in a specific checkout-dir. Cheers, Karim 67105 1 srifenbark 2016-10-10 20:23:12 +0000 Thanks Karim... Paul - Here is the SRC_URI variable description - http://www.yoctoproject.org/docs/2.2/ref-manual/ref-manual.html#var-SRC_URI. Is this just a matter of documenting a "missed" option? thanks, Scott 67117 2 bluelightning 2016-10-11 02:08:15 +0000 Correct - however note that this parameter is specific to the svn fetcher. We should add this to the "Subversion (SVN) Fetcher" section of the bitbake manual. Note that this brings up two issues: 1) Bug 8909 - fetcher-specific documentation doesn't seem to have a place in the variable glossary. Maybe it's just a matter of adding one, but it seems to me the format used in the bitbake manual makes more sense for this level of detail. 2) The bitbake manual isn't part of the mega manual. It seems to me that parts of it should be - we might not want the whole thing due to the duplication of generic bitbake information e.g. the variable reference, but there are things like the fetcher documentation that seems to only be in the bitbake manual. 67142 3 srifenbark 2016-10-11 15:06:23 +0000 Thanks Paul, This is messy... I am going to not deal with it until 2.2 is out the door. Scott 68120 4 srifenbark 2016-11-08 17:33:50 +0000 Updated length of perfect days from 1 to 3. 69915 5 srifenbark 2017-01-18 23:30:43 +0000 Hi, The one thing I can do right now is document the "path_spec" option in the SVN fetcher section (http://www.yoctoproject.org/docs/2.3/bitbake-user-manual/bitbake-user-manual.html#svn-fetcher). But, I am confused between what are calling fetcher options and parameters. In the current SVN Fetcher section, we list a bunch of parameters. I can certainly put the "path_spec" option in there on its own after the parameters but that raises the question of are there other options we want to include? Thanks, Scott 70288 6 srifenbark 2017-01-31 16:32:04 +0000 Need response to Comment 5. Moving to 2.3 M3. 70791 7 markus.lehtonen 2017-02-17 09:15:21 +0000 I don't know why this was assigned to me but I'll answer the best I can :) Assigning back to Scott. Please re-assign to me if you have further questions. (In reply to comment #5) > The one thing I can do right now is document the "path_spec" option in the > SVN fetcher section > (http://www.yoctoproject.org/docs/2.3/bitbake-user-manual/bitbake-user- > manual.html#svn-fetcher). But, I am confused between what are calling > fetcher options and parameters. I didn't quite get this. I think all of th are called 'parameters', consistently. As they should. And, 'path_spec' should be just an addition to the list of parameters > In the current SVN Fetcher section, we list > a bunch of parameters. I can certainly put the "path_spec" option in there > on its own after the parameters but that raises the question of are there > other options we want to include? I took a quick peek at the code and the 'date' parameter is not actually supported by the svn backend anymore. It was dropped already six years ago by this commit: http://git.openembedded.org/bitbake/commit/?id=9e6185fc17a8f7be8382bcd6d54dfe4aa5f0bed9 Thus, 'date' parameter should be removed from the documentation as well. The code looks somewhat broken and it seems that the documentation regarding the 'rsh' parameter and protocol should be removed but I'll get back to this later, after I've decrypted what the code should be doing 70840 8 srifenbark 2017-02-20 18:39:22 +0000 Hi, I have attempted to update the SVN Fetcher section in the BitBake manual to address this bug. First, I removed the "date" parameter from both the list of parameters and the examples at the end of the section. I was unsure on the whole "rsh" business in the "protocol" parameter so I left that untouched and you can clarify that for me. I was not sure if just the "rsh" part was supposed to be removed or the whole "protocol" parameter. So, I left it as is. Secondly, I added the "path_spec" parameter. Please look over the simple explanation and tell me if we need more detail. I did add a third example at the end of the section that uses the "path_spec" parameter. Notice also that I made the three examples consistent with what I saw for syntax in the comments here in the bug. Regarding the other comments about duplicating fetcher-specific information into the ref-manual... I did not address that. We probably need more discussion on that. Here is the area to look.. let me know. http://www.yoctoproject.org/docs/2.3/bitbake-user-manual/bitbake-user-manual.html#svn-fetcher thanks, Scott 70857 9 markus.lehtonen 2017-02-21 15:02:44 +0000 (In reply to comment #8) > Hi, > > I have attempted to update the SVN Fetcher section in the BitBake manual to > address this bug. First, I removed the "date" parameter from both the list > of parameters and the examples at the end of the section. Good. > I was unsure on > the whole "rsh" business in the "protocol" parameter so I left that > untouched and you can clarify that for me. I was not sure if just the "rsh" > part was supposed to be removed or the whole "protocol" parameter. So, I > left it as is. Let's keep it as it is, for a while. I have a bitbake patch under review that tries to clarify the functionality. Let's get back to this when that patch gets merged (or rejected). > Secondly, I added the "path_spec" parameter. Please look over the simple > explanation and tell me if we need more detail. The level of detail should be enough, I think. However, the explanation is a bit misleading in that it talks about "source file" although it should be about the svn module. A better wording could be something like: "A specific directory in which to checkout the specified svn module." 70929 10 markus.lehtonen 2017-02-24 14:00:53 +0000 My patch for svn fetcher was merge: http://git.openembedded.org/bitbake/commit/?id=5b364b02270b0d7c2b7ca8d67fa2731bf93720ee Now the documentation should be changed, accordingly. All references to 'rsh' should be removed, and, a new 'ssh' parameter should be added. You could use something like (you probably find better wording): - "protocol": The protocol to use, which defaults to "svn". If "protocol" is set to "svn+ssh" the "ssh" parameter is also used. - "ssh": An optional parameter used when "protocol" is set to "svn+ssh". May be used to specify the ssh program used by svn. 70990 11 srifenbark 2017-02-28 18:21:15 +0000 Markus, I changed the "path_spec" description to "A specific directory in which to checkout the specified svn module." as suggested. See http://www.yoctoproject.org/docs/2.3/bitbake-user-manual/bitbake-user-manual.html#svn-fetcher. New entry in the list for "ssh" has been added to the list. Updated the "protocol" description. Your wording was fine -- just added a comment after the opening clause. See the same list. For the "method" description, an occurrence of "rsh" existed as follows: "If "method" is set to "ext", BitBake examines the "rsh" parameter and sets CVS_RSH." I completely removed that sentence leaving the bit about "dir". That occurrence of "rsh" was the only one in the SVN Fetcher. There is on in the CVS Fetcher. I assume I leave that one? All changes in. Let me know if there are any problems. http://www.yoctoproject.org/docs/2.3/bitbake-user-manual/bitbake-user-manual.html#svn-fetcher Thanks, Scott 71005 12 markus.lehtonen 2017-03-01 07:32:08 +0000 (In reply to comment #11) > I changed the "path_spec" description to "A specific directory in which to > checkout the specified svn module." as suggested. See > http://www.yoctoproject.org/docs/2.3/bitbake-user-manual/bitbake-user-manual. > html#svn-fetcher. > > New entry in the list for "ssh" has been added to the list. > > Updated the "protocol" description. Your wording was fine -- just added a > comment after the opening clause. See the same list. > > For the "method" description, an occurrence of "rsh" existed as follows: "If > "method" is set to "ext", BitBake examines the "rsh" parameter and sets > CVS_RSH." I completely removed that sentence leaving the bit about "dir". > That occurrence of "rsh" was the only one in the SVN Fetcher. There is on > in the CVS Fetcher. I assume I leave that one? AFAIU, the sentence you removed (If "method" is set to "ext"...) was actually under CVS fetcher documentation and should not have been removed. The "rsh" parameter is totally valid there. So please re-add the part describing the "rsh" parameter under the CVS fetcher. Otherwise, looks good to me. 71020 13 srifenbark 2017-03-01 15:16:28 +0000 Markus, That was a goof on my part. I put that back in. Sorry about that. So I think we are good now. http://www.yoctoproject.org/docs/2.3/bitbake-user-manual/bitbake-user-manual.html#cvs-fetcher. Thanks, Scott 71025 14 markus.lehtonen 2017-03-01 20:31:30 +0000 OK :) Looks good to me, now 71370 15 srifenbark 2017-03-16 18:35:38 +0000 Hi, Setting this bug to RESOLVED and putting the doc flag to "done." Thanks, Scott