Bug 13553 - Mega-manual fails to build (but other manuals do build)
Summary: Mega-manual fails to build (but other manuals do build)
Status: RESOLVED FIXED
Alias: None
Product: Mega Manual
Classification: Documentation
Component: mega-manual (show other bugs)
Version: unspecified
Hardware: x86 Multiple
: Undecided normal
Target Milestone: 3.1
Assignee: akuster
QA Contact:
URL:
Whiteboard: 12 December 2019: IN PROGRESS REVIEW
Depends on:
Blocks:
 
Reported: 2019-09-25 18:38 UTC by Kevin Cribbs
Modified: 2020-04-16 06:54 UTC (History)
4 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Yes (doc changes required)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Kevin Cribbs 2019-09-25 18:38:42 UTC
Unless I make the changes shown below, mega-manual doesn't build. The problem might not be noticed when building in a shallow directory hierarchy. This was confirmed in a very recent poky 81f9e815d36848761a9dfa94b00ad998bb39a4a6.


diff --git a/documentation/mega-manual/mega-manual.xml b/documentation/mega-manual/mega-manual.xml
index b66b933..c2038de 100644
--- a/documentation/mega-manual/mega-manual.xml
+++ b/documentation/mega-manual/mega-manual.xml
@@ -332,22 +332,22 @@
     </para>
 
     <xi:include
-        xmlns:xi="http://www.w3.org/2003/XInclude" href="../../../bitbake/doc/bitbake-user-manual/bitbak
+        xmlns:xi="http://www.w3.org/2003/XInclude" href="../../bitbake/doc/bitbake-user-manual/bitbake-u
 
     <xi:include
-        xmlns:xi="http://www.w3.org/2003/XInclude" href="../../../bitbake/doc/bitbake-user-manual/bitbak
+        xmlns:xi="http://www.w3.org/2003/XInclude" href="../../bitbake/doc/bitbake-user-manual/bitbake-u
 
     <xi:include
-        xmlns:xi="http://www.w3.org/2003/XInclude" href="../../../bitbake/doc/bitbake-user-manual/bitbak
+        xmlns:xi="http://www.w3.org/2003/XInclude" href="../../bitbake/doc/bitbake-user-manual/bitbake-u
 
     <xi:include
-        xmlns:xi="http://www.w3.org/2003/XInclude" href="../../../bitbake/doc/bitbake-user-manual/bitbak
+        xmlns:xi="http://www.w3.org/2003/XInclude" href="../../bitbake/doc/bitbake-user-manual/bitbake-u
 
     <xi:include
-        xmlns:xi="http://www.w3.org/2003/XInclude" href="../../../bitbake/doc/bitbake-user-manual/bitbak
+        xmlns:xi="http://www.w3.org/2003/XInclude" href="../../bitbake/doc/bitbake-user-manual/bitbake-u
 
     <xi:include
-        xmlns:xi="http://www.w3.org/2003/XInclude" href="../../../bitbake/doc/bitbake-user-manual/bitbak
+        xmlns:xi="http://www.w3.org/2003/XInclude" href="../../bitbake/doc/bitbake-user-manual/bitbake-u
 
 </book>
Comment 1 Richard Purdie 2019-09-26 14:57:50 UTC
Can you confirm you were using poky to do this? I think our tech writer who usually runs this is probably using separate checkouts of the bitbake and yocto-docs repos which would explain the difference in behavior.
Comment 2 Kevin Cribbs 2019-10-01 13:45:55 UTC
I used poky 81f9e815d36848761a9dfa94b00ad998bb39a4a6 when I confirmed this problem remains.
Comment 3 Armin Kuster 2019-10-02 20:10:43 UTC
(In reply to comment #2)
> I used poky 81f9e815d36848761a9dfa94b00ad998bb39a4a6 when I confirmed this
> problem remains.

ok. that is in master (2.8_M2), not a stable branch

git branch --contains 81f9e815d36848761a9dfa94b00ad998bb39a4a6
  master
Comment 4 Kevin Cribbs 2019-10-10 11:03:23 UTC
(In reply to comment #3)
> (In reply to comment #2)
> > I used poky 81f9e815d36848761a9dfa94b00ad998bb39a4a6 when I confirmed this
> > problem remains.
> 
> ok. that is in master (2.8_M2), not a stable branch
> 
> git branch --contains 81f9e815d36848761a9dfa94b00ad998bb39a4a6
>   master

Just in case any confusion remains ...

I found this problem while working with poky 2.7.1.

I tried the "very recent poky 81f9e815d36848761a9dfa94b00ad998bb39a4a6" in an attempt to avoid creating a bug report for something that was already fixed.
Comment 5 Scott Rifenbark 2019-10-10 14:52:57 UTC
Hi, 

I don't quite understand why this is happening.  When the mega-manual builds, it includes the BitBake manual's HTML file from a separately cloned bitbake directory. 

I don't checkout by hash much.  So, are you saying if you clone poky using 81f9e815d36848761a9dfa94b00ad998bb39a4a6, the mega-manual.xml file is corrupted?

Scott
Comment 6 Kevin Cribbs 2019-10-28 14:44:57 UTC
I still see the error described in the 2019-09-25 18:38:42 report when I do the following on a Linux Mint 19 PC:

After git cloning and checking out the current Warrior branch (commit: 4ada58f74c9bb5ab2fd4ef7a0c6ee4219b579d4b) ...

cd /tmp/poky_warrior_2.7.2_prerelease
. oe-init-build-env
cd ../documentation
make DOC=mega-manual html
 
******** Building mega-manual
 
cd mega-manual; xsltproc --stringparam html.stylesheet mega-style.css --stringparam  chapter.autolabel 1 --stringparam  section.autolabel 1 --stringparam  section.label.includes.component.label 1 --xinclude -o mega-manual.html mega-manual-customization.xsl mega-manual.xml; cd ..
warning: failed to load external entity "../../../bitbake/doc/bitbake-user-manual/bitbake-user-manual-intro.xml"
mega-manual.xml:340: element include: XInclude error : could not load ../../../bitbake/doc/bitbake-user-manual/bitbake-user-manual-intro.xml, and no fallback was found
warning: failed to load external entity "../../../bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.xml"
mega-manual.xml:343: element include: XInclude error : could not load ../../../bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.xml, and no fallback was found
warning: failed to load external entity "../../../bitbake/doc/bitbake-user-manual/bitbake-user-manual-metadata.xml"
mega-manual.xml:346: element include: XInclude error : could not load ../../../bitbake/doc/bitbake-user-manual/bitbake-user-manual-metadata.xml, and no fallback was found
warning: failed to load external entity "../../../bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.xml"
mega-manual.xml:349: element include: XInclude error : could not load ../../../bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.xml, and no fallback was found
warning: failed to load external entity "../../../bitbake/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.xml"
mega-manual.xml:352: element include: XInclude error : could not load ../../../bitbake/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.xml, and no fallback was found
warning: failed to load external entity "../../../bitbake/doc/bitbake-user-manual/bitbake-user-manual-hello.xml"
mega-manual.xml:355: element include: XInclude error : could not load ../../../bitbake/doc/bitbake-user-manual/bitbake-user-manual-hello.xml, and no fallback was found
 
******** Using mega-manual.sed to process external links
 
cd mega-manual; sed -f ../tools/mega-manual.sed < mega-manual.html > mega-output.html; cd ..
/bin/sh: mega-manual.html: No such file or directory
 
******** Cleaning up transient file mega-output.html
 
cd mega-manual; rm mega-manual.html; mv mega-output.html mega-manual.html; cd ..
rm: cannot remove 'mega-manual.html': No such file or directory
mv: cannot stat 'mega-output.html': No such file or directory



Then edit mega-manual/mega-manual.xml to replace "../../.." with "../..".

make DOC=mega-manual html
 
******** Building mega-manual
 
cd mega-manual; xsltproc --stringparam html.stylesheet mega-style.css --stringparam  chapter.autolabel 1 --stringparam  section.autolabel 1 --stringparam  section.label.includes.component.label 1 --xinclude -o mega-manual.html mega-manual-customization.xsl mega-manual.xml; cd ..
Note: namesp. cut : stripped namespace before processing           
            Yocto Project Mega-Manual
        
Note: namesp. cut : processing stripped document                   
            Yocto Project Mega-Manual
        
Attributed 36142 IDs for element, cleaned up 0
 
******** Using mega-manual.sed to process external links
 
cd mega-manual; sed -f ../tools/mega-manual.sed < mega-manual.html > mega-output.html; cd ..
 
******** Cleaning up transient file mega-output.html
 
cd mega-manual; rm mega-manual.html; mv mega-output.html mega-manual.html; cd ..



Using the current Warrior branch Creation of the mega-manual only works after replacing all ocurences of "../../.." with "../.." in "mega-manual/mega-manual.xml". The problem also remains on the current master branch (commit: 3a751d5564fc6ee9aef225653cc7b8630fd25a35).
Comment 7 Scott Rifenbark 2019-10-28 16:56:11 UTC
This could be because of where the bitbake directory is located.  In my build environment where I work on the manuals, I have the bitbake directory cloned at a top level directory - $HOME/bitbake.  When I build the BB manual, it looks up three levels to fine the HTML files for the BB manual.

When you clone poky, the bitbake manual repository is at the same level as the "documentation" folder, which makes the "bitbake" folder one level closer or lower.  So if a person builds the mega-manual from the poky repository locally it will not find the BB manuals by going ../../..   They would only be found using ../..  

I will likely have to change my production location for the bitbake manuals to follow the same hierarchy as if building the manual from the poky documentation folder.
Comment 8 Scott Rifenbark 2019-10-28 17:24:14 UTC
Kevin, 

I made the changes on my end.  My production environment now hierarchically matches the same structure as poky.  So, the HTML files for BitBake are found in the same level in my environment and in a poky-cloned environment.  I updated the mega-manual.xml file to use the ../.. search instead of the ../../../ search. 

You can't see the effects of this change and commit until the commit makes it into the poky repository.  Right now, the commit is pushed to the yocto-docs repo.  I don't control when changes get into the poky repository.

Scott
Comment 9 Scott Rifenbark 2019-12-06 00:56:56 UTC
Hi, 

Mega-manual builds correctly for "warrior", "zeus", and "master" Git development branches for all released versions of the manuals.  It works in my development environment.  It should work in a user's environment as long as the bitbake repository is in the "poky" folder, which it is.  

The updated files (mega-manual.xml) that correct the change had to use a "../.." file path rather than the "../../.." one that was causing the error.  The change will not work in a user environment until the commits that are responsible for the mega-manual.xml file are sucked into the poky upstream Git repository.  I believe Richard's automated scripts will deal with this.

Scott
Comment 10 Armin Kuster 2019-12-07 23:36:49 UTC
thanks Scott.
Comment 11 Scott Rifenbark 2019-12-12 21:43:47 UTC
I am going to leave this in IN PROGRESS REVIEW for a while.  We will need to verify it works when all changes are merged into poky.

Scott
Comment 12 akuster 2020-04-16 06:54:17 UTC
I think this is fix. I have been building locally fine if I have bitbake in the correct location relative to the documentation build location.