14114 2020-11-04 14:16:15 +0000 Implement proper sphinx docs publishing 2021-04-01 15:38:42 +0000 1 1 9 Documentation General Docs docs-general unspecified x86 Multiple RESOLVED FIXED Medium+ normal 3.3 M3 1 ndec13 ndec13 akuster808 mhalstead ndec13 randy.macleod --- --- No (bug/feature does not impact docs) oldest_to_newest 88547 0 ndec13 2020-11-04 14:16:15 +0000 now that we moved to Sphinx, and that we have a docs website, we need to implement how to properly publish all docs. It is not as simple as one would believe! Let's list what 'docs' we produce: 1. for yocto-docs 1.a. 'docs' for master branch 1.b. 'docs' for gatesgarth branch 1.c. 'docs' for hardknott branch (eventually) 1.d. 'docs' for the transition branch, which produces a 'wrapper' around the docbook docs for all previous releases (0.9->3.1.3 as of today) 2. for bitbake 2.a. 'docs' for master branch 2.b. 'docs' for 1.48 branch 2.c. 'docs' for 1.50 branch (eventually) We use the intersphinx extension to link yocto-docs and bitbake docs together, so in the yocto-docs conf.py file, we have a config variable that links to the location of the bitbake doc, e.g intersphinx_mapping = { 'bitbake': ('https://docs.yoctoproject.org/bitbake/', None) } That means we need to update conf.py for each yocto-docs branch accordingly. It's nice and simple, and all of that can be done in CI/CD style automatically. We end up with something like that on the docs server: folder on website | git branch | intersphinx in conf.py | Notes ----------------------------|--------------|--------------------------------|----------------- |-- index.html | master | docs.yp.org/bitbake | | | | | |-- 0.9 | | | | |-- index.html | transition | | | `-- */*.html | . | | Docbook files |-- ... | . | | | |-- index.html | transition | | | `-- */*.html | . | | Docbook files |-- ... | . | | | |-- index.html | transition | | | `-- */*.html | . | | Docbook files |-- 3.1 | | | | |-- index.html | transition | | | `-- */*.html | . | | Docbook files | | | | |-- gatesgarth | | | | `-- index.html | gatesgarth | docs.yp.org/bitbake/1.48 | |-- hardknott | | | | `-- index.html | hardknott | docs.yp.org/bitbake/1.50 | | | | | |-- bitbake | | | | |-- index.html | master | | | |-- 1.48 | | | | | `-- index.html | 1.48 | | | |-- 1.50 | | | | | `-- index.html | 1.50 | | | | | | | in CI/CD , we monitor each branch, and whenever there is a change, we rebuild and push at the right location. Note that up until now, we were not publishing docs from 'branches', but only for releases, so that's a great improvement. Now we want to publish docs for releases as well, and that's when the problems come! * How do we create the <3.2> 'docs' release? * What is the intersphinx config in that case? It's not simple "cp -a ./gatesgarth/* ./3.2" since we need to copy the bitbake docs as well, *and* we need to update all links from yocto-docs which point to bitbake! We basically need to these folders on the server: | |-- 3.2 | | | How do we make a 'doc' release. It's not a simple 'copy' | `-- index.html | XXX | docs.yp.org/bitbake/<REL 3.2> | Since there are links to bitbake which need to be updated. |-- 3.3 | XXX | docs.yp.org/bitbake/<REL 3.2> | | `-- index.html | | | | | | | |-- bitbake | | | | | | | | | |-- XXX | | | YP release 3.2 docs. | | `-- index.html | | | | | | | | | |-- XXX | | | YP release 3.3 docs. | | `-- index.html | | | 88594 1 ndec13 2020-11-12 13:54:22 +0000 a new builder in the AB was defined which monitors the yocto-docs and bitbake branches, and build/publish to the docs website. docs.yp.org is then fully generated from https://autobuilder.yoctoproject.org/typhoon/#/builders/114 this takes care of building all 'branches'. we now need a mechanism/script to publish a 'release or tag' (e.g. docs.yp.org/3.2) 88775 2 ndec13 2020-12-15 07:42:05 +0000 I believe we could close this one. the only thing missing really is a tool or script (or set of instructions) to create a doc release, but the bulk of what this bug was about is now properly implemented and automated in the AB. If we close this one, I can create another one to track the 'docs release process'. 88931 3 randy.macleod 2021-01-14 16:03:25 +0000 Yes please close this one this week and create the other bug. Thanks. 88981 4 randy.macleod 2021-01-21 16:04:08 +0000 Nico, ping? 89851 5 akuster808 2021-04-01 14:58:48 +0000 isn't this done? 89855 6 ndec13 2021-04-01 15:38:42 +0000 the publishing of the yp docs web site is implemented in the AB. We can close this bug. there are further enhancements we want to make, but we can open bugs for them as needed!