2421 2012-05-04 13:33:09 +0000 [Performance] Further improvements for turning off doc generation for -native (and similar) packages 2017-06-13 10:52:25 +0000 1 1 7 Build System, Metadata & Runtime Meta-yocto meta-yocto 1.3 x86 Multiple RESOLVED FIXED Low enhancement Future 1 richard.purdie unassigned elizabeth.flanagan max.eliaser poky.bs.watcher poky.watcher ross.burton sgw --- --- Yes (doc changes required) oldest_to_newest 21104 0 richard.purdie 2012-05-04 13:33:09 +0000 There is no point in generating documentation for -native packages (or -cross for that matter). We should see if this gives any performance improvement (both speed and size of sstate in particular), If so, it would be good to add PACKAGECONFIG[doc] entries to recipes and default to turning this flag off in the native case. 42483 1 max.eliaser 2014-04-09 23:26:30 +0000 Okay, I've been looking into 42558 2 max.eliaser 2014-04-10 17:12:28 +0000 I've 42561 3 max.eliaser 2014-04-10 17:49:51 +0000 Sorry about those comment fragments. I'm still new here. Anyway, I've been looking into this, and I've made some discoveries. A consequence of the design of PACKAGECONFIG is that none of the flags can be default-on. If a flag is not specified, that means "off," not "don't know if off or on, can use default." So the flag will have to be "no-doc" instead. Also, PACKAGECONFIG supports configure flags only with Autotools, CMake, and Waf-- recipies that use other build systems only get the build and run-time dependencies. For example, Perl's Module::Build seems to have a really clean separation of docs and code, but you can't get configure flags to it with PACKAGECONFIG as far as I can tell. Anyway, I'm just focusing on Autotools for now. Defining the no-doc flag in the Autotools class is easy, then it's a matter of setting the flag on by default in every -native and -cross recipe that uses Autotools. I've identified 59 such recipies, and will start making those changes now. 43152 4 max.eliaser 2014-05-07 17:00:21 +0000 OK, all recipes (native or otherwise) that require the texinfo utilities in order to build now inherit texinfo.bbclass. In trunk right now, texinfo.bbclass is still blank. In the poky-contrib repository, I've got a branch meliaser/texinfo-build that adds a texinfo-dummy-native recipe. This recipe has a set of "fake" texinfo utilities that install to a subdir and can then be conditionally used or not used by altering the PATH. These programs suppress documentation generation by simply not doing anything when they are invoked, save for creating some blank output files. (NOTE: some upstream projects use other programs from outside the Texinfo suite, like groff and pod2man, and these are neither intercepted nor suppressed by texinfo-dummy-native.) I've experimented with different combinations of using the texinfo-native recipe we ship, the texinfo-dummy-native recipe I've added, and the host system's texinfo utils, to see which combinations have the best performance. Different combinations can be achieved by putting different code in texinfo.bbclass. All comparisons are based on the time to build eglibc with a clean sstate-cache and working directory: rm -rf /yoctostate/sstate-cache/ tmp/ time bitbake -k eglibc I believe eglibc must be built before any of the other target architecture recipes, making it a major bottleneck and a good way to isolate the compilation time of the native and cross toolchain. That justifies its use as a benchmark. Since I was also doing other stuff with my workstation while running these timing runs, I'd say the fastest run in each configuration is probably closest to the true number. Completely blank texinfo.bbclass, imitating the status quo: Run 1: real 15m8.546s user 70m5.365s sys 13m56.883s Run 2: real 15m17.377s user 69m49.976s sys 13m56.380s Run 3: real 15m13.980s user 70m5.775s sys 13m58.166s With changes to texinfo.bbclass to suppress doc gen for -native: Code: TEXDEP = "texinfo-native" TEXDEP_class-native = "texinfo-dummy-native" DEPENDS_append = " ${TEXDEP}" PATH_prepend_class-native = "${STAGING_BINDIR_NATIVE}/texinfo-dummy-native:" Run 1: real 15m25.430s user 70m17.126s sys 14m6.556s Run 2: real 15m38.784s user 70m2.807s sys 14m6.507s Run 3: real 15m32.600s user 70m32.279s sys 14m7.440s Suppressing doc gen for -native *and* -cross: Code: TEXDEP = "texinfo-native" TEXDEP_class-native = "texinfo-dummy-native" TEXDEP_class-cross = "texinfo-dummy-native" DEPENDS_append = " ${TEXDEP}" PATH_prepend_class-native = "${STAGING_BINDIR_NATIVE}/texinfo-dummy-native:" PATH_prepend_class-cross = "${STAGING_BINDIR_NATIVE}/texinfo-dummy-native:" Run 1: real 15m4.869s user 70m26.672s sys 14m15.161s Run 2: real 15m23.064s user 70m50.325s sys 14m16.383s Run 3: real 15m2.125s user 70m28.214s sys 14m26.571s Suppressing doc gen for *all* recipes: (NOTE: this saves building texinfo-native, which is likely the main source of the speedup) Code: DEPENDS_append = " texinfo-dummy-native" PATH_prepend = "${STAGING_BINDIR_NATIVE}/texinfo-dummy-native:" Run 1: real 14m14.873s user 68m43.605s sys 13m57.721s Run 2: real 14m21.572s user 68m43.443s sys 13m58.856s Run 3: real 14m21.632s user 68m41.563s sys 13m55.674s Suppressing doc gen for -native and -cross, doc gen using host system's texinfo for target arch: Code: DEPENDS_append_class-native = " texinfo-dummy-native" DEPENDS_append_class-cross = " texinfo-dummy-native" PATH_prepend_class-native = "${STAGING_BINDIR_NATIVE}/texinfo-dummy-native:" PATH_prepend_class-cross = "${STAGING_BINDIR_NATIVE}/texinfo-dummy-native:" Run 1: real 14m49.500s user 69m28.189s sys 14m8.641s Run 2: real 14m54.150s user 69m14.851s sys 14m9.262s Run 3: real 15m5.754s user 69m23.275s sys 14m10.814s 43346 5 max.eliaser 2014-05-14 21:26:34 +0000 At Richard Purdie's suggestion, the final code is a variation on the 3rd test case of the previous post, but with texinfo-native added to ASSUME_PROVIDED, which causes it to behave identically to the last test case. I only did one timing run, and the results are close to what I was expecting: real 14m42.082s user 69m21.533s sys 14m4.483s Pretty much in the same range as the last test case of my previous comment. I am submitting a patch series with this code now. 45380 6 max.eliaser 2014-09-03 22:13:12 +0000 Unassigning this bug given my pending departure. The texinfo stuff works well, but there are other systems, like groff, that could also be attacked. Perl's module build system (which we have a class for already) has a really clean way to disable documentation building that could be used. Ignore references to a universal Autotools configure flag to disable doc generation, as mentioned in previous comments. It doesn't exist. 74047 7 ross.burton 2017-06-13 10:52:25 +0000 There's now an api-documentation DISTRO_FEATURE which the manpages and gtk-doc classes respect and turn off for native. I consider this closed and there's now a standard way to do it.