Bug 1088

I do not have build or runtime statistics regarding this issue beyond that of Gary's.  I have sent out an RFI email in hopes of gathering more information on the issue.  Once I have some solid stuff I will continue.

Scott: Joshua did give you links to some info:

https://lists.yoctoproject.org/pipermail/poky/2011-May/006363.html
https://lists.yoctoproject.org/pipermail/poky/2011-May/006362.html

so we should be able to sort this bug out?

The ultimate fix for this bug is a new chapter that will be put in the YP development manual that addresses build performance, which would include speed and space.  I can see space issues and speed issues partially addressed by the package type being used.  I will get that section in the YP dev manual eventually. 

For now, I would like to propose some package information and references in several areas of the YP documentation set:

* YP Ref Manual section C.12 "Packaging"
* Glossary item PACKAGE_CLASSES
* YP QS "Building an Image"
* ADT Manual "Package Management Systems" and "'Configuring the PMS"

I will use the baseline 30% more time for RPM as the general piece of information.

How about we do that but also include the two mailing list links? The links are to mailing list archives and should be stable...

No problem.  I will work the links into the text as well.

To address this bug, I have added information to three manuals: The Yocto Project Reference Manual, the Yocto Project Quick Start, and the Yocto Project ADT Manual.  There will eventually be a separate area in the Yocto Project Development Manual as well but I don't know if I can address that spot before 1.1 releases.

Here are the sections:


YP REFERENCE MANUAL UNDER REFERENCE: CLASSES APPENDIX....

C.12. Packaging - package*.bbclass

The packaging classes add support for generating packages from a build's output. The core generic functionality is in package.bbclass. The code specific to particular package types is contained in various sub-classes such as package_deb.bbclass, package_ipk.bbclass, and package_rpm.bbclass. Most users will want one or more of these classes. 

You can control the list of resulting package formats by using the PACKAGE_CLASSES variable defined in the local.conf configuration file found in the Yocto Project file's conf directory. When defining the variable, you can specify one or more package types. Since images are generated from packages, a packaging class is needed to enable image generation. The first class listed in this variable is used for image generation. 

The package class you choose can affect build-time performance and has space ramifications. In general, building a package with RPM takes about thirty percent more time as compared to using IPK to build the same or similar package. This comparison takes into account a complete build of the package with all dependencies previously built. The reason for this discrepancy is because the RPM package manager creates and processes more metadata than the IPK package manager. Consequently, you might consider setting PACKAGE_CLASSES to "package_ipk" if you are building smaller systems. 

Keep in mind, however, that RPM starts to provide more abilities than IPK due to the fact that it processes more metadata. For example, this information includes individual file types, file checksum generation and evaluation on install, sparse file support, conflict detection and resolution for multilib systems, ACID style upgrade, and repackaging abilities for rollbacks. 

Another consideration for packages built using the RPM package manager is space. For smaller systems, the extra space used for the Berkley Database and the amount of metadata can affect your ability to do on-device upgrades. 

You can find additional information on the effects of the package class at these two Yocto Project mailing list links: 

	https://lists.yoctoproject.org/pipermail/poky/2011-May/006362.html
	https://lists.yoctoproject.org/pipermail/poky/2011-May/006363.html

ALSO IN THE YP REFERENCE MANUAL IN THE GLOSSARY...

PACKAGE_CLASSES

This variable, which is set in the local.conf configuration file found in the Yocto Project file's conf directory, specifies the package manager to use when packaging data. You can provide one or more arguments for the variable with the first argument being the package manager used to create images: 

     PACKAGE_CLASSES ?= "package_rpm package_deb package_ipk"
                    
For information on build performance effects as a result of the package manager use, see Packaging - package*.bbclass in this manual. 



IN THE YP QUICK START WITHIN THE BUILDING AN IMAGE SECTION....

Another consideration before you build is the package manager used when creating the image. By default, the Yocto Project build system uses the RPM package manager. You can control this configuration by using the PACKAGE_CLASSES variable. For additional package manager selection information, see Packaging - package*.bbclass in The Yocto Project Reference Manual. 


NOTE I
IN THE YP ADT MANUAL WHERE IT TALKS ABOUT PACKAGE CONFIGURATION....

3.2. Configuring the PMS

Whichever PMS you are using, you need to be sure that the PACKAGE_CLASSES variable in the conf/local.conf file is set to reflect that system. The first value you choose for the variable specifies the package file format for the root filesystem at sysroot. Additional values specify additional formats for convenience or testing. See the configuration file for details. 

Note
For build performance information related to the PMS, see Packaging - package*.bbclass in The Yocto Project Reference Manual.

These are useful data but part of what I was asking for in this bug is more about the how and why of using the various packaging systems.  For example, how does one set up external package repositories and then how to make them useful for the target system.

In particular, I had to figure out by trial and error how to use opkg since I had a hard time finding any documentation on it.  Having a section in the manuals on how to set up an opkg package server as well as how that might be used on the target would be useful.  This is especially important when there are multiple packaging systems and choices since both the server and target setups differ depending on the packaging choice.

Gary, 

What you suggest would be great to have in the YP development manual.  Do you have any raw information that I could use to build up a section such as you suggest?  You will notice that what I included is just build performance info related between two of the three packaging systems: RPM and IPK.  It would be great to have information such as you suggest for all three systems.  I don't have the ability to gather this type of information.  So, if you have any information that would lead to a more complete fix for this bug it would be great.

Thanks.

(In reply to comment #8)
> Gary, 
> 
> What you suggest would be great to have in the YP development manual.  Do you
> have any raw information that I could use to build up a section such as you
> suggest?  You will notice that what I included is just build performance info
> related between two of the three packaging systems: RPM and IPK.  It would be
> great to have information such as you suggest for all three systems.  I don't
> have the ability to gather this type of information.  So, if you have any
> information that would lead to a more complete fix for this bug it would be
> great.
> 
> Thanks.

I'll send you what I know about opkg.  Hopefully you can take that and put it into a useful format in the manual.

Mark Hatle is probably the best source for info on RPM.

Changing status to NEEDINFO.

Hi, 

Trevor Woerner was wondering about the status of this bug.  It was filed a long time ago and since then there has been a fair amount of "packaging" information put into the manual.  Whether that information covers this particular bug I don't know.  I would like others opinions on whether or not we have enough packaging information in the doc set to RESOLVE this bug.  

The mega-manual (http://www.yoctoproject.org/docs/1.5/mega-manual/mega-manual.html) is a good link to load and then CTRL-F search for "opkg", "package", etc to get a feel for the type of information covered in the set for packaging.  If you care to, please look through the mega-manual and comment. 

Thanks, 
Scott

Gary, any comments?

(In reply to comment #12)
> Gary, any comments?

I'm quite happy with the current explanation in the mega manual.

Feel free to close this bug.

Great, thanks.