1894 2012-01-11 22:41:01 +0000 Incremental ipk image generation 2014-02-26 02:00:17 +0000 1 1 7 Build System, Metadata & Runtime OE-Core core 1.2 x86 Multiple RESOLVED FIXED (Test Case Ready) Low enhancement 1.6 M4 1 liezhi.yang hongxu.jia hongxu.jia liezhi.yang meta.mr.watcher meta.watcher richard.purdie sgw song.liu yi.zhao --- --- Don't know oldest_to_newest 18099 0 liezhi.yang 2012-01-11 22:41:01 +0000 Please refer to http://bugzilla.pokylinux.org/show_bug.cgi?id=1651 we divide the 1651 into 3 part: the rpm, deb and ipk, the ipk and deb part will be done in M3, this one is for ipk. 18823 1 liezhi.yang 2012-03-01 09:30:38 +0000 I found that the following two tasks can't be done because of the limitation of opkg-cl and apt-get utilities: 1894 Incremental IPK image generation (Developing) 1893 Incremental DEB image generation (May not do it in 1.2 as talked before) If we want the incremental image generation, we need the following 3 types information: 1) which pkg would be newly installed 2) which pkg would be removed 3) which pkg has been changed since the last generation To get the 3 types of information, we need to know the pkg list that has been installed, and the pkg list that *to be* installed, for the rpm pkg, this is ok, (and the RPM part has been done in M2), but for the ipk or deb, we can't the integrity pkg list that would be installed, the opkg-cl or apt-get would install the pkg according to the "Depends" and "Recommends", also maybe the "Suggests" or "Conflicts" (Priority?), I tried to write a script to figure out the pkg that to be installed, but the result is not what we need: 1) If I just grab the pkg from "Depends" area, then the result would be less than the pkgs that really installed. 2) If I grab the pkg from both the "Depends" and "Recommends" area, then the result(pkg_to_be) would be more than the pkgs that really installed (pkg_installed), and some pkg in pkg_installed is still not in pkg_to_be list Things seems complicated than what I thought before, we may need re-implement the algorithm of opkg-cl and apt-get, but this seems not worth. So I suggested that we don't do it any more. 29026 2 hongxu.jia 2013-01-22 11:45:10 +0000 Support incremental ipk image generation, here is the design: 1) Config opkg 2) If build deb image from zero: 1) Install packages: lingus, normal and attemptonly. 2) Dump solution manifest for the next incremental image session. 3) Else if ipk incremental generation is supported: 1) Dump solution manifest which lists *to be installed* packages names 2) Compare solution manifest with previous installed manifest to dump unused package names and remove them. 3) Dump updated packages list and reinstall them. 4) Attempt to correct broken dependencies which caused by remove. 4) Change solution manifest to installed manifest for next image session. 5) Back up opkg database for next image session. 36143 3 hongxu.jia 2013-09-02 09:02:34 +0000 Move to 1.6 40411 4 hongxu.jia 2014-02-18 09:45:04 +0000 package_manager.py/ootfs.py: support ipk incremental image generation Test Cases --------------------------------------------- Case 1: Rebuild in place 1. Edit local.conf, enable multilib and ipk incremental image generation ... 46 MACHINE ?= "qemux86-64" 254 require conf/multilib.conf 255 MULTILIBS = "multilib:lib32" 256 DEFAULTTUNE_virtclass-multilib-lib32 = "x86" 257 258 IMAGE_INSTALL_append = " lib32-gzip" 260 IMAGE_FEATURES_append = " package-management ssh-server-dropbear " 262 263 INC_IPK_IMAGE_GEN = "1" ... bb core-image-minimal 2. Edit local.conf, add some spaces in IMAGE_INSTALL_append vim local.conf ... 258 IMAGE_INSTALL_append = " lib32-gzip " ... 3. bb core-image-minimal, the image should not be rebuilt vim log.do_rootfs: ... 16 NOTE: Package packagegroup-core-ssh-dropbear (1.0-r1) installed in root is up to date. 17 Package opkg-collateral (1.0-r2) installed in root is up to date. 18 Package run-postinsts (1.0-r9) installed in root is up to date. 19 Package packagegroup-core-boot (1.0-r11) installed in root is up to date. 20 Package opkg (1:0.2.0-r0) installed in root is up to date. 21 Package poky-feed-config-opkg (1.0-r2) installed in root is up to date. ... 53 NOTE: Package lib32-gzip (1.6-r7) installed in root is up to date. ... vim installed_pkgs.txt ... lib32-gzip x86 ... 4. runqemu qemux86-64, ssh could work and lib32-gzip is installed On host: runqemu qemux86-64 ssh root@192.168.7.2 and run 'opkg-cl list-installed' ... lib32-gzip - 1.6-r7 ... --------------------------------------------- Case 2: Decremental Rebuild 1. Edit local.conf, enable multilib and ipk incremental image generation ... 46 MACHINE ?= "qemux86-64" 254 require conf/multilib.conf 255 MULTILIBS = "multilib:lib32" 256 DEFAULTTUNE_virtclass-multilib-lib32 = "x86" 257 258 IMAGE_INSTALL_append = " lib32-gzip openssl" 260 IMAGE_FEATURES_append = " package-management ssh-server-dropbear " 262 263 INC_IPK_IMAGE_GEN = "1" ... bb core-image-minimal 2. Edit local.conf, remove 'lib32-gzip openssl' vim local.conf ... 258 # IMAGE_INSTALL_append = " lib32-gzip openssl" ... 3. bb core-image-minimal, lib32-gzip and openssl should be removed. vim log.do_rootfs: ... 20 NOTE: Removing package lib32-eglibc from root... 21 Removing package lib32-gzip from root... 22 Removing package lib32-update-alternatives-opkg from root... 23 Removing package libcrypto1.0.0 from root... 24 Removing package libssl1.0.0 from root... 25 Removing package openssl from root... 26 Removing package openssl-conf from root... ... 31 Package opkg-collateral (1.0-r2) installed in root is up to date. 32 Package run-postinsts (1.0-r9) installed in root is up to date. 33 Package packagegroup-core-boot (1.0-r11) installed in root is up to date. 34 Package opkg (1:0.2.0-r0) installed in root is up to date. 35 Package poky-feed-config-opkg (1.0-r2) installed in root is up to date. ... --------------------------------------------- Case 3: Incremental Rebuild 1. Edit local.conf, enable multilib and ipk incremental image generation ... 46 MACHINE ?= "qemux86-64" 254 require conf/multilib.conf 255 MULTILIBS = "multilib:lib32" 256 DEFAULTTUNE_virtclass-multilib-lib32 = "x86" 257 258 # IMAGE_INSTALL_append = " lib32-gzip openssl" 260 IMAGE_FEATURES_append = " package-management ssh-server-dropbear " 262 263 INC_IPK_IMAGE_GEN = "1" ... bb core-image-minimal 2. Edit local.conf, append 'lib32-gzip openssl' to IMAGE_INSTALL vim local.conf ... 258 IMAGE_INSTALL_append = " lib32-gzip openssl" ... 3. bb core-image-minimal, lib32-gzip and openssl should be added. vim log.do_rootfs: ... 20 NOTE: Package packagegroup-core-ssh-dropbear (1.0-r1) installed in root is up to date. 21 Package opkg-collateral (1.0-r2) installed in root is up to date. 22 Package run-postinsts (1.0-r9) installed in root is up to date. 23 Installing openssl (1.0.1e-r15.0) to root... ... 31 Package packagegroup-core-boot (1.0-r11) installed in root is up to date. 32 Package opkg (1:0.2.0-r0) installed in root is up to date. 33 Package poky-feed-config-opkg (1.0-r2) installed in root is up to date. ... 54 NOTE: Installing the following packages: lib32-gzip ... --------------------------------------------- Case 4: Rebuild in place and Upgrade package 1. Edit local.conf, enable ipk incremental image generation ... 258 IMAGE_INSTALL_append = " gzip" 263 INC_IPK_IMAGE_GEN = "1" ... bb core-image-minimal 2. Edit local.conf, upgrade gzip's PR ... 258 PR_pn-gzip = "r1" ... 3. bb core-image-minimal, gzip should be upgraded to r1 vim log.do_rootfs: ... 21 NOTE: Package packagegroup-core-ssh-dropbear (1.0-r1) installed in root is up to date. 22 Package opkg-collateral (1.0-r2) installed in root is up to date. 23 Package run-postinsts (1.0-r9) installed in root is up to date. 24 Package packagegroup-core-boot (1.0-r11) installed in root is up to date. 25 Package opkg (1:0.2.0-r0) installed in root is up to date. 26 Upgrading gzip on root from 1.6-r0 to 1.6-r1... ... --------------------------------------------- Case 5: Test BAD_RECOMMENDATIONS 1. Edit local.conf, enable ipk incremental image generation ... 258 IMAGE_INSTALL_append = " util-linux" 263 INC_IPK_IMAGE_GEN = "1" ... bb core-image-minimal The util-linux-fdisk is in util-linux's RRECOMMENDS vim installed_pkgs.txt ... 41 util-linux-fdisk core2-64 ... 2. Edit local.conf, add some spaces to IMAGE_INSTALL and add util-linux-fdisk to core-image-minimal's BAD_RECOMMENDATIONS ... 260 IMAGE_INSTALL_append = " util-linux " 265 BAD_RECOMMENDATIONS_pn-core-image-minimal += "util-linux-fdisk" ... 3. bb core-image-minimal, util-linux-fdisk should not be installed vim log.do_rootfs: ... 34 NOTE: Requested ignored recommendation util-linux-fdisk is not a package ... 119 util-linux: ignoring recommendation for util-linux-fdisk at user request 120 util-linux: unsatisfied recommendation for util-linux-fdisk ... The util-linux-fdisk doesn't exist in installed_pkgs.txt any more. --------------------------------------------- Case 5: Test NO_RECOMMENDATIONS 1. Edit local.conf, enable ipk incremental image generation ... 258 IMAGE_INSTALL_append = " util-linux" 263 INC_IPK_IMAGE_GEN = "1" ... bb core-image-minimal The util-linux-fdisk util-linux-cfdisk util-linux-sfdisk util-linux-mount util-linux-readprofile util-linux-mkfs are in util-linux's RRECOMMENDS vim installed_pkgs.txt ... 4 util-linux-mkfs core2-64 18 util-linux-sfdisk core2-64 20 util-linux-mount core2-64 38 util-linux-cfdisk core2-64 41 util-linux-fdisk core2-64 46 util-linux-readprofile core2-64 ... 2. Edit local.conf, add some spaces to IMAGE_INSTALL and assign core-image-minimal's NO_RECOMMENDATIONS with 1 ... 260 IMAGE_INSTALL_append = " util-linux " 267 NO_RECOMMENDATIONS_pn-core-image-minimal = "1" ... 3. bb core-image-minimal, The six packages should not be installed vim log.do_rootfs: ... 34 NOTE: Requested ignored recommendation util-linux-fdisk is not a package ... 113 util-linux: ignoring recommendation for util-linux-sfdisk at user request 114 util-linux: unsatisfied recommendation for util-linux-sfdisk 115 util-linux: ignoring recommendation for util-linux-cfdisk at user request 116 util-linux: unsatisfied recommendation for util-linux-cfdisk 117 util-linux: ignoring recommendation for util-linux-mount at user request 118 util-linux: unsatisfied recommendation for util-linux-mount 119 util-linux: ignoring recommendation for util-linux-fdisk at user request 120 util-linux: unsatisfied recommendation for util-linux-fdisk 121 util-linux: ignoring recommendation for util-linux-mkfs at user request 122 util-linux: unsatisfied recommendation for util-linux-mkfs 123 util-linux: ignoring recommendation for util-linux-readprofile at user request 124 util-linux: unsatisfied recommendation for util-linux-readprofile ... They don't exist in installed_pkgs.txt any more. //Hongxu 40687 5 hongxu.jia 2014-02-26 02:00:17 +0000 Merged into master: commit 6ffc15655068f3da6032d8c782a8005a3f73aed9 Author: Hongxu Jia <hongxu.jia@windriver.com> Date: Fri Feb 21 15:28:48 2014 +0800 manifest.py: remove redundant import line