Bug 6966 - Ipk-based images contain unusued build process feed info on /var/lib/opkg/lists
Summary: Ipk-based images contain unusued build process feed info on /var/lib/opkg/lists
Status: VERIFIED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: core (show other bugs)
Version: 1.7
Hardware: x86 Multiple
: Medium major
Target Milestone: 2.1 M3
Assignee: Alejandro del Castillo
QA Contact: Bogdan Alexandru Voiculescu
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2014-11-13 20:44 UTC by Alejandro del Castillo
Modified: 2016-03-22 12:36 UTC (History)
7 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alejandro del Castillo 2014-11-13 20:44:09 UTC
I noticed that the OE build process is leaving files at /var/lib/opkg/lists (~1.5 MB on my system), which I believe are the feed content used to build the image. The target opkg.conf file configure lists_dir to be at /var/lib/opkg (instead of the default /var/lib/opkg/lists), so the feed content under /var/lib/opkg/lists is not used at all. 

Commit d950ef40a3eae4b54cc62828e66d84a62d78c447 removed:

rm -f ${IMAGE_ROOTFS}${OPKGLIBDIR}/opkg/lists/*

step which was not implemented on rootfs.py.
Comment 1 Alejandro Hernandez 2015-06-11 21:46:05 UTC
I am not able to reproduce this, /var/lib/opkg/lists does in fact exist, but it seems to be empty in my builds, is there anything else on your configuration that might be affecting this?
Comment 2 Alejandro del Castillo 2015-07-09 20:40:06 UTC
You are right, the folder is incorrectly left-over, but it is empty (I must have been running another test, which filled it).

From what I can tell, opkg default lists_dir folder is /var/lib/opkg/lists, which is created during opkg update if it doesn't exists. If the conf file /etc/opkg/opkg.conf doesn't have a lists_dir entry, then it goes with the default. I think that's what happens during the offline root image creation.

The inconsistency comes from the fact that the opkg-collateral recipes defaults lists_dir to /var/lib/opkg:

echo "lists_dir ext ${OPKGLIBDIR}/opkg" >>${WORKDIR}/opkg.conf
Comment 3 Stephen K Jolley 2015-07-13 20:04:03 UTC
It appears the NEEDINFO has been completed.   Please put this back in NEEDINFO if there are still questions.
Comment 4 Alejandro Hernandez 2015-07-14 18:05:58 UTC
(In reply to comment #2)
> You are right, the folder is incorrectly left-over, but it is empty (I must
> have been running another test, which filled it).
> 
> From what I can tell, opkg default lists_dir folder is /var/lib/opkg/lists,
> which is created during opkg update if it doesn't exists. If the conf file
> /etc/opkg/opkg.conf doesn't have a lists_dir entry, then it goes with the
> default. I think that's what happens during the offline root image creation.
> 
> The inconsistency comes from the fact that the opkg-collateral recipes
> defaults lists_dir to /var/lib/opkg:
> 
> echo "lists_dir ext ${OPKGLIBDIR}/opkg" >>${WORKDIR}/opkg.conf

Ok, we should probably solve inconsistencies by using /var/lib/opkg/lists, but I still need to check, because this "default" behavior might be caused by the line EXTRA_OE_CONF = "--with-opkglibdir=${OPKGLIBDIR}" in our recipe, which according to opkg's git repository, that option was replaced since opkg v0.3.0
Comment 5 Alejandro Hernandez 2015-07-31 18:48:35 UTC
Well... for starters were barely on opkg v0.2.4, secondly about some of the comments before, depending on how we're adding package management to the image we get two different behaviors, if we do it via an IMAGE_FEATURE the lists are populated, hence the extra 1.5MB mentioned before, but if we do it via installing the opkg package, then the list aren't populated initially, this is apparently desired behavior so we should leave it to the user to decide which behavior is more accurate for a specific applciation.

In any way, inconsistencies should be resolved by using the same lists directory which will end up being "/var/lib/opkg/lists", sending patch to list.
Comment 7 Bogdan Alexandru Voiculescu 2015-10-20 13:12:36 UTC
Verified on master e35c404537db0b46047fcb2ee7d3645e3e0935c5
Directory is updated after: opkg update with correct information.


Still the original information is NOT correct: oe, oe-all, oe-i586 and oe-qemux86 are pointing to some directories that were on the system where qemu image was built!    

root@qemux86:/var/lib/opkg/lists# ls -ila
   2338 drwxr-xr-x    2 root     root          1024 Oct 20 12:36 .
    833 drwxr-xr-x    4 root     root          1024 Oct 20 11:29 ..
   9442 -rw-r--r--    1 root     root         37868 Oct 20 13:01 all
   9446 -rw-r--r--    1 root     root       6032588 Oct 20 13:01 i586
   2339 lrwxrwxrwx    1 root     root            59 Oct 20 12:16 oe -> /home/yocto/yocto/BBB/Bug8048/build/tmp/deploy/ipk/Packages
   2340 lrwxrwxrwx    1 root     root            63 Oct 20 12:16 oe-all -> /home/yocto/yocto/BBB/Bug8048/build/tmp/deploy/ipk/all/Packages
   2341 lrwxrwxrwx    1 root     root            64 Oct 20 12:16 oe-i586 -> /home/yocto/yocto/BBB/Bug8048/build/tmp/deploy/ipk/i586/Packages
   2342 lrwxrwxrwx    1 root     root            67 Oct 20 12:16 oe-qemux86 -> /home/yocto/yocto/BBB/Bug8048/build/tmp/deploy/ipk/qemux86/Packages
   9444 -rw-r--r--    1 root     root        332247 Oct 20 13:01 qemux86

Observe 
root@qemux86:/var/lib/opkg/lists# ls oe-i586
ls: oe-i586: No such file or directory
Comment 8 Stephen K Jolley 2015-11-19 15:55:41 UTC
This was REOPENED
Comment 9 Alejandro Hernandez 2016-02-10 18:36:21 UTC
(In reply to comment #7)
> Verified on master e35c404537db0b46047fcb2ee7d3645e3e0935c5
> Directory is updated after: opkg update with correct information.
> 
> 
> Still the original information is NOT correct: oe, oe-all, oe-i586 and
> oe-qemux86 are pointing to some directories that were on the system where
> qemu image was built!    
> 
> root@qemux86:/var/lib/opkg/lists# ls -ila
>    2338 drwxr-xr-x    2 root     root          1024 Oct 20 12:36 .
>     833 drwxr-xr-x    4 root     root          1024 Oct 20 11:29 ..
>    9442 -rw-r--r--    1 root     root         37868 Oct 20 13:01 all
>    9446 -rw-r--r--    1 root     root       6032588 Oct 20 13:01 i586
>    2339 lrwxrwxrwx    1 root     root            59 Oct 20 12:16 oe ->
> /home/yocto/yocto/BBB/Bug8048/build/tmp/deploy/ipk/Packages
>    2340 lrwxrwxrwx    1 root     root            63 Oct 20 12:16 oe-all ->
> /home/yocto/yocto/BBB/Bug8048/build/tmp/deploy/ipk/all/Packages
>    2341 lrwxrwxrwx    1 root     root            64 Oct 20 12:16 oe-i586 ->
> /home/yocto/yocto/BBB/Bug8048/build/tmp/deploy/ipk/i586/Packages
>    2342 lrwxrwxrwx    1 root     root            67 Oct 20 12:16 oe-qemux86
> -> /home/yocto/yocto/BBB/Bug8048/build/tmp/deploy/ipk/qemux86/Packages
>    9444 -rw-r--r--    1 root     root        332247 Oct 20 13:01 qemux86
> 
> Observe 
> root@qemux86:/var/lib/opkg/lists# ls oe-i586
> ls: oe-i586: No such file or directory

Again, I am not able to reproduce this, I tried both methods (including opkg on the image, and also via IMAGE_FEATURES, to me the "lists" doesn't exist, it is created after running opkg update, but still after it's empty:

root@qemux86:/var/lib/opkg# ls
root@qemux86:/var/lib/opkg# opkg update
root@qemux86:/var/lib/opkg# ls
info  lists
root@qemux86:/var/lib/opkg# cd lists/
root@qemux86:/var/lib/opkg/lists# ls -l
total 0

Could this be happening because of another test you are running?
Comment 10 Bogdan Alexandru Voiculescu 2016-03-22 12:35:51 UTC
Looks ok now.
Comment 11 Bogdan Alexandru Voiculescu 2016-03-22 12:36:16 UTC
Verified on master 8037ba4d86a65ee296cfdcee736dfd614c568818