Bug 10749 - Add a class to update package index automatically
Summary: Add a class to update package index automatically
Status: NEW
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: core (show other bugs)
Version: 2.2
Hardware: x86 Multiple
: Medium enhancement
Target Milestone: 6.99
Assignee: Unassigned
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2016-11-30 20:11 UTC by Paul Eggleton
Modified: 2026-03-05 16:09 UTC (History)
5 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Paul Eggleton 2016-11-30 20:11:28 UTC
If you have a target machine with an image on it that has package management enabled pointing to a feed served from your build machine, you can install / update packages from it as they get built. However, you currently have to remember to explicitly run bitbake package-index every time so that the index is up-to-date before you run the package manager. It would be fairly easy to write a class that used a bb.event.BuildCompleted event handler to run the same thing that package-index does at the end of every build so that the index is always up-to-date - it could even try to be smart and only do it when it needs to, e.g. check if the mtime of any files under the package feed has changed.
Comment 1 Ross Burton 2016-12-15 17:01:29 UTC
I love it when "fairly easy" turns out to be otherwise.

BuildComplete handlers can't have dependencies, so there's no way of saying that it needs apt-native etc to execute.  We can't have the class add PACKAGEINDEXDEPS to DEPENDS as that introduces circular build dependencies.  We need the dependencies in case packages are pulled in from sstate and the native tools are not.

I'm now profiling the index generation and wonder if we should have an option to run it after every do_package_*...
Comment 2 Ross Burton 2016-12-15 17:18:19 UTC
Hm, package-index takes 10 seconds (just the generate_index_files call) after building a core-image-minimal, so that's way too heavy for every do_package.
Comment 3 brian avery 2017-06-27 16:19:16 UTC
Is there a way to schedule this after all rpm's are done rather than for each one by looking at all the targets, maybe?  It might also be nice to be able to turn this on and off with a variable as it is very frustrating to forget to run the package-index and have the dnf install on the target fail. but if I'm not maintaining a package index. it is pretty useless to run it.
Comment 4 brian avery 2017-07-18 21:25:02 UTC
Paul's idea was to have it be a last run type operation like buildhistory, which is also controlled by a local.conf setting; which seems like a good fit to me. adding paul+henry to cc.
Comment 5 Robin Jordan 2018-02-26 20:35:35 UTC
Please provide a status on this.  Will it make it into YP 2.5 M3?
Comment 6 Robin Jordan 2018-02-26 20:41:27 UTC
Appologies...I see that it has been pushed.
Comment 7 Ross Burton 2020-01-29 16:31:58 UTC
poky-contrib:ross/index has a prototype of this.
Comment 8 Ross Burton 2020-01-29 17:03:59 UTC
So it sort of works.  Need to repeat the path generalising I did for opkg to rpm so it can find the binaries it needs.
Comment 9 Ross Burton 2020-01-29 17:04:50 UTC
I lie, it actually works. \o/

Needs to not spam the console with logging from the index.
Comment 10 Ross Burton 2020-02-21 12:06:21 UTC
Another note: the class should at least warn if PRSERV_HOST isn't set, as a feed without prserv isn't useful.
Comment 11 Ross Burton 2022-04-28 15:08:53 UTC
Bruce: is this something the binary distros working group is interested in?
Comment 12 Bruce Ashfield 2022-04-28 16:14:50 UTC
Yep, this is something that would be handy as part of a "getting started" guide to extending/generating the feeds.

"inherit this class" (or we template it), and you are off to the races with a feed.

I'd think we'd also want that same enable class to consider where the packages are hosted/served, so it could optionally rsync them AND make sure the on-target package manager has the details. But obviously, those are extensions / next steps.
Comment 13 Randy MacLeod 2023-04-21 18:45:43 UTC
Moved en masse from 4.2 to 4.3 by Randy.