Bug 1500 - Document invalidation of the sstate on a per class basis
Summary: Document invalidation of the sstate on a per class basis
Status: RESOLVED FIXED
Alias: None
Product: Meta-yocto
Classification: Build System, Metadata & Runtime
Component: meta-yocto (show other bugs)
Version: unspecified
Hardware: x86 Multiple
: Low enhancement
Target Milestone: 1.2
Assignee: Scott Rifenbark
QA Contact:
URL: 7493f5e7ec641e7cd79f8e3e6c045cadf7a6999c
Whiteboard: 14-dec-2011: Resolved/fixed
Depends on:
Blocks:
 
Reported: 2011-09-20 08:43 UTC by Mark Hatle
Modified: 2011-12-14 10:39 UTC (History)
4 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Mark Hatle 2011-09-20 08:43:11 UTC
In the recent work for 1499, a change was needed to rpmdeps output.  The result of this change is that all of the "package", and "package_write_rpm"/deploy-rpm sstate-cache items should have been invalidated.  However there is currently no mechanism to invalidate certain elements of the sstate-cache based on external changes.

I would suggest that we add an ability for classes, and recipes to inject additional versioning information into specific steps for the use by the sstate system to determine if part of the cache needs to be invalidated.  In this case, I would have wanted to invalidated the 'package' step and expected the 'package_write_...' steps to be regenerated as an artifact of that change.
Comment 1 Richard Purdie 2011-09-20 12:36:41 UTC
This isn't strictly true, all you need to do is make some change to th package_rpm.bbclass code itself and that would trigger all the rpm packages to get regenerated. This could be some version number you add there or we could just rely on making some changes to the code itself (even a comment at the top of the main function would likely be enough).
Comment 2 Mark Hatle 2011-09-20 13:02:42 UTC
The place where I hit it, was because I changed "rpmdeps" which is used by package.bbclass, which the output of which is used by package_write_rpm.

So simply changing package_rpm.bbclass would not have regenerated the RPM packages properly, as the built-in dependency information would still be wrong -- until package.bbclass was re-run.

If we can simply make a small change to package.bbclass (say a phantom version variable or something) that would be enough to fix the issue I was seeing.

The trick of course is for someone to know that if I change rpmdeps within rpm-native, that I need to uprev the version in packages otherwise we don't regenerate.
Comment 3 Richard Purdie 2011-09-20 14:05:29 UTC
Yes, changing a version in package.bbclass would work in this case (and the rpms would then rebuild since they depend on that).
Comment 4 Richard Purdie 2011-10-07 05:42:11 UTC
I'm going to change this to a documentation issue for sstate. I believe we can invalidate the cache, we just need to better expose the mechanism.

The change in question is http://git.yoctoproject.org/cgit.cgi/poky/commit/meta/classes/package.bbclass?id=737f8bbb4f27b4837047cb9b4fbfe01dfde36d54
Comment 5 Richard Purdie 2011-11-25 09:02:52 UTC
Proposed manual addition:

"""
If its desired to change the checksum of a given subset of tasks, maybe
due to a change which isn't directly visible in the code itself (e.g. a
tool changed its output) its possible to do this by changing a function
comments since the sstate checksums include the body of functions. To
invalidate package sstate files for example, do_package or one of the
functions it calls can be changed, even if its just a cosmetic change to
the commends.
http://git.yoctoproject.org/cgit.cgi/poky/commit/meta/classes/package.bbclass?id=737f8bbb4f27b4837047cb9b4fbfe01dfde36d54 is an example of a commit which does this.
"""
Comment 6 Scott Rifenbark 2011-12-14 10:39:29 UTC
New chapter introduced into the YP Reference manual.  This chapter is titled "Technical Details" and contains information currently about YP components and shared state code.  The section that describes shared state includes some history behind how the problem was solved and ways to deal with specific problems around it.  One of the problems addresses how to invalidate the sstate on a per class basis.  This information is in "Tips and Tricks."

The text is on review and might have some edits to it later.

To see the text, go here:

http://www.yoctoproject.org/docs/latest/poky-ref-manual/poky-ref-manual.html#shared-state-cache