Bug 11290

Taking now a closer look, this has never been supported.

AFAIU, nothing in bitbake is broken. The bitbake fetcher is written so that it doesn't calculate checksums if the build system is not supporting them. If I read your description correctly, you are using the stamp files in some internal tool to determine if sources need to be re-synced(?) Couldn't you just use rsync or something.

If I undestood everything correctly, I would change this to a feature request (or close as wontfix). Changing now to needinfo and assigning back to reporter.

In that case, please make this a feature request. It would be really nice that git repos would behave the same way as other mirrored packages. 

We also use rsync as the backend but still require the pickle data for the packages to have everything working correctly.

Assigning back, please change status/assignee if this was not correct.

WaitForUpstream is inappropriate, please don't use it.

Resetting back to NEEDINFO to Tomas.  What do you expect to be in the donestamp for a git tarball?  There's no point putting the SHAs in it as nothing in bitbake will check them.  I'd suggest that your tooling is making incorrect assumptions that every donestamp contains a pickle with checksums in.

I would expect the donestamp to contain the SHA for the entire repository the same way as it has been archived by bitbake. Indeed, the SHA for the single revision used for building could be enough if git repos were used by bitbake that way. 

Would it make sense for bitbake to also be aware of the checksum to know when to re-archive the repository and when to skip archiving?

He appears to have answers your question.

There is no single revision/SHA that would unambiguously define the state a Git repository. It could contain the checksum of the git repository tarball, though(?)

Yes, checksum of the git repository tarball is how I would also assume it to be done. That would be analogous to other archive handling.

I'm still not sure about this to be honest.  The donestamp for files which have been downloaded over HTTP is the *expected* checksum.  There is no way to provide the expected checksum for a generated tarball.

If you're using the donestamps to determine if files need to re-synced, why not just use timestamps or the actual checksum of the file?

In the case of most tarballs they have a specific checksum which a tarball of that name represents. The use of this by bitbake is very specific, it means bitbake downloaded the file, calculated a checksum of the download and wrote it into the file. This checksum is compared against the checksum in the recipe and if they don't match, the build is aborted. It doesn't have to re-calculate the checksum each time it wants to use it.

Mirror tarballs have varied contents depending on what was in the git repository at the time of creation. There is no one checksum which represents this and there is no guarantee than any given mirror tarball would have any given checksum.

Integrity handling of git repos is handled at the git level (ditto other scm formats).

If we were to add a checksum here what would its purpose be? Regardless, it would have to have a different meaning and usage to existing checksums and this would be extremely confusing.

Since we don't need it an the semantics don't match I can't see a good reason to add this.

Note that bitbake will regenerate these tarballs any time it updates the main git repo (i.e. its potentially fetched something). The file's timestamp is going to be just as good a measure of whether its potentially changed whether or not a checksum is written into the done stamp.