I've noticed this behavior with externalsrc.bbclass which does:
d.setVar('S', externalsrc)
in https://git.openembedded.org/openembedded-core/tree/meta/classes/externalsrc.bbclass#n60

If the externalsrc is set from EXTERNALSRC defined as:

inherit externalsrc
EXTERNALSRC = "${TOPDIR}/shit/bsp"
EXTERNALSRC[vardepsexclude] += "TOPDIR"

Then comparing signatures from 2 builds with TOPDIR kirkstone-official2 and kirkstone-official3:

$ bitbake-diffsigs BUILD/sstate-diff[23]/externalsrc-topdir-shit-bsp-exclude/*/*/key*/*do_deploy_source*
basehash changed from f76d3261854f98ba1ebbea7eba7c2304f35fffaac39b1374c6907a1ac79d79d5 to 01f3f898bb59327fbf3620deebc6d6c6e5e1a1eb874813f3bf66c3eeab7c2308
Variable S value changed from '/kirkstone-official2/shit/bsp' to '/kirkstone-official3/shit/bsp'

Which is the same as without the vardepsexclude as in:

inherit externalsrc
EXTERNALSRC = "${TOPDIR}/shit/bsp"

$ bitbake-diffsigs BUILD/sstate-diff[23]/externalsrc-topdir-shit-bsp/*/*/key*/*do_deploy_source*
basehash changed from f76d3261854f98ba1ebbea7eba7c2304f35fffaac39b1374c6907a1ac79d79d5 to 01f3f898bb59327fbf3620deebc6d6c6e5e1a1eb874813f3bf66c3eeab7c2308
Variable S value changed from 'kirkstone-official2/shit/bsp' to 'kirkstone-official3/shit/bsp'

while this is obviously fine:
inherit externalsrc
EXTERNALSRC = "/shit/bsp"

$ bitbake-diffsigs BUILD/sstate-diff[23]/externalsrc-shit-bsp/*/*/key*/*do_deploy_source*
$

I know that externalsrc is intended for fast local incremental builds and has many disadvantages compared to bitbake correctly fetching, tracking and maintaining the sources in S.

That it isn't mean to be used like this, in builds executed on different servers which might use different TOPDIR, but want to share sstate between the builds.

This is used by BSP from 3rd party where they maintain sources outside the metadata with "repo" and the recipes are then just pointing EXTERNALSRC to various directories inside this "repo" checkout.

I've already suggested not to use this BSP or ask the vendor not to use this, but it probably won't happen and this issue might be interesting in other cases than externalsrc as well.

In the meantime I've also suggested to at least to write a "migration" script which will convert the repo manifest into a list of SRC_URI/SRCREV overrides for recipes which were using EXTERNALSRC, so that when 3rd party updates the layers and repo, we can more easily refresh the revisions for new version.