When BitBake store the list of checksums for files in SRC_URI in the siginfo file, the full paths to the files are used; if you move your source directory and then do a bitbake-diffsigs between two build runs this will result in that reporting the path change as a difference even though the value that goes into the signature has not changed, which is less than ideal. We should be normalising the paths (probably making them relative to the recipe path) to avoid this.
Fix merged to master: http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=e9662e892653dcb8176a5308da1a5a829fc3f339 (In the end I opted to completely remove the path since making it relative to the recipe path wouldn't work for recipes bbappended across multiple layers.)