All machines within a layer should be run against a single command run. A common issue seems to be that users have interpreted current instructions as run the command one for each machine, but this won't be able to catch the case where two machines conflict with the recipe hashes. Instead instructions should state that the script be run with all of the machines within the layer to be tested.
CCing some WR devs as an FYI.
Richard says we may need to fix the YP autobuilder as well.
There's a blocker for this requirement: https://bugzilla.yoctoproject.org/show_bug.cgi?id=15982
A patch was merged recently to mention this option in the docs: https://git.yoctoproject.org/yocto-docs/commit/?id=2cd7b5c83978819179b8a47e213b9812b97d5720 And --dependency was missing too so it's part the above patch. Docs-wise we're good, so I'm closing this bug, while https://bugzilla.yoctoproject.org/show_bug.cgi?id=15982 will track the blocker.