FWIW, once I manually copied "pseudo" to the location it is expected to be in
(/data/master-zephyr/poky/build-multiconfig/tmp/sysroots/x86_64-linux/usr/bin,  /data/master-zephyr/poky/build-multiconfig/tmp/sysroots/x86_64-linux/usr/lib/pseudo/*),

everything worked fine and both SDKs  were built in their respective tmp-xxx/deploy/sdk folders. So the TMPDIR ***almost*** works for multiconfig.

Note the command line did not specify the "default" build.
Once the "default" build is specified, the error does not occur.

I suspect the fix is:

diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py
index da75021..638b9db 100644
--- a/bitbake/lib/bb/runqueue.py
+++ b/bitbake/lib/bb/runqueue.py
@@ -1003,8 +1003,9 @@ class RunQueue:
             magic = "decafbadbad"
         if fakeroot:
             magic = magic + "beef"
-            fakerootcmd = self.cfgData.getVar("FAKEROOTCMD")
-            fakerootenv = (self.cfgData.getVar("FAKEROOTBASEENV") or "").split()
+            mcdata = self.cooker.databuilder.mcdata[mc]
+            fakerootcmd = mcdata.getVar("FAKEROOTCMD")
+            fakerootenv = (mcdata.getVar("FAKEROOTBASEENV") or "").split()
             env = os.environ.copy()
             for key, value in (var.split('=') for var in fakerootenv):
                 env[key] = value

The suggested patch did not fix the problem. I ran:

$ bitbake multiconfig:qemux86-tmp:core-image-minimal multiconfig:qemuarm-tmp:core-image-minimal core-image-minimal


This resulted in:
ERROR: Failed to spawn fakeroot worker to run multiconfig:qemux86-tmp:/data/master-multi/poky/meta/recipes-kernel/linux-libc-headers/linux-libc-headers_4.8.bb:do_install: [Errno 2] No such file or directory: '/data/master-multi/poky/build-multi/tmp/sysroots/x86_64-linux/usr/bin/pseudo'


If I simply only build the "default", i.e.

$ bitbake core-image-minimal

I also observe an error:
ERROR: Failed to spawn fakeroot worker to run /data/master-multi/poky/meta/recipes-kernel/linux-libc-headers/linux-libc-headers_4.8.bb:do_install: [Errno 2] No such file or directory: '/data/master-multi/poky/build-multi-glob-1/tmp-qemuarm/sysroots/x86_64-linux/usr/bin/pseudo'

(Note that for some reason bitbake now tries to use "tmp-qemuarm" instead of "tmp")

I added some printouts,also displaying the previous ("default" unpatched) value. I see this:
fakerootcmd:       <...>/tmp-qemuarm/sysroots/x86_64-linux/usr/bin/pseudo
fakerootcmd(prev): <...>/tmp/sysroots/x86_64-linux/usr/bin/pseudo

So it seems the argument "mc" for _start_worker has incorrect value.

Sadly this is only a partial fix. I reproduced the issue and have a better fix in patch review which seems to work in my local testing.

I've been adding fixes for things but there is only so much we can do. I fixed the issues in bitbake itself but we're now hitting class issues in the metadata, primarily due to there only being one BuildStarted event which runs against the main sysroot. My current testing has a multiconfig file like:

MACHINE = "qemuarm"
ERR_REPORT_DIR := "${LOG_DIR}/error-report"
UNINATIVE_STAGING_DIR := "${STAGING_DIR}"
TMPDIR = "${TOPDIR}/arm

We need to expand ERR_REPORT_DIR, else if an error occurs, it can't find the base data created by the BuildStarted event, We need to set UNINATIVE_STAGING_DIR since otherwise we'd have to extract uninative into each TMPDI (again from BuildStarted).

I then hit problems with gdk-pixbuf not having run in each TMPDIR and the loaders were therefore not configured which broke my core-image-sato build in gtk+3 for each multiconfig.

This issue should get resolved when we move to recipe specific sysroots so I'm not planning to chase that issue any further.

Using separate TMPDIRs is likely something the user will need to do with caution.

Recipe specific sysroots will probably remove the main reason for separate TMPDIRs altogether, as separate TMPDIRs are needed mostly to avoid sysroot clashes. (presently the other reason would be to avoid races)

I merged my various fixes, this now really depends on recipe specific sysroots. I suspect STAGING_DIR may need to be redefined for builds with incompatible sysrooot contents instead of TMPDIR in that case but time will tell (since the sysroots components still have to be extracted somewhere).

Fixed by a series of patches by Richard, notably:

https://git.yoctoproject.org/cgit/cgit.cgi/poky/log/?id=7d18d05d451d4a69d827f7f78e7e17b69f950269

https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=5d620b7f90a6d43967b271bc165d23239b25d61c

multiconfig: TMPDIR  handled properly with success

Build Configuration:
BB_VERSION        = "1.33.3"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "universal"
TARGET_SYS        = "i586-poky-linux"
MACHINE           = "qemux86"
DISTRO            = "poky"
DISTRO_VERSION    = "2.2+snapshot-20170418"
TUNE_FEATURES     = "m32 i586"
TARGET_FPU        = ""
meta              
meta-poky         
meta-yocto-bsp    = "HEAD:eff56e4f0d59b1d965a68e4f009b7f07717b7edd"
 
STEP:
I have this setup:

conf/local.conf: 
    BBMULTICONFIG = "qemux86 qemuarm"

conf/multiconfig/qemux86.conf:
    MACHINE="qemux86"
    TMPDIR="${TOPDIR}/tmp-x86"

conf/multiconfig/qemuarm.conf:
    MACHINE="qemuarm"
    TMPDIR="${TOPDIR}/tmp-arm"

I run this command:
$ bitbake multiconfig:qemuarm:meta-toolchain multiconfig:qemux86:meta-toolchain

EXPECTED OUTCOME:
multiconfig: TMPDIR  handled properly with success

ACTUAL OUTCOME:
multiconfig: TMPDIR  handled properly with success

NOTES:
 
oscar@oscar-ThinkPad-T540p:~/qemu/build$ bitbake multiconfig:qemuarm:meta-toolchain multiconfig:qemux86:meta-toolchain
Parsing recipes: 100% |#########################################| Time: 0:00:50
Parsing of 830 .bb files complete (0 cached, 830 parsed). 3894 targets, 144 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies
NOTE: Resolving any missing task queue dependencies
NOTE: Resolving any missing task queue dependencies
Initialising tasks: 100% |######################################| Time: 0:00:07
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 5980 tasks of which 4155 didn't need to be rerun and all succeeded.