Here's the error I get. clean and cleansstate didn't help.
ARCHIVER_MODE[src] = "original" works.
The archiver-work/ folder is almost empty here.

NOTE: gcc-source-5.3.0-5.3.0-r0 do_unpack_and_patch: Unpacking /stuff/sources/poky/meta/recipes-devtools/gcc/gcc-5.3/CVE-2016-4490.patch to /stuff/project/tmp/work-shared/gcc-5.3.0-r0/archiver-work/
NOTE: gcc-source-5.3.0-5.3.0-r0 do_unpack_and_patch: Applying patch '0001-gcc-4.3.1-ARCH_FLAGS_FOR_TARGET.patch' (../sources/poky/meta/recipes-devtools/gcc/gcc-5.3/0001-gcc-4.3.1-ARCH_FLAGS_FOR_TARGET.patch)
ERROR: gcc-source-5.3.0-5.3.0-r0 do_unpack_and_patch: Command Error: 'quilt --quiltrc /stuff/project/tmp/sysroots/x86_64-linux/etc/quiltrc push' exited with 1  Output:
Applying patch 0001-gcc-4.3.1-ARCH_FLAGS_FOR_TARGET.patch
can't find file to patch at input line 18
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|From 6029bb338305a5d1403ee23427ed8d58eae1ff53 Mon Sep 17 00:00:00 2001
|From: Khem Raj <raj.khem@gmail.com>
|Date: Fri, 29 Mar 2013 08:37:11 +0400
|Subject: [PATCH 01/46] gcc-4.3.1: ARCH_FLAGS_FOR_TARGET
|
|Signed-off-by: Khem Raj <raj.khem@gmail.com>
|
|Upstream-Status: Inappropriate [embedded specific]
|---
| configure    | 2 +-
| configure.ac | 2 +-
| 2 files changed, 2 insertions(+), 2 deletions(-)
|
|diff --git a/configure b/configure
|index a3f66ba..1cba3a9 100755
|--- a/configure
|+++ b/configure
--------------------------
No file to patch.  Skipping patch.
1 out of 1 hunk ignored
can't find file to patch at input line 31
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|diff --git a/configure.ac b/configure.ac
|index 987dfab..d3adb95 100644
|--- a/configure.ac
|+++ b/configure.ac
--------------------------
No file to patch.  Skipping patch.
1 out of 1 hunk ignored
Patch 0001-gcc-4.3.1-ARCH_FLAGS_FOR_TARGET.patch does not apply (enforce with -f)
ERROR: gcc-source-5.3.0-5.3.0-r0 do_unpack_and_patch: Function failed: patch_do_patch
ERROR: Logfile of failure stored in: /stuff/project/tmp/work-shared/gcc-5.3.0-r0/temp/log.do_unpack_and_patch.5062
ERROR: Task 3250 (/stuff/sources/poky/meta/recipes-devtools/gcc/gcc-source_5.3.bb, do_unpack_and_patch) failed with exit code '1'