Bug 12883

Summary: Possible Dos with funky recipe
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Armin Kuster <akuster>
Component: coreAssignee: Armin Kuster <akuster>
Status: CLOSED FIXED QA Contact:
Severity: normal    
Priority: High CC: meta.mr.watcher, meta.watcher, richard.purdie
Version: 2.5.2   
Target Milestone: 2.5.2   
Hardware: x86   
OS: Multiple   
Whiteboard: backport 2.4.4 and 2.5.2
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Don't know

Description Armin Kuster 2018-08-12 22:47:46 UTC
I was working on a recipe and ended up hanging bitbake

recipe:
recipes-xfce/guvcview/guvcview_2.0.5.bb

LIC_FILES_CHKSUM = "files://COPYING;md5=12345"
LICENSE = "GPL-2.0"

SRC_URI = "http://archive.ubuntu.com/ubuntu/pool/universe/g/guvcview/guvcview_2.0.5+debian.orig.tar.xz"

SRC_URI[md5sum] = "0bcdf1057cab6949176eae644c675047"
SRC_URI[sha256sum] = "89efcbbbcbca5f725f4d8f9ea2fc5cbd51c6b22dc7a7cfa520a8285f641f4cc1"

Bitbake guvcview --> output on screen


WARNING: Unable to get checksum for guvcview SRC_URI entry vmlinuz.old: [Errno 13] Permission denied: '/vmlinuz.old'
WARNING: Unable to get checksum for guvcview SRC_URI entry vmlinuz: [Errno 13] Permission denied: '/vmlinuz'
WARNING: Unable to get checksum for guvcview SRC_URI entry bitmap: [Errno 13] Permission denied: '/sys/kernel/mm/page_idle/bitmap'
WARNING: Unable to get checksum for guvcview SRC_URI entry evm: [Errno 13] Permission denied: '/sys/kernel/security/evm'
WARNING: Unable to get checksum for guvcview SRC_URI entry policy: [Errno 13] Permission denied: '/sys/kernel/security/ima/policy'
WARNING: Unable to get checksum for guvcview SRC_URI entry violations: [Errno 13] Permission denied: '/sys/kernel/security/ima/violations'
WARNING: Unable to get checksum for guvcview SRC_URI entry runtime_measurements_count: [Errno 13] Permission denied: '/sys/kernel/security/ima/runtime_measurements_count'
WARNING: Unable to get checksum for guvcview SRC_URI entry ascii_runtime_measurements: [Errno 13] Permission denied: '/sys/kernel/security/ima/ascii_runtime_measurements'
WARNING: Unable to get checksum for guvcview SRC_URI entry binary_runtime_measurements: [Errno 13] Permission denied: '/sys/kernel/security/ima/binary_runtime_measurements'
WARNING: Unable to get checksum for guvcview SRC_URI entry .null: [Errno 13] Permission denied: '/sys/kernel/security/apparmor/.null'
Comment 1 Ross Burton 2018-08-12 22:50:27 UTC
Initial guess is that the LIC_FILE_CHECKSUM code is dumb and just does string splits when it should be doing a proper URL parse and resolve.
Comment 2 Armin Kuster 2018-08-12 22:52:01 UTC
(In reply to comment #1)
> Initial guess is that the LIC_FILE_CHECKSUM code is dumb and just does
> string splits when it should be doing a proper URL parse and resolve.

it used to not be an issue and fail telling me the proper checksum.
Comment 3 Armin Kuster 2018-08-12 22:52:09 UTC
Local.conf

PACKAGE_CLASSES ?= "package_ipk package_rpm"
SDKMACHINE ?= "x86_64"
EXTRA_IMAGE_FEATURES ?= "debug-tweaks tools-sdk"
USER_CLASSES ?= "buildstats image-mklibs image-prelink"
PATCHRESOLVE = "noop"
BB_DISKMON_DIRS = "\
    STOPTASKS,${TMPDIR},1G,100K \
    STOPTASKS,${DL_DIR},1G,100K \
    STOPTASKS,${SSTATE_DIR},1G,100K \
    STOPTASKS,/tmp,100M,100K \
    ABORT,${TMPDIR},100M,1K \
    ABORT,${DL_DIR},100M,1K \
    ABORT,${SSTATE_DIR},100M,1K \
    ABORT,/tmp,10M,1K"

PACKAGECONFIG_append_pn-qemu-native = " sdl"
PACKAGECONFIG_append_pn-nativesdk-qemu = " sdl"

CONF_VERSION = "1"
DL_DIR ?= "/home/akuster/oss/downloads"
SSTATE_DIR ?= "/home/build/master/sstate-cache"
SOURCE_MIRROR_URL ?= "http://downloads.yoctoproject.org/mirror/sources/"
SSTATE_MIRRORS ?= "file://.* http://sstate.yoctoproject.org/PATH"
SOURCE_MIRROR_FETCH = "1"
PACKAGE_CLASSES = "package_rpm"
SDKMACHINE ?= "x86_64"
IMAGE_FSTYPES_append = " tar.gz"
BB_TASK_NICE_LEVEL = "5"
BB_TASK_NICE_LEVEL_task-testimage = "0"
BB_TASK_IONICE_LEVEL = "2.7"
BB_TASK_IONICE_LEVEL_task-testimage = "2.1"
PREMIRRORS = ""
BB_GENERATE_MIRROR_TARBALLS = "1"
QEMU_USE_KVM = "True"
SDK_EXT_TYPE = "minimal"
SDK_INCLUDE_TOOLCHAIN = "1"
SANITY_TESTED_DISTROS = ""
Comment 4 Armin Kuster 2018-08-12 22:52:39 UTC
fails on MACHINE=odroid-xu4 and qemux86-64
Comment 5 Armin Kuster 2018-08-12 22:53:15 UTC
then this is a specially crafted recipe causing a DoS : )
Comment 6 Armin Kuster 2018-08-12 22:57:43 UTC
cleaning tmp and other dirs. will try with clean env-ish
Comment 7 Armin Kuster 2018-08-13 01:19:45 UTC
no sstate-cache, tmp* or cache and it fails still. 

core:
3d493838f0 (origin/master, origin/HEAD) binutils: enable x86_64-pep for producing EFI binaries on x86-64


bitbake:
ec037d3e (HEAD -> master, origin/master, origin/HEAD) runqueue: Allow the hash validate function to have an idea of build completion


/etc/os-release 
NAME="Ubuntu"
VERSION="18.04.1 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.1 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic
Comment 8 Armin Kuster 2018-08-13 14:45:43 UTC
I don't think its the LIC_FILES_CHKSUM.

I put the correct values in and I still got the same weird console message.

"S" is not correct.
Comment 9 Armin Kuster 2018-08-13 15:32:45 UTC
tried another host and got the same issue.
Comment 10 Armin Kuster 2018-08-13 15:36:33 UTC
layer with issues
https://github.com/akuster/meta-demos
Comment 11 Ross Burton 2018-08-13 16:30:39 UTC
The thousands of messages about SRC_URI are definitely LIC_FILES_CHKSUM: files://foo gets broken as method:files host:foo path:/ and then the checksum code goes to checksum / recursively.

This is because file: is special and not handled the same way as other URL handlers (and gets broken to method=file, host='', path=foo).  The fix I have here is to verify the method in the checksum lookup code.
Comment 12 Ross Burton 2018-08-13 22:51:22 UTC
I believe the root cause of this is the 'files:' in the LIC_FILES_CHKSUM.  Patches on the lists to handle that (and bail), and also catch a request to checksum / recursively (and bail), and improve checksumming (so checksumming / doesn't DoS the system).
Comment 13 Armin Kuster 2018-08-13 22:54:26 UTC
patches work for me.

thanks
Comment 15 Armin Kuster 2018-08-17 02:34:12 UTC
backport to sumo. in stable/sumo-nmut
Comment 16 Armin Kuster 2018-08-17 05:19:37 UTC
sumo being built.
Comment 17 Armin Kuster 2018-08-23 14:44:17 UTC
in rocko. having build issues but don't think the issue is related to this bug.
Comment 18 Armin Kuster 2018-08-30 14:58:28 UTC
Changes in rocko
Comment 19 Armin Kuster 2018-08-30 15:01:21 UTC
closing