13746 2020-01-22 16:32:32 +0000 Unable to create RPM package from repo that contains filenames with "[" and "]" characters 2022-08-22 13:45:00 +0000 1 1 11 Runtime Package Management Issues RPM 3.0.1 x86 Multiple RESOLVED FIXED Medium+ normal 4.1 M4 1 bartlomiej.burdukiewicz pavel randy.macleod --- --- No (bug/feature does not impact docs) oldest_to_newest 86083 0 4613 bartlomiej.burdukiewicz 2020-01-22 16:32:32 +0000 Created attachment 4613 Recipe affected by bug. Preconditions/Environment ------------------------- Gentoo Build Configuration: BB_VERSION = "1.44.0" BUILD_SYS = "x86_64-linux" NATIVELSBSTRING = "universal" TARGET_SYS = "arm-poky-linux-gnueabi" MACHINE = "raspberrypi4" DISTRO = "retro" DISTRO_VERSION = "3.0.1" TUNE_FEATURES = "arm vfp cortexa7 neon vfpv4 thumb callconvention-hard" TARGET_FPU = "hard" meta-intel = "zeus:b04e1edb9300a57e200a187a3255f67b50519202" meta-multimedia meta-networking meta-oe meta-python = "zeus:e855ecc6d35677e79780adc57b2552213c995731" meta-raspberrypi = "zeus:85b75588fdedf72242870628f7c22f034d308740" meta-retro = "zeus:feb5594ae9695a3a4189a67fe4c8f6e3a39e80e7" meta meta-poky meta-yocto-bsp = "zeus:cf92a2d567260b91a259652bad0ecd790750f710" Triggering Action/Cause ----------------------- run the following command: bitbake retroarch-database-simple Expectation ----------- Proper RPM package, ipk package working fine for this recipe Actual Result ------------- Full log: https://pastebin.com/fp2Tebt4 Partial log: ERROR: retroarch-database-simple-1.0-r100 do_package_write_rpm: Error executing a python function in exec_python_func() autogenerated: The stack trace of python calls that resulted in this exception/failure was: File: 'exec_python_func() autogenerated', lineno: 2, function: <module> 0001: *** 0002:do_package_rpm(d) 0003: File: '/home/dev/embedded/retro-build-enviroment/poky/meta/classes/package_rpm.bbclass', lineno: 710, function: do_package_rpm 0706: 0707: # Build the rpm package! 0708: d.setVar('BUILDSPEC', cmd + "\n" + cleanupcmd + "\n") 0709: d.setVarFlag('BUILDSPEC', 'func', '1') *** 0710: bb.build.exec_func('BUILDSPEC', d) 0711: 0712: if d.getVar('RPM_SIGN_PACKAGES') == '1': 0713: bb.build.exec_func("sign_rpm", d) 0714:} File: '/home/dev/embedded/retro-build-enviroment/poky/bitbake/lib/bb/build.py', lineno: 250, function: exec_func 0246: with bb.utils.fileslocked(lockfiles): 0247: if ispython: 0248: exec_func_python(func, d, runfile, cwd=adir) 0249: else: *** 0250: exec_func_shell(func, d, runfile, cwd=adir) 0251: 0252: try: 0253: curcwd = os.getcwd() 0254: except: File: '/home/dev/embedded/retro-build-enviroment/poky/bitbake/lib/bb/build.py', lineno: 451, function: exec_func_shell 0447: with open(fifopath, 'r+b', buffering=0) as fifo: 0448: try: 0449: bb.debug(2, "Executing shell function %s" % func) 0450: with open(os.devnull, 'r+') as stdin, logfile: *** 0451: bb.process.run(cmd, shell=False, stdin=stdin, log=logfile, extrafiles=[(fifo,readfifo)]) 0452: finally: 0453: os.unlink(fifopath) 0454: 0455: bb.debug(2, "Shell function %s finished" % func) File: '/home/dev/embedded/retro-build-enviroment/poky/bitbake/lib/bb/process.py', lineno: 182, function: run 0178: if not stderr is None: 0179: stderr = stderr.decode("utf-8") 0180: 0181: if pipe.returncode != 0: *** 0182: raise ExecutionError(cmd, pipe.returncode, stdout, stderr) 0183: return stdout, stderr Exception: bb.process.ExecutionError: Execution of '/home/dev/embedded/retro-build-enviroment/build/tmp/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi/retroarch-database-simple/1.0-r100/temp/run.BUILDSPEC.1898842' failed with exit code 1: Building target platforms: cortexa7t2hf_neon_vfpv4-poky-linux Building for target cortexa7t2hf_neon_vfpv4-poky-linux Processing files: retroarch-database-simple-dbg-1.0-r100.cortexa7t2hf_neon_vfpv4 Processing files: retroarch-database-simple-cheats-1.0-r100.cortexa7t2hf_neon_vfpv4 error: File not found: /home/dev/embedded/retro-build-enviroment/build/tmp/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi/retroarch-database-simple/1.0-r100/package/usr/share/retroarch/database/cheats/Nintendo - Nintendo DS/Transformers Autobots (U) ?v1.1?.cht error: File not found: /home/dev/embedded/retro-build-enviroment/build/tmp/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi/retroarch-database-simple/1.0-r100/package/usr/share/retroarch/database/cheats/Nintendo - Nintendo DS/Soma Bringer (J) ?English by Darth Nemesis?.cht error: File not found: /home/dev/embedded/retro-build-enviroment/build/tmp/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi/retroarch-database-simple/1.0-r100/package/usr/share/retroarch/database/cheats/Nintendo - Nintendo DS/Dragon Quest VI Wandler zwischen den Welten (U) ?German? ?Hacked by DSRP?.cht error: File not found: /home/dev/embedded/retro-build-enviroment/build/tmp/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi/retroarch-database-simple/1.0-r100/package/usr/share/retroarch/database/cheats/Nintendo - Nintendo DS/SNK Vs Capcom Card Fighters DS (U) ?V1?.cht Reproducibility --------------- Always 86084 1 bartlomiej.burdukiewicz 2020-01-22 16:35:27 +0000 I tried with tar.gz with content of that repo but problem remains. 86085 2 bartlomiej.burdukiewicz 2020-01-22 16:39:16 +0000 poky/meta/classes/package_rpm.bbclass Contains some logic that replacing "[" and "]" but I'm not sure what to do with that, suspected lines: path = path.replace("[", "?") path = path.replace("]", "?") ... dir = dir.replace("[", "?") dir = dir.replace("]", "?") 86087 3 4614 bartlomiej.burdukiewicz 2020-01-22 17:04:51 +0000 Created attachment 4614 Recipe affected by bug. (without additional stuff) 86277 4 matthew.zeng 2020-02-03 16:08:48 +0000 I cannot reproduce this log with the following error messages: ``` Initialising tasks: 100% |#########################################################################################################################################################################| Time: 0:00:02 Sstate summary: Wanted 5 Found 0 Missed 5 Current 128 (0% match, 96% complete) NOTE: Executing Tasks NOTE: Setscene tasks completed NOTE: Task .../poky/meta/recipes-devtools/gcc/gcc-runtime_9.2.bb:do_package_write_rpm unihash changed to a94da629ef03fa3b4e66a026c821fdf1c795c433824567f370a8786860a8e619 ERROR: retroarch-database-simple-1.0-r0 do_package: QA Issue: retroarch-database-simple: Files/directories were installed but not shipped in any package: /usr /usr/share /usr/share/test /usr/share/test/configure /usr/share/test/COPYING /usr/share/test/Makefile /usr/share/test/README.md /usr/share/test/cht /usr/share/test/cursors /usr/share/test/dat /usr/share/test/metadat ...... /usr/share/test/scripts/Gameboy/Xploder/Batman Beyond Return Of The Joker.cht /usr/share/test/scripts/Gameboy/Xploder/Battle Ships.cht /usr/share/test/scripts/Gameboy/Xploder/Battle Tanx.cht /usr/share/test/scripts/Gameboy/Xploder/Bill And Teds Excellent GameBoy Adventure.cht Please set FILES such that these items are packaged. Alternatively if they are unneeded, avoid installing them or delete them within do_install. retroarch-database-simple: 16464 installed and not shipped files. [installed-vs-shipped] ERROR: retroarch-database-simple-1.0-r0 do_package: Fatal QA errors found, failing task. ERROR: Logfile of failure stored in: .../poky/build/tmp/work/core2-64-poky-linux/retroarch-database-simple/1.0-r0/temp/log.do_package.34546 ERROR: Task (.../retroarch-database-simple/retroarch-database-simple.bb:do_package) failed with exit code '1' NOTE: Tasks Summary: Attempted 529 tasks of which 521 didn't need to be rerun and 1 failed. Summary: 1 task failed: .../retroarch-database-simple/retroarch-database-simple.bb:do_package Summary: There were 2 ERROR messages shown, returning a non-zero exit code. ``` However, by inspecting poky/meta/classes/package_rpm.bbclass, this change was initially introduced in d008867e45b276e69da43adaed93a668800cccf3, the reason for it is [ and ] characters will be treated as wildcard characters by rpm. By changing it to question mark ?, it is another wildcard that matches any single character. Did you verify that the files can be found under /home/dev/embedded/retro-build-enviroment/build/tmp/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi/retroarch-database-simple/1.0-r100/package/usr/share/retroarch/database/cheats/ 86291 5 bartlomiej.burdukiewicz 2020-02-04 03:13:20 +0000 > Please set FILES such that these items are packaged. Alternatively if they are unneeded, avoid installing them or delete them within do_install. Sadly I introduced a bug in simplified recipe, I'll upload fixed recipe. (retroarch-database-simple-v2.bb) > Did you verify that the files can be found under /home/dev/embedded/retro-build-enviroment/build/tmp/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi/retroarch-database-simple/1.0-r100/package/usr/share/retroarch/database/cheats/ Yes I checked multiple times, files was there. But those logs are from different version of retroarch-database-simple recipe (probably before I changed installation path and introduced a installed-vs-shipped). ----- I must apologize for inconvenience. From fresh start with retroarch-database-simple-v2.bb #~ bitbake retroarch-database-simple-v2 Full output: https://pastebin.com/USzGFJFa #~ ls -lah "/home/dev/embedded/retro-build-enviroment/build/tmp/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi/retroarch-database-simple-v2/1.0-r0/package/usr/share/test/cht/Nintendo - Nintendo DS/" Full output: https://pastebin.com/d2k3jJ1W Quick look into results: line 19 contains '42 All-Time Classics (E) [v1.1].cht' error: File not found: /home/dev/embedded/retro-build-enviroment/build/tmp/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi/retroarch-database-simple-v2/1.0-r0/package/usr/share/test/cht/Nintendo - Nintendo DS/42 All-Time Classics (E) ?v1.1?.cht And by checking path "/home/dev/embedded/retro-build-enviroment/build/tmp/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi/retroarch-database-simple-v2/1.0-r0/package/usr/share/test/cht/Nintendo - Nintendo DS/42 All-Time Classics (E) [v1.1].cht" by ls, this file exists. 86292 6 4629 bartlomiej.burdukiewicz 2020-02-04 03:14:49 +0000 Created attachment 4629 retroarch-database-simple-v2.bb 86293 7 bartlomiej.burdukiewicz 2020-02-04 03:30:10 +0000 Maybe this will be helpful somewhat: This is build environment that I'm using: https://github.com/dev-0x7C6/retro-build-environment Original affected recipe: https://github.com/dev-0x7C6/meta-retro/blob/zeus/recipes-retroarch/retroarch-database/retroarch-database.bb 86484 8 matthew.zeng 2020-02-19 17:01:41 +0000 > path = path.replace("[", "?") > path = path.replace("]", "?") It seems that these changes were introduced in 2017 assuming rpm build treats '[' and ']' as wildcard characters, so they're replaced with wildcard '?' that matches any character. However from this error, rpm in the yocto clearly did not treat '?' as wildcard, in fact it didn't treat any character as wildcards. Commenting out the relevent lines resolves the error. We now see another error, rpm doesn't like file names that has spaces in it. error: File not found: .../retroarch-database/1.0-r100/package/usr/share/retroarch/database/cheats/Nintendo error: Path is outside buildroot: - error: Path is outside buildroot: Family error: Path is outside buildroot: Computer error: Path is outside buildroot: Disk error: Path is outside buildroot: System/Yume error: Path is outside buildroot: Koujou error: Path is outside buildroot: Doki error: Path is outside buildroot: Doki error: Path is outside buildroot: Panic error: Path is outside buildroot: (1987)(Nintendo)[tr error: Path is outside buildroot: en].cht 86632 9 bartlomiej.burdukiewicz 2020-03-01 01:00:51 +0000 I was also tried to remove this lines and I end-up with same problem with those spaces. I'm not sure how to fix this. 93856 10 pavel 2022-08-18 17:23:05 +0000 Reproduced with yocto master: | DEBUG: PKGWRITEDIR: /mnt/builds/yocto/build/build/tmp/work/core2-64-poky-linux/testrpm/0.0.1-r0/deploy-rpms/core2_64 | DEBUG: Executing shell function BUILDSPEC | Building target platforms: core2_64-poky-linux | Building for target core2_64-poky-linux | Processing files: testrpm-0.0.1-r0.core2_64 | error: File not found: /mnt/builds/yocto/build/build/tmp/work/core2-64-poky-linux/testrpm/0.0.1-r0/package/42 All-Time Classics (E) ?v1.1?.cht | File not found: /mnt/builds/yocto/build/build/tmp/work/core2-64-poky-linux/testrpm/0.0.1-r0/package/42 All-Time Classics (E) ?v1.1?.cht | 93866 11 pavel 2022-08-19 09:22:34 +0000 https://lists.openembedded.org/g/openembedded-core/message/169616 https://lists.openembedded.org/g/openembedded-core/message/169617 93871 12 pavel 2022-08-22 13:45:00 +0000 This bug should be fixed in https://git.openembedded.org/openembedded-core/commit/?id=9916212b4810280108107d94cd2c0cd3e07ae6ed Feel free to reopen this bug report if the issue still exists. 4613 2020-01-22 16:32:32 +0000 2020-01-22 16:32:32 +0000 Recipe affected by bug. retroarch-database-simple.bb application/octet-stream 1507 bartlomiej.burdukiewicz U1VNTUFSWSA9ICJSZXRyb0FyY2ggRGF0YWJhc2UgY29udGFpbmluZyBjaGVhdGNvZGUgZmlsZXMs IGNvbnRlbnQgZGF0YSBmaWxlcywgZXRjLiIKREVTQ1JJUFRJT04gPSAiUmV0cm9BcmNoIGluY29w b3JhdGVzIGEgUk9NIHNjYW5uaW5nIHN5c3RlbSB0byBhdXRvbWF0aWNhbGx5IFwKcHJvZHVjZSBw bGF5bGlzdHMuIEVhY2ggUk9NIHRoYXQgaXMgc2Nhbm5lZCBieSB0aGUgcGxheWxpc3QgZ2VuZXJh dG9yIFwKaXMgY2hlY2tlZCBhZ2FpbnN0IGEgZGF0YWJhc2Ugb2YgUk9NcyB0aGF0IGFyZSBrbm93 biB0byBiZSBnb29kIGNvcGllcy4iCgpIT01FUEFHRSA9ICJodHRwczovL3d3dy5yZXRyb2FyY2gu Y29tLyIKQlVHVFJBQ0tFUiA9ICJodHRwczovL2dpdGh1Yi5jb20vbGlicmV0cm8vbGlicmV0cm8t ZGF0YWJhc2UvaXNzdWVzIgoKTElDRU5TRSA9ICJNSVQiCkxJQ19GSUxFU19DSEtTVU0gPSAiZmls ZTovL0NPUFlJTkc7bWQ1PTQ1OTI3N2Q4MDQ2MWMyOTA4YjRjZjE0OTQ5ZjhkY2Q1IgoKUFIgPSAi cjEwMCIKClMgPSAiJHtXT1JLRElSfS9naXQiClNSQ19VUkkgPSAiZ2l0c206Ly9naXRodWIuY29t L2xpYnJldHJvL2xpYnJldHJvLWRhdGFiYXNlLmdpdDtwcm90b2NvbD1odHRwcyIKU1JDUkVWID0g IiR7QVVUT1JFVn0iCgppbmhlcml0IHJldHJvYXJjaC1wYXRocwoKUEFDS0FHRVMgPSsgIiBcCiAg JHtQTn0tY2hlYXRzIFwKICAke1BOfS1jdXJzb3JzIFwKICAke1BOfS10aXRsZXMgXAoiCgpGSUxF U18ke1BOfS1jaGVhdHMgPSAiJHtSRVRST0FSQ0hfREFUQUJBU0VfQ0hFQVRTX0RJUn0iCkZJTEVT XyR7UE59LWN1cnNvcnMgPSAiJHtSRVRST0FSQ0hfREFUQUJBU0VfQ1VSU09SU19ESVJ9IgpGSUxF U18ke1BOfS10aXRsZXMgPSAiJHtSRVRST0FSQ0hfREFUQUJBU0VfVElUTEVTX0RJUn0iCgpGSUxF U18ke1BOfSArPSAiJHtSRVRST0FSQ0hfREFUQUJBU0VfRElSfSIKCkRFUEVORFMgKz0gInJzeW5j LW5hdGl2ZSIKCmRvX2NvbmZpZ3VyZVtub2V4ZWNdID0gIjEiCmRvX2NvbXBpbGVbbm9leGVjXSA9 ICIxIgoKZG9faW5zdGFsbCgpIHsKICBpbnN0YWxsIC1kICR7RH0ke1JFVFJPQVJDSF9EQVRBQkFT RV9ESVJ9CiAgaW5zdGFsbCAtZCAke0R9JHtSRVRST0FSQ0hfREFUQUJBU0VfQ0hFQVRTX0RJUn0K ICBpbnN0YWxsIC1kICR7RH0ke1JFVFJPQVJDSF9EQVRBQkFTRV9USVRMRVNfRElSfQogIGluc3Rh bGwgLWQgJHtEfSR7UkVUUk9BUkNIX0RBVEFCQVNFX0NVUlNPUlNfRElSfQoKIyBGSVhNRTogZmls ZXMgd2l0aCBjaGFyYWN0ZXIgWyBvciBdIGZhaWxpbmcgYXQgcnBtIHBhY2thZ2Ugc3RhZ2UKICBy c3luYyAtcmxwdEQgJHtTfS9jaHQvKiAke0R9JHtSRVRST0FSQ0hfREFUQUJBU0VfQ0hFQVRTX0RJ Un0vCiAgcnN5bmMgLXJscHREICR7U30vcmRiLyogJHtEfSR7UkVUUk9BUkNIX0RBVEFCQVNFX1RJ VExFU19ESVJ9LwogIHJzeW5jIC1ybHB0RCAke1N9L2N1cnNvcnMvKiAke0R9JHtSRVRST0FSQ0hf REFUQUJBU0VfQ1VSU09SU19ESVJ9Lwp9Cg== 4614 2020-01-22 17:04:51 +0000 2020-01-22 17:04:51 +0000 Recipe affected by bug. (without additional stuff) retroarch-database-simple.bb application/octet-stream 754 bartlomiej.burdukiewicz U1VNTUFSWSA9ICJSZXRyb0FyY2ggRGF0YWJhc2UgY29udGFpbmluZyBjaGVhdGNvZGUgZmlsZXMs IGNvbnRlbnQgZGF0YSBmaWxlcywgZXRjLiIKREVTQ1JJUFRJT04gPSAiUmV0cm9BcmNoIGluY29w b3JhdGVzIGEgUk9NIHNjYW5uaW5nIHN5c3RlbSB0byBhdXRvbWF0aWNhbGx5IFwKcHJvZHVjZSBw bGF5bGlzdHMuIEVhY2ggUk9NIHRoYXQgaXMgc2Nhbm5lZCBieSB0aGUgcGxheWxpc3QgZ2VuZXJh dG9yIFwKaXMgY2hlY2tlZCBhZ2FpbnN0IGEgZGF0YWJhc2Ugb2YgUk9NcyB0aGF0IGFyZSBrbm93 biB0byBiZSBnb29kIGNvcGllcy4iCgpIT01FUEFHRSA9ICJodHRwczovL3d3dy5yZXRyb2FyY2gu Y29tLyIKQlVHVFJBQ0tFUiA9ICJodHRwczovL2dpdGh1Yi5jb20vbGlicmV0cm8vbGlicmV0cm8t ZGF0YWJhc2UvaXNzdWVzIgoKTElDRU5TRSA9ICJNSVQiCkxJQ19GSUxFU19DSEtTVU0gPSAiZmls ZTovL0NPUFlJTkc7bWQ1PTQ1OTI3N2Q4MDQ2MWMyOTA4YjRjZjE0OTQ5ZjhkY2Q1IgoKUyA9ICIk e1dPUktESVJ9L2dpdCIKU1JDX1VSSSA9ICJnaXRzbTovL2dpdGh1Yi5jb20vbGlicmV0cm8vbGli cmV0cm8tZGF0YWJhc2UuZ2l0O3Byb3RvY29sPWh0dHBzIgpTUkNSRVYgPSAiJHtBVVRPUkVWfSIK CkZJTEVTXyR7UE59ICs9ICIke0R9JHtkYXRhZGlyfS90ZXN0IgoKZG9faW5zdGFsbCgpIHsKICBp bnN0YWxsIC1kICR7RH0ke2RhdGFkaXJ9L3Rlc3QvCiAgY3AgLXJmICR7U30vKiAke0R9JHtkYXRh ZGlyfS90ZXN0Lwp9Cg== 4629 2020-02-04 03:14:49 +0000 2020-02-04 03:14:49 +0000 retroarch-database-simple-v2.bb retroarch-database-simple-v2.bb application/octet-stream 751 bartlomiej.burdukiewicz U1VNTUFSWSA9ICJSZXRyb0FyY2ggRGF0YWJhc2UgY29udGFpbmluZyBjaGVhdGNvZGUgZmlsZXMs IGNvbnRlbnQgZGF0YSBmaWxlcywgZXRjLiIKREVTQ1JJUFRJT04gPSAiUmV0cm9BcmNoIGluY29w b3JhdGVzIGEgUk9NIHNjYW5uaW5nIHN5c3RlbSB0byBhdXRvbWF0aWNhbGx5IFwKcHJvZHVjZSBw bGF5bGlzdHMuIEVhY2ggUk9NIHRoYXQgaXMgc2Nhbm5lZCBieSB0aGUgcGxheWxpc3QgZ2VuZXJh dG9yIFwKaXMgY2hlY2tlZCBhZ2FpbnN0IGEgZGF0YWJhc2Ugb2YgUk9NcyB0aGF0IGFyZSBrbm93 biB0byBiZSBnb29kIGNvcGllcy4iCgpIT01FUEFHRSA9ICJodHRwczovL3d3dy5yZXRyb2FyY2gu Y29tLyIKQlVHVFJBQ0tFUiA9ICJodHRwczovL2dpdGh1Yi5jb20vbGlicmV0cm8vbGlicmV0cm8t ZGF0YWJhc2UvaXNzdWVzIgoKTElDRU5TRSA9ICJNSVQiCkxJQ19GSUxFU19DSEtTVU0gPSAiZmls ZTovL0NPUFlJTkc7bWQ1PTQ1OTI3N2Q4MDQ2MWMyOTA4YjRjZjE0OTQ5ZjhkY2Q1IgoKUyA9ICIk e1dPUktESVJ9L2dpdCIKU1JDX1VSSSA9ICJnaXRzbTovL2dpdGh1Yi5jb20vbGlicmV0cm8vbGli cmV0cm8tZGF0YWJhc2UuZ2l0O3Byb3RvY29sPWh0dHBzIgpTUkNSRVYgPSAiJHtBVVRPUkVWfSIK CkZJTEVTXyR7UE59ICs9ICIke2RhdGFkaXJ9L3Rlc3QvIgoKZG9faW5zdGFsbCgpIHsKICBpbnN0 YWxsIC1kICR7RH0ke2RhdGFkaXJ9L3Rlc3QvCiAgY3AgLXJmICR7U30vKiAke0R9JHtkYXRhZGly fS90ZXN0Lwp9Cg==