| Summary: | Unable to create RPM package from repo that contains filenames with "[" and "]" characters | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | [Runtime] Package Management Issues | Reporter: | Bartłomiej Burdukiewicz <bartlomiej.burdukiewicz> | ||||||||
| Component: | RPM | Assignee: | Pavel Zhukov <pavel> | ||||||||
| Status: | RESOLVED FIXED | QA Contact: | |||||||||
| Severity: | normal | ||||||||||
| Priority: | Medium+ | CC: | randy.macleod | ||||||||
| Version: | 3.0.1 | ||||||||||
| Target Milestone: | 4.1 M4 | ||||||||||
| Hardware: | x86 | ||||||||||
| OS: | Multiple | ||||||||||
| Whiteboard: | |||||||||||
| OS type for building Yocto: | --- | Type of Regression: | --- | ||||||||
| Verified: | Documentation change: | No (bug/feature does not impact docs) | |||||||||
| Attachments: |
|
||||||||||
|
Description
Bartłomiej Burdukiewicz
2020-01-22 16:32:32 UTC
I tried with tar.gz with content of that repo but problem remains. poky/meta/classes/package_rpm.bbclass
Contains some logic that replacing "[" and "]" but I'm not sure what to do with that, suspected lines:
path = path.replace("[", "?")
path = path.replace("]", "?")
...
dir = dir.replace("[", "?")
dir = dir.replace("]", "?")
Created attachment 4614 [details]
Recipe affected by bug. (without additional stuff)
I cannot reproduce this log with the following error messages:
```
Initialising tasks: 100% |#########################################################################################################################################################################| Time: 0:00:02
Sstate summary: Wanted 5 Found 0 Missed 5 Current 128 (0% match, 96% complete)
NOTE: Executing Tasks
NOTE: Setscene tasks completed
NOTE: Task .../poky/meta/recipes-devtools/gcc/gcc-runtime_9.2.bb:do_package_write_rpm unihash changed to a94da629ef03fa3b4e66a026c821fdf1c795c433824567f370a8786860a8e619
ERROR: retroarch-database-simple-1.0-r0 do_package: QA Issue: retroarch-database-simple: Files/directories were installed but not shipped in any package:
/usr
/usr/share
/usr/share/test
/usr/share/test/configure
/usr/share/test/COPYING
/usr/share/test/Makefile
/usr/share/test/README.md
/usr/share/test/cht
/usr/share/test/cursors
/usr/share/test/dat
/usr/share/test/metadat
......
/usr/share/test/scripts/Gameboy/Xploder/Batman Beyond Return Of The Joker.cht
/usr/share/test/scripts/Gameboy/Xploder/Battle Ships.cht
/usr/share/test/scripts/Gameboy/Xploder/Battle Tanx.cht
/usr/share/test/scripts/Gameboy/Xploder/Bill And Teds Excellent GameBoy Adventure.cht
Please set FILES such that these items are packaged. Alternatively if they are unneeded, avoid installing them or delete them within do_install.
retroarch-database-simple: 16464 installed and not shipped files. [installed-vs-shipped]
ERROR: retroarch-database-simple-1.0-r0 do_package: Fatal QA errors found, failing task.
ERROR: Logfile of failure stored in: .../poky/build/tmp/work/core2-64-poky-linux/retroarch-database-simple/1.0-r0/temp/log.do_package.34546
ERROR: Task (.../retroarch-database-simple/retroarch-database-simple.bb:do_package) failed with exit code '1'
NOTE: Tasks Summary: Attempted 529 tasks of which 521 didn't need to be rerun and 1 failed.
Summary: 1 task failed:
.../retroarch-database-simple/retroarch-database-simple.bb:do_package
Summary: There were 2 ERROR messages shown, returning a non-zero exit code.
```
However, by inspecting poky/meta/classes/package_rpm.bbclass, this change was initially introduced in d008867e45b276e69da43adaed93a668800cccf3, the reason for it is [ and ] characters will be treated as wildcard characters by rpm. By changing it to question mark ?, it is another wildcard that matches any single character. Did you verify that the files can be found under /home/dev/embedded/retro-build-enviroment/build/tmp/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi/retroarch-database-simple/1.0-r100/package/usr/share/retroarch/database/cheats/
> Please set FILES such that these items are packaged. Alternatively if they are unneeded, avoid installing them or delete them within do_install. Sadly I introduced a bug in simplified recipe, I'll upload fixed recipe. (retroarch-database-simple-v2.bb) > Did you verify that the files can be found under /home/dev/embedded/retro-build-enviroment/build/tmp/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi/retroarch-database-simple/1.0-r100/package/usr/share/retroarch/database/cheats/ Yes I checked multiple times, files was there. But those logs are from different version of retroarch-database-simple recipe (probably before I changed installation path and introduced a installed-vs-shipped). ----- I must apologize for inconvenience. From fresh start with retroarch-database-simple-v2.bb #~ bitbake retroarch-database-simple-v2 Full output: https://pastebin.com/USzGFJFa #~ ls -lah "/home/dev/embedded/retro-build-enviroment/build/tmp/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi/retroarch-database-simple-v2/1.0-r0/package/usr/share/test/cht/Nintendo - Nintendo DS/" Full output: https://pastebin.com/d2k3jJ1W Quick look into results: line 19 contains '42 All-Time Classics (E) [v1.1].cht' error: File not found: /home/dev/embedded/retro-build-enviroment/build/tmp/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi/retroarch-database-simple-v2/1.0-r0/package/usr/share/test/cht/Nintendo - Nintendo DS/42 All-Time Classics (E) ?v1.1?.cht And by checking path "/home/dev/embedded/retro-build-enviroment/build/tmp/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi/retroarch-database-simple-v2/1.0-r0/package/usr/share/test/cht/Nintendo - Nintendo DS/42 All-Time Classics (E) [v1.1].cht" by ls, this file exists. Created attachment 4629 [details]
retroarch-database-simple-v2.bb
Maybe this will be helpful somewhat: This is build environment that I'm using: https://github.com/dev-0x7C6/retro-build-environment Original affected recipe: https://github.com/dev-0x7C6/meta-retro/blob/zeus/recipes-retroarch/retroarch-database/retroarch-database.bb > path = path.replace("[", "?")
> path = path.replace("]", "?")
It seems that these changes were introduced in 2017 assuming rpm build treats '[' and ']' as wildcard characters, so they're replaced with wildcard '?' that matches any character. However from this error, rpm in the yocto clearly did not treat '?' as wildcard, in fact it didn't treat any character as wildcards. Commenting out the relevent lines resolves the error.
We now see another error, rpm doesn't like file names that has spaces in it.
error: File not found: .../retroarch-database/1.0-r100/package/usr/share/retroarch/database/cheats/Nintendo
error: Path is outside buildroot: -
error: Path is outside buildroot: Family
error: Path is outside buildroot: Computer
error: Path is outside buildroot: Disk
error: Path is outside buildroot: System/Yume
error: Path is outside buildroot: Koujou
error: Path is outside buildroot: Doki
error: Path is outside buildroot: Doki
error: Path is outside buildroot: Panic
error: Path is outside buildroot: (1987)(Nintendo)[tr
error: Path is outside buildroot: en].cht
I was also tried to remove this lines and I end-up with same problem with those spaces. I'm not sure how to fix this. Reproduced with yocto master: | DEBUG: PKGWRITEDIR: /mnt/builds/yocto/build/build/tmp/work/core2-64-poky-linux/testrpm/0.0.1-r0/deploy-rpms/core2_64 | DEBUG: Executing shell function BUILDSPEC | Building target platforms: core2_64-poky-linux | Building for target core2_64-poky-linux | Processing files: testrpm-0.0.1-r0.core2_64 | error: File not found: /mnt/builds/yocto/build/build/tmp/work/core2-64-poky-linux/testrpm/0.0.1-r0/package/42 All-Time Classics (E) ?v1.1?.cht | File not found: /mnt/builds/yocto/build/build/tmp/work/core2-64-poky-linux/testrpm/0.0.1-r0/package/42 All-Time Classics (E) ?v1.1?.cht | https://lists.openembedded.org/g/openembedded-core/message/169616 https://lists.openembedded.org/g/openembedded-core/message/169617 This bug should be fixed in https://git.openembedded.org/openembedded-core/commit/?id=9916212b4810280108107d94cd2c0cd3e07ae6ed Feel free to reopen this bug report if the issue still exists. |