14943 2022-10-20 18:00:18 +0000 security_flags.inc: Segmentation fault observed using shared object from python (using ctypes module) when PIE is enabled 2022-11-28 09:29:41 +0000 1 1 7 Build System, Metadata & Runtime OE-Core configuration unspecified All Multiple RESOLVED NOTABUG Medium+ major 4.2 M1 1 BHARGAV_DAS sundeep.kokkonda randy.macleod richard.purdie yashinde145 --- --- Don't know oldest_to_newest 94224 0 4908 BHARGAV_DAS 2022-10-20 18:00:18 +0000 Created attachment 4908 Files to reproduce issue security_flags.inc: Segmentation fault while using shared object (.so) from python, ctypes module when PIE is enabled. This was first observed in dunfell branch but it is also reproducible is poky master. Steps to reproduce, 1. Clone poky 2. source poky/oe-init-build-env -b build-qemu qemux86-64 3. Add the following in local.conf - require conf/distro/include/security_flags.inc - IMAGE_INSTALL_append = “ openssh python3 python3-ctypes” #(this is required to bring in the test binaries and for the python modules) 4. Build core-image-minimal 5 .Build toolchain/sdk using, - bitbake -c populate_sdk core-image-minimal 6. After the sdk build, install the sdk in a location, sdk is present in tmp/deploy/sdk, <sdk>.sh 7. Export the variables, using - source environment-setup-aarch64-mel-linux (this file will be present inside the SDK installation location) 8. For the next step, use the attached files from the attachments. 9. Compile the .so (shared object) using, (make sure hellopython.cc and hellopython.h is present in the same directory) - $CXX $CXXFLAGS $LDFLAGS -shared -Wl,-soname,libhellopython -o libhellopython.so -fPIC hellopython.cc 10.Boot qemux86-64 image 11.Transfer "libhellopython.so" and "hellopy.py" to the target. (use scp, since openssh is already added) 11.Copy "libhellopython.so" to /usr/lib in target (qemu). 12.Run the python script using - python3 ./hellopy.py 13.A 'segmentation fault' should be observed. Note: 1. 'segmentation fault' is not observed when security_flags.inc is not included in local.conf. 2. When python is excluded from compiling with PIE using the following line, 'segmentation fault' is not observed - SECURITY_CFLAGS_pn-python3 = "${SECURITY_NOPIE_CFLAGS}" 3. This issue not observed in ubuntu 18.04, ubuntu 20.04 and debian 11 4. When the compiled libhellopython.so is access by a native c++ script i.e. hellocxx.cpp (attached), 'segmentation fault' is not observed. 94249 1 richard.purdie 2022-10-27 13:48:59 +0000 If you compile the test case using a recipe, is the segfault still observed? If so that would simplify the reproduction of the bug and help track down the issue. We need to work out if the SDK toolchain is causing part of the problem somehow... 94259 2 randy.macleod 2022-10-27 14:35:25 +0000 See Richard's question above. 94303 3 BHARGAV_DAS 2022-10-28 07:30:42 +0000 @Richard, yes the segmentation fault is observed even when compiled using recipe. Please check the attached zip "test-app-python-recipe.zip" for the recipe. 94304 4 4910 BHARGAV_DAS 2022-10-28 07:32:05 +0000 Created attachment 4910 Test application recipe 94341 5 randy.macleod 2022-11-03 14:33:57 +0000 Seems to be reproducible with the core toolchain rather than just the SDK so that's good news. 94386 6 BHARGAV_DAS 2022-11-10 13:29:02 +0000 Hi Randy, is there any update on this issue or a timeline by which we expect this bug to be resolved ? 94406 7 yashinde145 2022-11-10 16:25:02 +0000 I am analysing the segmentation fault. It would be due to the shared object file generated from C++ sources and then using it to compile python file . C/C++ have pointers whereas python does not support them, so there might be some memory allocation/deallocation problem while compiling the python file with shared object generated from C++ sources. Checking on that. 94414 8 4912 yashinde145 2022-11-16 07:23:34 +0000 Created attachment 4912 The updated hellopy.py 94415 9 yashinde145 2022-11-16 07:36:08 +0000 (In reply to comment #8) > Created attachment 4912 [details] > The updated hellopy.py PFA hellopy.py file. According to the analysis done, it was found that the function argument types and function return types of the functions accessed from the shared object(generated from C++ files) were not mentioned in the given python file. Whenever working with ctypes in Python, the function argument types and return types should be informed explicitly(lines 5-10 in given hellopy.py) in python file otherwise it gives segmentation fault. Adding these lines fixes the segmentation fault. Since this is a code missing problem, adding SECURITY_CFLAGS_pn-python3 = "${SECURITY_NOPIE_CFLAGS}" in build still reproduces the segmentation fault. 94416 10 BHARGAV_DAS 2022-11-16 10:01:01 +0000 > Since this is a code missing problem, adding SECURITY_CFLAGS_pn-python3 = "${SECURITY_NOPIE_CFLAGS}" in build still reproduces the segmentation fault. @Yash in the master, when we add ' SECURITY_CFLAGS:pn-python3 = "${SECURITY_NOPIE_CFLAGS}" ' (check the new override syntax '_' vs ':') the segmentation fault is not observed. 94417 11 BHARGAV_DAS 2022-11-16 10:28:15 +0000 >> Since this is a code missing problem, adding SECURITY_CFLAGS_pn-python3 = "${SECURITY_NOPIE_CFLAGS}" in build still reproduces the segmentation fault. > @Yash in the master, when we add ' SECURITY_CFLAGS:pn-python3 = "${SECURITY_NOPIE_CFLAGS}" ' (check the new override syntax '_' vs ':') the segmentation fault is not observed. This is for the original hellopy.py. Checked the updated hellopy.py it works but what I don't understand is why the original code doesn't work when pie is used to compile python. 94418 12 yashinde145 2022-11-16 10:46:14 +0000 (In reply to comment #11) > >> Since this is a code missing problem, adding SECURITY_CFLAGS_pn-python3 = "${SECURITY_NOPIE_CFLAGS}" in build still reproduces the segmentation fault. > > > @Yash in the master, when we add > ' SECURITY_CFLAGS:pn-python3 = "${SECURITY_NOPIE_CFLAGS}" ' (check the new > override syntax '_' vs ':') the segmentation fault is not observed. > > This is for the original hellopy.py. > > Checked the updated hellopy.py it works but what I don't understand is why > the original code doesn't work when pie is used to compile python. Please refer the explanation given in comment 9. Occurring of segmentation fault is independent of using PIE . Python should be explicitly informed about the function argument types and returns types when using ctypes which was not present in the original hellopy.py. 94454 13 yashinde145 2022-11-28 08:33:39 +0000 If there are no other queries, can we close this issue? 94455 14 BHARGAV_DAS 2022-11-28 08:51:35 +0000 Yes, we can close this issue. 94457 15 yashinde145 2022-11-28 09:16:24 +0000 Closing the issue. 94458 16 yashinde145 2022-11-28 09:29:41 +0000 Issue closed. 4908 2022-10-20 18:00:18 +0000 2022-10-20 18:00:18 +0000 Files to reproduce issue segmentation-fault.zip application/x-zip-compressed 1633 BHARGAV_DAS UEsDBBQAAAAAAFy7VFUAAAAAAAAAAAAAAAATAAAAc2VnbWVudGF0aW9uLWZhdWx0L1BLAwQUAAAA CABYu1RV4AIzzJEAAADkAAAAHwAAAHNlZ21lbnRhdGlvbi1mYXVsdC9oZWxsb2N4eC5jcHBNjUEK wjAURPeB3GGom7ZIca917UoKPYDUNphI/JHkVxHx7sZWNLN8zLxZGOrtOChsAg/GVXorxY9lWlnr rg/WjiqdSSGFIcalM4QcN2cGFFI8pcA3vaPA6HXnS7AKHNgbOqGeRZhN2fo/2H14M+ES7nhGnaLD Xt3zIqm3ipto5Hby5nGwTH7Splc8esIqotcbUEsDBBQAAAAIAHu4VFWSEUnXmgAAAPMAAAAdAAAA c2VnbWVudGF0aW9uLWZhdWx0L2hlbGxvcHkucHlljrEOwjAMRHd/RbckEmRmYaIDQ4Uq9QMiGgwt CkmUGKH+PW5oYWCz7t7deXzEkKiyNEXMAAM6F+Jkq/0i6UPdNFK4sV8sGoLXOQgFQJgpUxr9jXFR /OoDCIDQ31ld+/RxPtpimhO+pAJrOMvIr0Wjt+GCUjzput3xwjfdIbVMUFc4yd2b9T9r7HBOJsrS p0rof7JGh4RzUMEbUEsDBBQAAAAIADSDFlWTTv9J7AAAAA0CAAAhAAAAc2VnbWVudGF0aW9uLWZh dWx0L2hlbGxvcHl0aG9uLmNjZVBLbsIwEN37FNN0k1Rt2Scpm3bRFULiABGKB2LJ2JE94SMUzo5t IJjEC0tv3sfP8y5ULTuOUAptyeB6N2dsGCYNSqnbEzVafTcJY/8eLwPO8wikGZxZP+IvYwG4szRC 0YrcvU2zwnv2WnB4MUaawWeJ53mtO4KyhNYLKhsUlRvcWFRchkQ8EhoFyW/izHEn+IhAtcDDEG+Q OmdReIiruIbgAqcdqz+USJi+ZFuUm0ee2KQewtsPqE7Klkxg7ocHN3hFEYb9sIgVUvT96QOfUGtl Cepmbdzkkeqpr/nIbLMiIqeb72E2g+ey2BVQSwMEFAAAAAgANIMWVQom2hPkAAAAygEAACAAAABz ZWdtZW50YXRpb24tZmF1bHQvaGVsbG9weXRob24uaHWQwWrDMBBE7/sVSwzFKf0Cu+2lLfgQEkN6 6Um40roRCDlYm7QluN/elQ1BTsiepMfsjEaZbb2hFqu31Wqj6o/3arNWFWTCrKdLDJn12h0M4WPg 3vqvZwDtmhCwIue6+pd3nUc4AcrsD5/O6kLO46SSfFme8d8NfuyswS1xLTm8HdNy3fnAGNgUxZR/ h2GJJ9xHjZqQwicMJQ5zo9QlAgmKgrh5bJjOr0zN57YlDLJDP0y9x8XLQlpC2vo+uag1fccuY3TK X8kRUz7bC+TaqL3ue617wOkL9K7phcjaABl5Y1v4B1BLAQI/ABQAAAAAAFy7VFUAAAAAAAAAAAAA AAATACQAAAAAAAAAEAAAAAAAAABzZWdtZW50YXRpb24tZmF1bHQvCgAgAAAAAAABABgA4gSjVq3k 2AEhMCmIreTYAQwGjaiJxtgBUEsBAj8AFAAAAAgAWLtUVeACM8yRAAAA5AAAAB8AJAAAAAAAAAAg AAAAMQAAAHNlZ21lbnRhdGlvbi1mYXVsdC9oZWxsb2N4eC5jcHAKACAAAAAAAAEAGAB1jDdSreTY AVn8FJCt5NgBM8iSH63k2AFQSwECPwAUAAAACAB7uFRVkhFJ15oAAADzAAAAHQAkAAAAAAAAACAA AAD/AAAAc2VnbWVudGF0aW9uLWZhdWx0L2hlbGxvcHkucHkKACAAAAAAAAEAGAC9wZMfquTYAVhf 6JWt5NgBAJ6ztxW22AFQSwECPwAUAAAACAA0gxZVk07/SewAAAANAgAAIQAkAAAAAAAAACAAAADU AQAAc2VnbWVudGF0aW9uLWZhdWx0L2hlbGxvcHl0aG9uLmNjCgAgAAAAAAABABgAAJ6ztxW22AHE FhKQreTYAQCes7cVttgBUEsBAj8AFAAAAAgANIMWVQom2hPkAAAAygEAACAAJAAAAAAAAAAgAAAA /wIAAHNlZ21lbnRhdGlvbi1mYXVsdC9oZWxsb3B5dGhvbi5oCgAgAAAAAAABABgAAJ6ztxW22AEj 3xCQreTYAQCes7cVttgBUEsFBgAAAAAFAAUAKgIAACEEAAAAAA== 4910 2022-10-28 07:32:05 +0000 2022-10-28 07:32:05 +0000 Test application recipe test-app-python-recipe.zip application/zip 2125 BHARGAV_DAS UEsDBAoAAAAAABZnXFUAAAAAAAAAAAAAAAAQABwAdGVzdC1hcHAtcHl0aG9uL1VUCQADNIRbYzeE W2N1eAsAAQToAwAABOgDAABQSwMEFAAAAAgAm2ZcVUlFmO4fAQAAawIAACIAHAB0ZXN0LWFwcC1w eXRob24vdGVzdC1hcHAtcHl0aG9uLmJiVVQJAANOg1tjs4NbY3V4CwABBOgDAAAE6AMAAI1SXWuD MBR9z6+4iA8ba3RvQmEPndohs7UY+jEYiNUUA9FI9UXE/74k7Uq7urE8XHLPyTk5+SDrxWIWxx/w AkZLmxbqri1EBTWjBgoD118SX3FuGBHfMxAisZus40Bh8IlAjgPjdGrbBeVc1J1Vd6O4crWy7Heu kJT0z0WSibKW/MNjr9ea7m6nyzycvREwQ+80wU2RHmkOeMsnuBFVWtIJZ/srT8ACbhGrEYAPq8AF s99G8bsXxMOPhGjQIVjVtCnn3yHOLeBcKr3BNvs9q3J2HG7ZEp4dxxkxV9fyl/BiK/P+x9bsV8sB y7oZ7Psjjnjpoc42D0KfTLUenuQrXgJdRTVgZNnZ7X47A5EoDF6J+hO61drkBHr+RuHyZb8AUEsD BAoAAAAAAJtmXFUAAAAAAAAAAAAAAAAgABwAdGVzdC1hcHAtcHl0aG9uL3Rlc3QtYXBwLXB5dGhv bi9VVAkAA06DW2Ong1tjdXgLAAEE6AMAAAToAwAAUEsDBBQAAAAIAJtmXFWTTv9J6wAAAA0CAAAu ABwAdGVzdC1hcHAtcHl0aG9uL3Rlc3QtYXBwLXB5dGhvbi9oZWxsb3B5dGhvbi5jY1VUCQADToNb Y9mDW2N1eAsAAQToAwAABOgDAABlUL1ugzAQ3v0UV7JA1TY7kCzt0KlC4gFQhC/FkmMj+2haVfTZ a1uBGPBg6fv1+XZCtXLgCKXQlgyeLkfGdhOZdCil7n+o0+qlSxh797gKOM8jkGbwy8aV/rc2gDuV EYpqcvdnmhU+86UFh0Vw4bnlLPE8b/VAUJbQe0Njg6PxRFBRcRka8ZvQKEheExeOZ4LHCDQfeJ3r DdLgIgqvsJi6AF+4mbF5Q4mE6aLbojxPfeKceggPB1CDlD2ZSfGHhzR4RxHIcV5EjRR/f/PAE7Ra WYK2OxnHzOtx0vNxFbZZEYnbzY+w38N9WewfUEsDBBQAAAAIAJtmXFUKJtoT5QAAAMoBAAAtABwA dGVzdC1hcHAtcHl0aG9uL3Rlc3QtYXBwLXB5dGhvbi9oZWxsb3B5dGhvbi5oVVQJAANOg1tj3INb Y3V4CwABBOgDAAAE6AMAAHWQwWrDMBBE7/sVSwzBKf0Cu+2lLfgQHENy6Uk40joWCDlY67QhON9e 2aaNnFKdpMfszI4iXVlFFWbv6/VGFB+7bJOLDCLPtKV7DJG20nSK8Mlxq+3hBUCa0jnMyJimOHPd WIQLoD/Hbm+0TMb7cAJJvEp/8fUffmq0wi1x4XN4O6bFsrGO0bFKkil/iW6FFzwOGjEhgc/oUuzn RqHLAHzQtGSrTyXTbcvAfG6bQu9n6Iuptbh4XfiWELZ+CB4ip8+hyxgd8jcyxBTP5hyZ6kd71/eP 7hGnL5B12Xrix3qIyCpdwTdQSwMEFAAAAAgAm2ZcVZIRSdeaAAAA8wAAACoAHAB0ZXN0LWFwcC1w eXRob24vdGVzdC1hcHAtcHl0aG9uL2hlbGxvcHkucHlVVAkAA06DW2PTg1tjdXgLAAEE6AMAAATo AwAATY7BDoIwEETv+xXc2ibasxdPcvBADAkf0EhdBVNp0y4x/L1LBeW2mXkzs/0r+EiFpSlgAujQ OR8mWxwXSZ/KqpLC9e1iUecHnbxQAISJEsV+eDAusl98AQHg2yera58+z0edTXPBt1RgDWcZ+bdo HKy/oRQj3fcHXvilG6SaCWoyJ7l7t/5nje2u0QSZ+9QmtJ0s0SHhHFTwAVBLAQIeAwoAAAAAABZn XFUAAAAAAAAAAAAAAAAQABgAAAAAAAAAEAD9QQAAAAB0ZXN0LWFwcC1weXRob24vVVQFAAM0hFtj dXgLAAEE6AMAAAToAwAAUEsBAh4DFAAAAAgAm2ZcVUlFmO4fAQAAawIAACIAGAAAAAAAAQAAALSB SgAAAHRlc3QtYXBwLXB5dGhvbi90ZXN0LWFwcC1weXRob24uYmJVVAUAA06DW2N1eAsAAQToAwAA BOgDAABQSwECHgMKAAAAAACbZlxVAAAAAAAAAAAAAAAAIAAYAAAAAAAAABAA/UHFAQAAdGVzdC1h cHAtcHl0aG9uL3Rlc3QtYXBwLXB5dGhvbi9VVAUAA06DW2N1eAsAAQToAwAABOgDAABQSwECHgMU AAAACACbZlxVk07/SesAAAANAgAALgAYAAAAAAABAAAAtIEfAgAAdGVzdC1hcHAtcHl0aG9uL3Rl c3QtYXBwLXB5dGhvbi9oZWxsb3B5dGhvbi5jY1VUBQADToNbY3V4CwABBOgDAAAE6AMAAFBLAQIe AxQAAAAIAJtmXFUKJtoT5QAAAMoBAAAtABgAAAAAAAEAAAC0gXIDAAB0ZXN0LWFwcC1weXRob24v dGVzdC1hcHAtcHl0aG9uL2hlbGxvcHl0aG9uLmhVVAUAA06DW2N1eAsAAQToAwAABOgDAABQSwEC HgMUAAAACACbZlxVkhFJ15oAAADzAAAAKgAYAAAAAAABAAAAtIG+BAAAdGVzdC1hcHAtcHl0aG9u L3Rlc3QtYXBwLXB5dGhvbi9oZWxsb3B5LnB5VVQFAANOg1tjdXgLAAEE6AMAAAToAwAAUEsFBgAA AAAGAAYAewIAALwFAAAAAA== 4912 2022-11-16 07:23:34 +0000 2022-11-16 07:23:34 +0000 The updated hellopy.py hellopy.py application/octet-stream 543 yashinde145 aW1wb3J0IGN0eXBlcwpoZWxsb3B5YyA9IGN0eXBlcy5DRExMKCdsaWJoZWxsb3B5dGhvbi5zbycp CnRlc3RzdHJpbmcgPSAnaGVsbG8gcHl0aG9uJwoKaGVsbG9weWMuU2V0UHJpbnRTdHJpbmcuYXJn dHlwZXMgPSBbY3R5cGVzLmNfdm9pZF9wLCBjdHlwZXMuY19jaGFyX3BdCmhlbGxvcHljLlNldFBy aW50U3RyaW5nLnJlc3R5cGUgPSBOb25lCmhlbGxvcHljLkhlbGxvUHl0aG9uX0RlbGV0ZS5hcmd0 eXBlcyA9IFtjdHlwZXMuY192b2lkX3BdCmhlbGxvcHljLkhlbGxvUHl0aG9uX0RlbGV0ZS5yZXN0 eXBlID0gTm9uZQpoZWxsb3B5Yy5IZWxsb1B5dGhvbl9OZXcuYXJndHlwZXMgPSAoKQpoZWxsb3B5 Yy5IZWxsb1B5dGhvbl9OZXcucmVzdHlwZSA9IGN0eXBlcy5jX3ZvaWRfcAoKb2JqID0gaGVsbG9w eWMuSGVsbG9QeXRob25fTmV3KCkKY19zdHIgPSB0ZXN0c3RyaW5nLmVuY29kZSgndXRmLTgnKQpo ZWxsb3B5Yy5TZXRQcmludFN0cmluZyhvYmosIGN0eXBlcy5jX2NoYXJfcChjX3N0cikpCmhlbGxv cHljLkhlbGxvUHl0aG9uX0RlbGV0ZShvYmopCgoK