The Fedora packaging has been modified to compile libpython with -fno-semantic-interposition flag: it makes Python up to 1.3x faster without having to touch any line of the C code! Should we do the same? https://bugs.python.org/issue38980 From a recent build, it seems that flag is not used: $ rgrep no-semantic-interposition tmp-glibc/work/core2-64-wrs-linux/python3/3.9.2-r0 $ echo "No hits" Trevor has expressed interest in this but I didn't want it to get lost so I've created this enhancement.
You may want to read https://stackoverflow.com/questions/35745543/new-option-in-gcc-5-3-fno-semantic-interposition first and then see whether visibility settings might be safer?
It is probably ok to enable this for an individual recipe, we just need to comment why and explain tit means library intercept/replacement won't work.
It will break things if we do it on python-native so only sdk and target.
python3 ptest durations without the flag added to the recipe: root@qemux86-64:~# grep -r "Total duration" python3-ptest-time-1.log:Total duration: 22 min 37 sec python3-ptest-time-2.log:Total duration: 24 min python3-ptest-time-4.log:Total duration: 23 min 17 sec python3-ptest-time-5.log:Total duration: 22 min 33 sec python3-ptest-time-3.log:Total duration: 30 min 3 sec root@qemux86-64:~# python3 ptest durations with -fno-semantic-interposition added to CPPFLAGS: root@qemux86-64:~# grep -r "Total duration" python3-ptest-with-opt-2.log:Total duration: 22 min 5 sec python3-ptest-with-opt-4.log:Total duration: 22 min 30 sec python3-ptest-with-opt-1.log:Total duration: 22 min 11 sec python3-ptest-with-opt-3.log:Total duration: 22 min 36 sec python3-ptest-with-opt-5.log:Total duration: 22 min 14 sec root@qemux86-64:~# This doesn't seem like a worthwhile change, but I'll keep looking to see if it manifests more meaningfully in other areas.
I agree, gains seems minimal I think python 3.13 is doing some useful work in this regard https://github.com/faster-cpython/ideas/blob/main/3.13/README.md but it wont be available until 2024
Closing this for now as it doesn't seem to provide a meaningful gain to target builds.
Thanks for checking on this again Trevor. I don't expect to see much of a difference in ptest results. It would be interesting to see if there's a difference in https://github.com/python/pyperformance as mentioned in: https://fedoraproject.org/wiki/Changes/PythonNoSemanticInterpositionSpeedup but you likely have better things to do...
In case you or someone else decides to test pyperformance, note that it is part of the phoronix-test-suite but I don't know if it's build by default. There's current isn't a pyperformance recipe. tmp-glibc/work/core2-64-wrs-linux/phoronix-test-suite/10.8.4-r0/phoronix-test-suite/ob-cache/test-profiles/pts/pyperformance-1.0.1
It doesn't appear to be installed with phoronix by default. I wrote a standalone recipe for pyperformance, but after getting all of the dependencies sorted, along with providing enough extra storage and a nameserver in /etc/resolv.conf (part of the benchmark setup involves creating a venv and installing packages with pip, even if you have the dependencies in the image), the benchmark ran for a while and then stalled. Not sure what to make of it.
gcc has enabled this option now as part of -Ofast with https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=458d2c689963d8461d84670a3d8988cd6ecbfd81 Therefore it would be interesting to see if we change SELECTED_OPTIMIZATIONS from -O2 to -Ofast for python and cpython recipes, we might get some interesting results.