7634 2015-04-21 09:00:09 +0000 recipetool: extend autotools recipe creation 2016-03-02 21:33:40 +0000 1 1 7 Build System, Metadata & Runtime OE-Core core 1.8 All Multiple RESOLVED FIXED Medium enhancement 2.1 1 bluelightning bluelightning kergoth meta.mr.watcher meta.watcher ross.burton --- --- No (bug/feature does not impact docs) oldest_to_newest 50434 0 bluelightning 2015-04-21 09:00:09 +0000 recipetool has the ability to create recipes for software that uses autoconf/automake with a fairly basic ability to extract dependencies and other information from configure.ac; however there are macros and syntax that this doesn't currently handle. Someone with a better understanding of autotools needs to extend it; we may need to properly parse the files instead of the keyword searching we are doing now. The implementation for this is in scripts/lib/recipetool/create_buildsys.py. 52181 1 kergoth 2015-07-07 16:18:56 +0000 I once had some prototyped autotools.bbclass changes which pulled e.g. the gettext macro from a call to autoconf rather than grep, we could probably do something similar here. E.g. this old snippet: + if [ -n "`autoconf -t AC_CANONICAL_BUILD`" ]; then + macrodir="`autoconf -t AC_CONFIG_MACRO_DIR|cut -d: -f4|tail -n 1`" + if [ -z "$macrodir" ]; then + macrodir="." + fi Admittedly running autoconf repeatedly wasn't ideal, but you get the idea -- you can use autoconf tracing to determine if a macro is called, and if so, with what arguments: -t, --trace=MACRO[:FORMAT] report the list of calls to MACRO 52486 2 kergoth 2015-07-20 17:32:48 +0000 $ cd bash $ autoconf -t AC_CHECK_LIB configure.ac:416:AC_CHECK_LIB:cposix:strerror:LIBS="$LIBS -lcposix" configure.ac:543:AC_CHECK_LIB:termcap:tgetent:bash_cv_termcap_lib=libtermcap:AC_CHECK_LIB(tinfo, tgetent, bash_cv_termcap_lib=libtinfo, [AC_CHECK_LIB(curses, tgetent, bash_cv_termcap_lib=libcurses, [AC_CHECK_LIB(ncurses, tgetent, bash_cv_termcap_lib=libncurses, bash_cv_termcap_lib=gnutermcap)])]) configure.ac:543:AC_CHECK_LIB:tinfo:tgetent:bash_cv_termcap_lib=libtinfo:AC_CHECK_LIB(curses, tgetent, bash_cv_termcap_lib=libcurses, [AC_CHECK_LIB(ncurses, tgetent, bash_cv_termcap_lib=libncurses, bash_cv_termcap_lib=gnutermcap)]) configure.ac:543:AC_CHECK_LIB:curses:tgetent:bash_cv_termcap_lib=libcurses:AC_CHECK_LIB(ncurses, tgetent, bash_cv_termcap_lib=libncurses, bash_cv_termcap_lib=gnutermcap) configure.ac:543:AC_CHECK_LIB:ncurses:tgetent:bash_cv_termcap_lib=libncurses:bash_cv_termcap_lib=gnutermcap configure.ac:855:AC_CHECK_LIB:dl:dlopen configure.ac:870:AC_CHECK_LIB:sun:getpwent configure.ac:875:AC_CHECK_LIB:socket:getpeername:bash_cv_have_socklib=yes:bash_cv_have_socklib=no:-lnsl configure.ac:875:AC_CHECK_LIB:nsl:t_open:bash_cv_have_libnsl=yes:bash_cv_have_libnsl=no configure.ac:1033:AC_CHECK_LIB:termcap:tgetent:bash_cv_termcap_lib=libtermcap:AC_CHECK_LIB(tinfo, tgetent, bash_cv_termcap_lib=libtinfo, [AC_CHECK_LIB(curses, tgetent, bash_cv_termcap_lib=libcurses, [AC_CHECK_LIB(ncurses, tgetent, bash_cv_termcap_lib=libncurses, bash_cv_termcap_lib=gnutermcap)])]) configure.ac:1033:AC_CHECK_LIB:tinfo:tgetent:bash_cv_termcap_lib=libtinfo:AC_CHECK_LIB(curses, tgetent, bash_cv_termcap_lib=libcurses, [AC_CHECK_LIB(ncurses, tgetent, bash_cv_termcap_lib=libncurses, bash_cv_termcap_lib=gnutermcap)]) configure.ac:1033:AC_CHECK_LIB:curses:tgetent:bash_cv_termcap_lib=libcurses:AC_CHECK_LIB(ncurses, tgetent, bash_cv_termcap_lib=libncurses, bash_cv_termcap_lib=gnutermcap) configure.ac:1033:AC_CHECK_LIB:ncurses:tgetent:bash_cv_termcap_lib=libncurses:bash_cv_termcap_lib=gnutermcap 54405 3 kergoth 2015-09-21 17:53:19 +0000 First, a note. You can pass any number of -t <macro> arguments to one autoconf execution, to get everything we need from one, and parse the output. Next, from my personal todo, added here for safe-keeping: - [ ] Use autoconf -t to reproduce the current capability for dependencies (AC_CHECK_LIB, PKG_CHECK_MODULES, AM_GNU_GETTEXT, AC_CHECK_PROG, AC_PATH_PROG, AC_PATH_X) - [ ] Possibly also handle AC_SEARCH_LIBS - [ ] Possibly also handle other prog variants: AC_CHECK_PROG, AC_CHECK_PROGS, AC_CHECK_TARGET_TOOL, AC_CHECK_TOOL, AC_CHECK_TARGET_TOOLS, AC_CHECK_TOOLS, AC_PATH_PROGS, AC_PATH_PROGS_FEATURE_CHECK, AC_PATH_TARGET_TOOL, AC_PATH_TOOL - [ ] Possibly also check specific tool macros: AC_PROG_INSTALL, AC_PROG_AWK, AC_PROG_GREP, AC_PROG_EGREP, AC_PROG_FGREP, AC_PROG_MKDIR_P, AC_PROG_LEX, AC_PROG_LN_S, AC_PROG_RANLIB, AC_PROG_SED, AC_PROG_YACC (yacc & lex being the important ones, for the flex-native/bison-native deps) - [ ] Consider: checking for cross-compilation issues. This is tougher, since -t doesn't tell us if the macro is only called inside a if block for cross_compilation=<yes,no>, or if the macro is called inside an AC_CACHE_CHECK (though we might be able to figure that out, either through line number or examining the AC_CACHE_CHECK arguments). We could warn about *possible* issues for AC_CHECK_FILE, AC_CHECK_FILES, AC_RUN_IFELSE, and catch AC_CACHE_CHECK to try to reduce the false positives. We could also warn about passing the cross-compilation fallback argument of AC_RUN_IFELSE, as that value isn't generally target specific, so could well be wrong. 54406 4 kergoth 2015-09-21 17:55:30 +0000 Example output with multiple -t's, with the macros we likely want to handle initially, run against GNU nano svn trunk: $ autoconf -t AC_CHECK_LIB -t AC_SEARCH_LIBS -t PKG_CHECK_MODULES -t AC_CHECK_PROG -t AC_CHECK_PROGS -t AC_PATH_PROG -t AC_PATH_PROGS -t AC _PATH_PROGS_FEATURE_CHECK -t AM_GNU_GETTEXT -t AC_PATH_X -t AC_PROG_YACC -t AC_PROG_LEX configure.ac:35:AC_CHECK_PROG:CC:${ac_tool_prefix}gcc:${ac_tool_prefix}gcc:: configure.ac:35:AC_CHECK_PROG:CC:${ac_tool_prefix}cc:${ac_tool_prefix}cc configure.ac:35:AC_CHECK_PROG:CC:cc:cc:::/usr/ucb/cc configure.ac:35:AC_CHECK_PROG:CC:$ac_tool_prefix$ac_prog:$ac_tool_prefix$ac_prog:: configure.ac:35:AC_CHECK_PROGS:ac_ct_CC:cl.exe:: configure.ac:35:AC_CHECK_PROG:ac_ct_CC:$ac_prog:$ac_prog:: configure.ac:36:AC_CHECK_PROG:CC:${ac_tool_prefix}gcc:${ac_tool_prefix}gcc:: configure.ac:36:AC_CHECK_PROG:CC:${ac_tool_prefix}cc:${ac_tool_prefix}cc configure.ac:36:AC_CHECK_PROG:CC:cc:cc:::/usr/ucb/cc configure.ac:36:AC_CHECK_PROG:CC:$ac_tool_prefix$ac_prog:$ac_tool_prefix$ac_prog:: configure.ac:36:AC_CHECK_PROGS:ac_ct_CC:cl.exe:: configure.ac:36:AC_CHECK_PROG:ac_ct_CC:$ac_prog:$ac_prog:: configure.ac:38:AC_SEARCH_LIBS:strerror:cposix configure.ac:243:AC_CHECK_LIB:${termlib}:tputs:tcap="-l$termlib" configure.ac:243:AC_CHECK_LIB:${termlib}:tputs:tcap="-l$termlib" configure.ac:491:AC_CHECK_PROG:NCURSESW_CONFIG:${ac_tool_prefix}ncursesw5-config:${ac_tool_prefix}ncursesw5-config:: configure.ac:496:AC_CHECK_LIB:ncursesw:get_wch:CURSES_LIB_NAME=ncursesw CURSES_LIB_WIDE=yes configure.ac:498:AC_CHECK_LIB:ncursesw:get_wch:CURSES_LIB="-lncursesw" CURSES_LIB_NAME=ncursesw CURSES_LIB_WIDE=yes configure.ac:504:AC_CHECK_LIB:ncurses:initscr:CURSES_LIB="-lncurses" CURSES_LIB_NAME=ncurses configure.ac:512:AC_CHECK_LIB:curses:get_wch:CURSES_LIB="-lcurses" CURSES_LIB_NAME=curses CURSES_LIB_WIDE=yes configure.ac:516:AC_CHECK_LIB:curses:initscr:CURSES_LIB="-lcurses" CURSES_LIB_NAME=curses configure.ac:529:AC_CHECK_LIB:$CURSES_LIB_NAME:use_default_colors: $as_echo "#define HAVE_USE_DEFAULT_COLORS 1" >>confdefs.h configure.ac:658:AC_CHECK_LIB:magic:magic_open configure.ac:658:AC_CHECK_LIB:z:inflate configure.ac:680:AC_CHECK_PROG:haveit:makeinfo:yes:no 57962 5 bluelightning 2016-01-19 18:01:35 +0000 I've made a number of improvements to recipetool's ability to extract dependencies from configure.ac; it's still not perfect, but it's better. I didn't end up going the route of using autoconf -t because it didn't really make things easier - I also found it didn't help with situations such as where other macros had been defined and then used within a statement; the code we have now is able to expand those. I'm not sure where we want to go with this bug. We're still stuck with situations such as where environment variables get used since they aren't really intended to be expanded until the configure script actually becomes a shell script; beyond actually implementing a proper M4 parser I can't see any practical way to improve this other than incrementally. 58096 6 ross.burton 2016-01-21 22:25:19 +0000 A number of relevant patches were merged into oe-core as of c58669fb0977f7f0cb79f252484d5c5ef0dfb7e4. 59641 7 bluelightning 2016-03-02 21:33:40 +0000 I think I'm going to call this done - if we need to extend the capabilities or fix issues then we can create more specific bugs.