It should be possible to skip the libdir QA check for a specific package by adding libdir to INSANE_SKIP_${PN}. For example, meta/recipes-qt/qt4/qt4.inc adds libdir to INSANE_SKIP for the examples and examples-dbg packages.
Fix merged to master: http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=77742dd0c977202f29087dd795554e8fe9c00ec5