INSANE_SKIP_${PN} = "pkgconfig"
... does not work at all.

But to skip other checks with the INSANE_SKIP feature works well, for example:

INSANE_SKIP_{PN} = "ldflags"
... works as expected.

The documentation seems to express that the feature INSANE_SKIP_ can be applied to any check of the list given in the docs.

My workaround was to do:
ERROR_QA_remove = "pkgconfig"
But I think this workaround is more global than necessary, as skipping the check should only affect the single recipe.