Add a hack to handle unknown chars in fontconfig.

The unicode long is added to the cache. So when fontconfig does fall back to
the default font (where there is no easy way to find this out from the
pattern) it isn't reloaded.