Hmm, further experiments with creating zillions of garbage ForeignPtrs (not just 1) reveals that the problem only occurs if *no* garbage collection has occured before the program shuts down. In other words, as long as at least one garbage collection has occured, it doesn't matter if library shutdown occurs immediately in response to killLibRef or if it's deferred until the reference count hits zero as a result of finalisers being called. (This test is without the explicit performGC of course.)
So (hoping I will not have to eat my words:-) I'm begining to suspect this is a buglet in the ghc rts somewhere.
It may be a bug; I can't see anything obviously wrong in your code. The best way to proceed is for you to send us a complete test case which is producing the (claimed) incorrect output, and we'll look into it. Cheers, Simon