Is there a way to change where the temporary directory is made?

There is not.

But it shouldn't be necessary, since during a complete test run, (most) generated files are deleted after each individual test has finished.

This behavior recently changed (#9758), see my previous email about this: https://mail.haskell.org/pipermail/ghc-devs/2016-April/011976.html