It's a small snippet and I've put in the comments stating how I run into "out of file handles" or simply file not getting read due to lazy IO.

I realize that putting ($!) using a trial/error approach is going to be futile. I'd appreciate some pointers into the tools I could use to get some idea of which expressions are building up huge thunks.

Have you tried System.IO.Strict's readFile? I had similar problems (too many file handles) and fixed it with

import qualified System.IO.Strict as S

and then using S.readFile instead of the standard prelude's readFile.

This is where I used the strict IO readFile in my toy project: https://github.com/carlohamalainen/checker/blob/master/Checker.hs