Last time I had problems (crazy transient issues, seemingly unrelated), Travis people asked me to open an issue at https://github.com/travis-ci/travis-ci/issues. Everything else failing, they might let you ssh to a machine to debug the issue. They seem to be very helpful and investigate quickly (even though I have next to 0 users).
A few other things I've noticed while looking around:
* For what it's worth, you're not really supposed to specify patch versions (7.4.2, as opposed to 7.4) of GHC in your Travis config.
However, you can probably reuse ekmett's Travis config (not sure about licenses), since it seems more flexible and installs a different GHC, maybe with a better setup (Travis seems to be short on resident Haskell experts: https://github.com/travis-ci/travis-ci/issues/2690#issuecomment-52875095)
https://github.com/ekmett/lens/blob/master/.travis.yml
* I've encountered something "like this", for crazy values of "like" — as in, even stranger universe implosions on Travis for Haskell (though that looks some orthogonal setup problem):
https://github.com/travis-ci/travis-ci/issues/2761
* That seems solved in later builds, but
why the GHC version is changing midway here? I've found that linked in a comment from your Travis configuration.
https://travis-ci.org/agda/agda/jobs/38699335#L402-L405