Aside from system libraries, if you don't configure your packages to be dynamic GHC works that way. You can also force system libraries if you use -static; but note that this also links libc static, which is problematic on at least Linux and Solaris. (Usually, the only problematic system library is gmp.)

Note that the gmp case is also not significantly different from any random Linux program, and you could in fact choose to bundle the appropriate libgmp.so.whatever with your program and use a wrapper script which sets LD_LIBRARY_PATH (on Linux or Solaris; DYLD_FALLBACK_LIBRARY_PATH on recent OS X).

Part of whats throwing you off, I suspect, is that GHC is a native code compiler. You might want to look at what it takes for applications for other native compilers, such as C and C++, to bundle necessary libraries. Perl is an interpreter, and bundling just means including the necessary modules in a local library and making use of local::lib, or at worst 'use lib'. It's a much simpler situation than native code compilation. Even in the case where a .exe is generated for Windows, it's still making use of interpreted code; Perl cannot be compiled to native code at all sanely. The .exe instead is instead either a bit like a self-unpacking archive, or a wrapper which invokes a Perl interpreter on a module tree with various environment variables and paths set. JForth, similarly, is based on the JVM instead of native code, and it's trivial to bundle up everything needed in a jar.

(Apple's done a fair amount of work to make bundling native code easier by means of special linker paths in application and framework bundles which both the compile-time ld and the runtime dyld can interpret, but this is of course not of any use on Windows or Linux.)