Re: [Haskell] hsc2hs can't find gcc under windows
On Thu, 2008-12-25 at 15:17 +0900, Curt Sampson wrote:
I'm trying to upgrade from ghc 6.8.3 to 6.10.1. Unfortunately, there seem to be problems with the Windows version. (I've installed it on two different machines, both Windows XP, and they've both had the same problem. Both worked fine with 6.8.3.)
When I run hsc2hs, on an hsc file, I get the error, "Can't find gcc".
Just as an experiment, I tried adding c:\ghc\ghc-6.10.1 to my path. That lets it find gcc, but then it can't find cc1. If I go even further down this road and add c:\ghc\ghc-6.10.1\gcc-lib to my path, it can find cc1, but then I get a stream of errors about not being able to find header files, such as HsFFI.h.
Does anybody have any clues here to help me? I can provide code, or even access to the machine, if needed.
So here's the story: Up until ghc-6.10, ghc shipped a fork of hsc2hs. The forked version used ghc as the C compiler rather than gcc. This had the advantage that it would know about ghc packages and the include files shipped with packages. The disadvantage was that it was a fork and behaved differently than using hsc2hs with hugs. It also meant that real C compiler options had to be double-escaped to get hsc2hs to pass them to ghc and get ghc to pass them to gcc. In ghc-6.10, ghc ships the normal hsc2hs. What we did not notice before 6.10.1 was released was that calling hsc2hs on its own now does not work without passing additional flags. We never noticed because few people actually do call hsc2hs directly, it is typically used via Cabal. Cabal tells it where gcc lives and what include dirs to use. So that's why at the moment in the Windows installer, hsc2hs cannot even find gcc, because gcc is not on the %PATH% and hsc2hs does not know about ghc's special bundled version of gcc. It also does not know about ghc's include directories. It's clear we'll need to think a bit more about how to get hsc2hs to be both a compiler neutral tool where we can share a single instance between multiple compilers and also have it work "out of the box".
We're also happy to pay for support, if someone out there is offering commercial support for ghc.
We offer commercial support through Well-Typed: http://www.well-typed.com/ Duncan
Duncan,
Good to talk to you again. You may remember me from ICFP.
Anyway, thanks for the hint; I can probably make a workaround by adding
the appropriate options for hsc2hs to the QAM build system. If you
happen to know what those options are, or just where in the source you
generate them, that would be a helpful pointer.
But I'd still very much like to have qam just calling Cabal with
appropriate options to build Haskell and related code. The issues, as I
recall, were:
1. Verbosity and repeitiveness of .cabal specification files. This is
actually not such a big deal, as I can have qam generate these from a
smaller specification that knows about qam's standard places to put
things.
2. Ability for each developer on a machine to install his own separate
copy (or multiple copies) of an entire application system, including
libraries, packages, and so on. I think with your --prefix and
--package-db options, this will work now, right? I can have two
different developers using two different prefixes, sharing a ghc and the
base packages from /usr/local, but using their own, say, libgtk+ and
haskell interface to it in /home/joe/project/release or whatever?
3. Fairly heavily customized file locations. Last we talked, this was
the major sticking point. Just to review, in QAM, we have a directory
for source and data for some fairly stand-alone library or chunk of
the project ("project/src/componentname", with src underneath that), a
directory for build intermediate files (subdirs under "build") and a
directory in which the entire release is installed ("release", with the
usual bin, lib, share, etc. directories under it).
(A detailed introduction to this is available here, by the way:
http://www.starling-software.com/en/qam-principles.html
One tricky thing was that we have, for those components that wish it, a
single directory in which we build all of the (non-main) object files,
and wrappers for runhaskell, ghci, and the like that use the built
sources here. This gives us an almost lisp-machine like environment for
wandering about the various parts of our system, recompiling only the
bits that are needed, experimenting interactively with whatever part
we like, including into the deep depths of libraries, and so on, while
still running very quickly. Unfortunately, there was some issue with
lack of ability to specify where the object and .hs files go, if I
recall. Is tehre any progress on this front?
Another trick was not to export our _unitTest function (exporting it
causes no end of hassles, as it turned out, when I was first setting up
the unit test framework), instead runnning the tests with runhaskell; by
using the same mechanism as above, we could interpret the file with the
tests themselves, but use compiled versions of everything else, which
saved a lot of time.
A last issue was dealing with modules that have a main function but are
also used as libraries by other modules (built without specifying a main)
function, or was it just the lack of ability to use the -main-is option
at all? I can't recall.
Anyway, any thoughts on these fronts? I suppose the common thread
here is that I use modules for several different purposes (production
release, playing with in the interpreter, as or as not the entry point
of a program, running unit tests) and I'd like to avoid compiling
different versions of each module or, even worse, different versions of
all modules for every single use of one particular module.
BTW, I'd also be happy to look at changing QAM to try to come a bit
closer to the "Cabal way of doing things," if I can be convinced that
they're better. But it could well be that, though they're better
for building stand-alone haskell programs, they don't have to deal
with a lot of the stuff that qam does (loading database and starting
servers for full-system functional tests, dealing with having several
different versions of PHP and apache on a system, generating appropriate
configuration files for those sorts of situations, etc.) and it's not
even worth thinking attempting that. (Having built this system over the
last four years or so, I can attest that dealing with this stuff is not
as trivial as one would hope.)
cjs
--
Curt Sampson
On Thu, 2008-12-25 at 15:17 +0900, Curt Sampson wrote:
I'm trying to upgrade from ghc 6.8.3 to 6.10.1. Unfortunately, there seem to be problems with the Windows version. (I've installed it on two different machines, both Windows XP, and they've both had the same problem. Both worked fine with 6.8.3.)
When I run hsc2hs, on an hsc file, I get the error, "Can't find gcc".
Just as an experiment, I tried adding c:\ghc\ghc-6.10.1 to my path. That lets it find gcc, but then it can't find cc1. If I go even further down this road and add c:\ghc\ghc-6.10.1\gcc-lib to my path, it can find cc1, but then I get a stream of errors about not being able to find header files, such as HsFFI.h.
Does anybody have any clues here to help me? I can provide code, or even access to the machine, if needed.
So here's the story:
Up until ghc-6.10, ghc shipped a fork of hsc2hs. The forked version used ghc as the C compiler rather than gcc. This had the advantage that it would know about ghc packages and the include files shipped with packages. The disadvantage was that it was a fork and behaved differently than using hsc2hs with hugs. It also meant that real C compiler options had to be double-escaped to get hsc2hs to pass them to ghc and get ghc to pass them to gcc.
In ghc-6.10, ghc ships the normal hsc2hs. What we did not notice before 6.10.1 was released was that calling hsc2hs on its own now does not work without passing additional flags. We never noticed because few people actually do call hsc2hs directly, it is typically used via Cabal. Cabal tells it where gcc lives and what include dirs to use.
So that's why at the moment in the Windows installer, hsc2hs cannot even find gcc, because gcc is not on the %PATH% and hsc2hs does not know about ghc's special bundled version of gcc. It also does not know about ghc's include directories.
It's clear we'll need to think a bit more about how to get hsc2hs to be both a compiler neutral tool where we can share a single instance between multiple compilers and also have it work "out of the box".
We're also happy to pay for support, if someone out there is offering commercial support for ghc.
We offer commercial support through Well-Typed: http://www.well-typed.com/
Duncan
On Tue, 2008-12-30 at 13:32 +0900, Curt Sampson wrote:
Duncan,
Good to talk to you again. You may remember me from ICFP.
Indeed. :-)
Anyway, thanks for the hint; I can probably make a workaround by adding the appropriate options for hsc2hs to the QAM build system. If you happen to know what those options are, or just where in the source you generate them, that would be a helpful pointer.
Run with -v to see all the options that cabal passes to hsc2hs. The most important ones are: It tells hsc2hs where gcc lives. Additionally on Windows ghc's bundled copy of gcc needs a -B flag to tell gcc where it is installed (and thus where the default include and linker dirs are). It passes -I for all the include dirs of all the packages you're using. Most significantly it does that for the rts and base packages. The rts include dir contains the HsFFI.h header which is essential for hsc2hs because the default hsc2hs template #includes it.
But I'd still very much like to have qam just calling Cabal with appropriate options to build Haskell and related code. The issues, as I recall, were:
I'll get back to you about this in January.
1. Verbosity and repeitiveness of .cabal specification files. This is actually not such a big deal, as I can have qam generate these from a smaller specification that knows about qam's standard places to put things.
A big thing here will be when the 'Simple' build system allows the multiple components in a package to list dependencies on each other, eg an executable that build-depends on the library in the same package.
2. Ability for each developer on a machine to install his own separate copy (or multiple copies) of an entire application system, including libraries, packages, and so on. I think with your --prefix and --package-db options, this will work now, right?
I think so.
I can have two different developers using two different prefixes, sharing a ghc and the base packages from /usr/local, but using their own, say, libgtk+ and haskell interface to it in /home/joe/project/release or whatever?
If you can install Gtk+ per-user then the Haskell bits are easy. The default for cabal-install is per-user installation.
3. Fairly heavily customized file locations. Last we talked, this was the major sticking point. Just to review, in QAM, we have a directory for source and data for some fairly stand-alone library or chunk of the project ("project/src/componentname", with src underneath that), a directory for build intermediate files (subdirs under "build") and a directory in which the entire release is installed ("release", with the usual bin, lib, share, etc. directories under it).
(A detailed introduction to this is available here, by the way: http://www.starling-software.com/en/qam-principles.html
One tricky thing was that we have, for those components that wish it, a single directory in which we build all of the (non-main) object files, and wrappers for runhaskell, ghci, and the like that use the built sources here. This gives us an almost lisp-machine like environment for wandering about the various parts of our system, recompiling only the bits that are needed, experimenting interactively with whatever part we like, including into the deep depths of libraries, and so on, while still running very quickly. Unfortunately, there was some issue with lack of ability to specify where the object and .hs files go, if I recall. Is tehre any progress on this front?
We can specify a directory where the build files go. So the location is configurable but the layout of dirs/files under that location is not under the user's control.
Another trick was not to export our _unitTest function (exporting it causes no end of hassles, as it turned out, when I was first setting up the unit test framework), instead runnning the tests with runhaskell; by using the same mechanism as above, we could interpret the file with the tests themselves, but use compiled versions of everything else, which saved a lot of time.
A last issue was dealing with modules that have a main function but are also used as libraries by other modules (built without specifying a main) function, or was it just the lack of ability to use the -main-is option at all? I can't recall.
We still cannot use the ghc -main-is option. I think the general consensus was that it's not portable and the portable workaround is pretty straightforward.
Anyway, any thoughts on these fronts? I suppose the common thread here is that I use modules for several different purposes (production release, playing with in the interpreter, as or as not the entry point of a program, running unit tests) and I'd like to avoid compiling different versions of each module or, even worse, different versions of all modules for every single use of one particular module.
One approach is sharing. Making something an explicit library and having other bits depend on that.
BTW, I'd also be happy to look at changing QAM to try to come a bit closer to the "Cabal way of doing things," if I can be convinced that they're better. But it could well be that, though they're better for building stand-alone haskell programs, they don't have to deal with a lot of the stuff that qam does (loading database and starting servers for full-system functional tests, dealing with having several different versions of PHP and apache on a system, generating appropriate configuration files for those sorts of situations, etc.) and it's not even worth thinking attempting that. (Having built this system over the last four years or so, I can attest that dealing with this stuff is not as trivial as one would hope.)
There are plenty of improvements we could and should make to Cabal to make it a better build system (as opposed to a distribution system which it is passably ok at the moment). It's a matter of thought and development time. Let's discuss this further in the new year. Duncan
participants (2)
-
Curt Sampson -
Duncan Coutts