are you plan to implement 64-bit windows GHC version?
The main thing standing in the way of this is the lack of a 64-
bit port of
mingw.
Why do you need mingw? What's wrong with MSVC++?
The largest problem is the build system: GHC uses autoconf with
custom makefiles. I have looked into porting the whole thing to a
Visual Studio project, using SCons (unreliable), CMake (limited
command line abilities--good for a one-shot build but really just a
"safe" lowest-common-denominator version of Make), Waf (another
python-based build system that started as a fork of SCons for the
KDevelop changeover from Autotools) and Jam. I would prefer to use
Jam but I'm afraid I would be the only one who would ever want to
support it. Nothing has the auto-configuration abilities you (John)
built into the Felix Interscript-based system but I do not porting
the build system (at least) to Interscript would go over well with
anyone else who wanted to maintain it and the build itself would
require heavy customisation. I have tested all of these on a small
scale (the replacement-Integer library). The best option seems to be
to create a VS project (not trivial--lots of glue) so a user may also
call that from Make (if under Mingw) or pure DOS.
There is also some gcc-specific code in the RTS (inline assembler,
use of extern inline, etc.) By the way, as of gcc-4.2 (I believe; I
know it is true for gcc-4.3) the use of 'extern inline' now conforms
strictly to the C99 standard so we will have to add the option '-
fgnu-89-inline' to get the old behaviour back--'extern inline' is
used in some of the headers. Converting those 'extern inline's to
'static inline' or best yet plain 'inline' would also solve the
problem. Ian Taylor's message at http://gcc.gnu.org/ml/gcc/2006-11/
msg00006.html describes this in greater detail; his proposal was
implemented.
I don't think we'll be able to drop the mingw route either, mainly
because while
the MS tools are free to download, they're not properly "free", and
we want to
retain the ability to have a completely free distribution with no
dependencies.
I don't know of any completely free 64-bit compilers for Windows.
The Intel compilers are free for 30-day evaluation but everything
else is for Win32. For the base Win32-native port there are many
compilers available but I have mostly worked on using CL and Yasm
(assembler) as replacement back-end compilers for GHC.
There are people that want a Cygwin port too; personally I think
this is heading
in the wrong direction, we want to be more "native" on Windows,
using the native
object format and interoperating directly with the native Windows
tools.
Cygwin has a real problem with gcc: it is far behind everything else
(gcc-3.4.4, though Mingw isn't much better) and it doesn't look like
that will change anytime soon. It is also only 32-bit, I believe.
Cheers,
Pete
I have looked into porting the whole thing to a
Visual Studio project, using SCons (unreliable), CMake (limited
command line abilities--good for a one-shot build but really just a
"safe" lowest-common-denominator version of Make), Waf (another
python-based build system that started as a fork of SCons for the
KDevelop changeover from Autotools) and Jam. I would prefer to use
Jam but I'm afraid I would be the only one who would ever want to
support it. Nothing has the auto-configuration abilities you (John)
built into the Felix Interscript-based system but I do not porting
the build system (at least) to Interscript would go over well with
anyone else who wanted to maintain it and the build itself would
require heavy customisation.
The Felix build system is more or less independent of Interscript.
There's NO WAY GHC should be ported to use Interscript.
We don't want to touch any of the source files.
For information of other readers: Felix uses two pieces of advanced
technology for building.
1. Interscript is a general purpose extensible literate programming (LP)
tool. The idea of LP is that code is embedded in documents. Interscript
documents are *.pak files, which when 'tangled' create the actual
sources. Interscript is different to other LP tools in that
you can embed arbitrary Python script inside a document and
use it to *generate* code (or documentation).
This just wipes out rubbish like autotools method of filling
in Makefile.am templates because it is (a) programmatic
and (b) applies to all sources the same way. I regularly use
tables to generate various parts of a program, eg a list of
tokens to generate lexing components as well as a pretty
printing function.
But LP is an invasive technology. You pay for using it:
it pervades the whole software base and it typically defeats
IDE's and syntax colouring etc.
2. The Felix build system is basically a 'debian like' package
manager. Although it hooks interscript, that's just another plugin
of the build system.
The key thing for the building portability is that the C and C++
compilers are represented by Python classes. There is a pre-programmed
class for gcc, and another for MSVC++. The way this works is we have
identified build abstractions like:
* make an object file for static linking
* make an object file for dynamic linking (-fPIC thing)
* make a dynamic link library from object files
* make a static link library from object files
* static link a program
* link a program for dynamic loading
plus some things peculiar to Felix. Each of these functionalities
is represented by a method of the Python class.
So the build scripts are portable, provided you use these methods
on an object of the appropriate compiler class (gcc or msvc).
Similarly, to manage files, scripts say stuff like:
fileparts = string.split(filename,"/")
osfilename = string.join(fileparts,os.sep)
which converts a 'unix' filename to your local OS convention.
I typically mandate Unix style filename literals even on Windows,
but it is tricky to get this right.
To build a library a package typically has a meta-description,
which is itself an executable Python script which is requires
to set some variables, such as a list of source files to
be compiled. The build system compiles them using both
the static and dynamic methods, and makes both shared and
static archive libraries.
Clearly GHC will have different requirements to Felix.
I'm not suggesting copying the Felix build system verbatim!
What I actually recommend is:
(a) Pick a portable scripting language which is readily available
on all platforms. I chose Python. Perl would also do. I can't
recommend Tcl, it's too messy. Scheme may be an excellent choice
I'm only just learning it and I'm not sure about availability,
but it seems really well suited if you can hack all the brackets :)
(b) Write the WHOLE build system using that language.
For parts that differ between OS, you use parameters.
These parameters can be simple things like
EXE_EXT = ".exe" # Windows
EXE_EXT = "" # Unix
etc, or they can be classes encapsulating complex behaviour.
Too much abstraction is bad .. environments are too quirky,
lots of cases with minor variations is the way to go unless
you're REALLY sure what you have is an abstraction.
(c) provide 'values' for the parameters for the platform
combinations you want to build on
(d) write configuration scripts to create a file of these
parameters -- if that fails the user can edit it. You can
also supply 'prebuilt' configurations for common platforms.
BTW: doing all this was more or less mandatory for Felix,
since it is a cross-cross-compiler. The Felix build system
actually splits building into 'phases' where each phase
can be executed on a distinct machine (with a shared
file system). These phases 'split' all the packaged components
into time slices, and the build process builds the packages
in dependency order for each slice in turn.
This is useful on Windows, eg use Cygwin to build some of the
tools, but use them in native windows shell to build
the next stage.
Since GHC is bootstrapped it will be even messier.. :)
There is also some gcc-specific code in the RTS (inline assembler,
use of extern inline, etc.)
MSVC++ for Vista and beyond does not allow any inline assembler.
This is probably because it wants to target x86, x86_64 and
also ia64 (as well as CLR).. and doesn't have gcc "interesting"
way of managing register usage.
I don't know of any completely free 64-bit compilers for Windows.
I don't know of any completely free 64 bit compilers for Linux.
gcc is GPL which is anything but free of encumberance ..
--
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net
The key thing for the building portability is that the C and C++
compilers are represented by Python classes. There is a pre-programmed
class for gcc, and another for MSVC++.
(a) Pick a portable scripting language which is readily available
on all platforms. I chose Python. Perl would also do.
If I had time to look into improving the GHC build system I'd definitely
use Haskell as the scripting language. What's the point of having to
learn more than one language to accomplish a task? Surely it is more
I suggest (for GHC) a Haskell class with instances for the different
combinations of
compilers and platforms and optimization levels.
productive to add functionality to a single language (especially when
it's only a case of writing some libraries) rather than grappling with
the different syntaxes and limitations and quirks of a multitude of
languages/tools and the problems of trying to get them all to work
harmoniously together on different platforms?
In concrete terms, for each sub-project within GHC ie each directory of
source files I'd have a little Haskell program that was responsible for
building that sub-project. Kind of like CM.make or the ML Basis system,
except with the full power of Haskell so that complicated things such as
building successive boot levels could easily be automated. (Ie instead
of having a special tool like make that is invoked on "dead" data
(supplied by autoconf etc) to drive the build process, instead use
Haskell as a scripting language.)
The top level build would be done by another Haskell program which
delegated responsibility to the sub-programs in each directory.
I'd include the complete source code for all the Haskell tools like Alex
and Happy so they could be built the same way.
In other words, the entire GHC project would be built by running a
single Haskell program, and every single piece of source would either be
a Haskell module or a file that is the input to a Haskell program that
the main build-driver has built first (from Haskell sources). Therefore
there would be no dependency on anything except GHC itself.
To port GHC to a completely new platform, you'd of course need a Haskell
compiler or interpreter already. However to bootstrap the process only a
slow interpreter would be needed so as long as a portable pre-built
bytecode version was available for download the only thing left to port
would be a byte code interpreter which could just be a single file of c
code.
Perhaps this is too long-term a vision for solving the short term
problems, but maybe it could be the basis of some future projects?
Best regards,
Brian.
each sub-project...have a...Haskell program...building that sub-project
I was trying to build something like this recently but hit a roadblock.
Rather than execute the script in each directory, I wanted to import it as a
module instead. This way you can, for example, pass functions, like a
logger, to a function in the imported module. Also, rather than execute the
script immediately, you can combine many sub-projects into a big graph, have
a look at common dependencies and use Concurrent Haskell to parallelize the
build.
Unfortunately, the imported module needs to have the line "module
X.Y.Zwhere", which means the file needs to be aware of its parent
directories. I
think that's too harsh a constraint, and makes it a pain to move things
around (true in everyday Haskell projects with local modules too!).
My plan for a workaround was to try to use a preprocessor, but would really
rather avoid that if anyone else has any ideas.
Thanks,
Greg
(a) Pick a portable scripting language which is readily available
on all platforms. I chose Python. Perl would also do.
If I had time to look into improving the GHC build system I'd definitely
use Haskell as the scripting language.
Two difficulties. The first, obviously, is that this will
only work for building Haskell when you already have Haskell,
so no good for the initial bootstrap of a first port.
The second is simply that dynamic typing is generally
better for build systems, because it allows code to
'self-adapt'. To do this with a statically typed language
you would need to generate text, compile it, and run it,
which is not possible because by specification you don't
have a compiler yet. Even at the point you have bootstrapped
far enough that you do have one .. it's still very messy
to get programlets to communicate using shells in a portable
way.. in some sense that's the problem you're trying to solve!
An alternative is to implement the build system in, say,
Scheme, and then write a Scheme interpreter in Haskell.
Scheme can self-adapt internally because its compiler
is built-in.
This approach removes the dependency on external vendors,
and you might even make the initial bootstrap builder
simple enough you could use a drop-in replacement,
eg Guile (GNU scheme) on unix systems.
--
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net
are you plan to implement 64-bit windows GHC version?
The main thing standing in the way of this is the lack of a 64-bit
port of
mingw.
Why do you need mingw? What's wrong with MSVC++?
The largest problem is the build system: GHC uses autoconf with custom
makefiles.
So autoconf won't work with MSVC++, that is indeed a problem. But this doesn't
mean we have to stop using Makefiles and GNU make - the rest of the build system
will work fine, provided it's told about the different conventions for names of
object files etc. I don't see a compelling enough reason to stop using GNU
make. The build system doesn't even need to invoke CL directly, since we can
use GHC as a "driver" (isn't this the way we agreed to do it before?).
We use autoconf in a pretty limited way (no automake), so I don't think it will
be hard to work around, even if we have to just hard-code all the configuration
results for Windows.
Cheers,
Simon
Why do you need mingw? What's wrong with MSVC++?
The largest problem is the build system: GHC uses autoconf with
custom makefiles.
So autoconf won't work with MSVC++, that is indeed a problem. But
this doesn't mean we have to stop using Makefiles and GNU make -
the rest of the build system will work fine, provided it's told
about the different conventions for names of object files etc. I
don't see a compelling enough reason to stop using GNU make. The
build system doesn't even need to invoke CL directly, since we can
use GHC as a "driver" (isn't this the way we agreed to do it before?).
We use autoconf in a pretty limited way (no automake), so I don't
think it will be hard to work around, even if we have to just hard-
code all the configuration results for Windows.
The make system does work well and must be kept in order to port GHC
to a new posix platform--too many parallel projects (pun intended)
work with the current system. I have not kept a good count of
monthly configuration-based bugs but there are at least a few a
month, for known platforms, including OS X (a significant user base)
and Mingw. If I could change one feature of the current system I
would set up a wiki page with specific build requirements (I mean
location, program/library with function declaration), and for known
systems use autoconf only to determine what the $(build) system is
and to ensure those programs are available, then jump into make which
would call pre-set configuration makefiles for that system.
I spent a good amount of time writing the replacement library build
system in GNU Make (min. 3.8--the current min is 3.79.1) to blend
seamlessly with the current system. It does use a custom "configure"
script written in Python (more consistently portable, no temporary
files of any kind in $(srcdir))--John, that is where I originally
used Interscript: to bake configuration settings into the setup
files. The configuration determines what system it is on and the
relative-path location of certain requirements if they are not
already available--for testing the processor type and os support
(when it can't read from something cool like /proc/cpuinfo) it does
build small programs but all building is done in the build directory
which may be located anywhere you want. It then sets those
parameters for configuration files that already contain other presets
for that platform; general guesses may go into the main GHC autoconf
and I will keep them very simple (new architectures get the generic C
library by default). I simply can't convince myself that it is
better to use a guess-based system for architectures I already know,
especially when it also makes cross-compiling more complex than
necessary. For Windows it uses a VS project and calls that from a
DOS-batch file (for setup parameters) so you can run it from the
command line.
What I hope you would agree on for Windows-GHC is a build that ran
parallel to the autoconf-make system. Of course that would require
some maintenance when things change in the main system but I could
write update scripts for trivial changes; I believe anything more
complex should be carefully checked in any case. VS is troublesome
(its project files are written in XML, but that may be automated).
If you would rather use a Make-like system I could do it in Jam and
then you would add only a few extra Jamfiles to the current system.
As a bonus either VS or Jam would reduce build times, especially re-
build times, would and probably reduce the number of configuration
bugs we see around here. I would not suggest CMake, SCons or WAF;
John wisely advised against anything invasive.
Cheers,
Pete
The make system does work well and must be kept in order to port GHC to
a new posix platform--too many parallel projects (pun intended) work
with the current system. I have not kept a good count of monthly
configuration-based bugs but there are at least a few a month, for known
platforms, including OS X (a significant user base) and Mingw. If I
could change one feature of the current system I would set up a wiki
page with specific build requirements (I mean location, program/library
with function declaration), and for known systems use autoconf only to
determine what the $(build) system is and to ensure those programs are
available, then jump into make which would call pre-set configuration
makefiles for that system.
So you'd hard-wire a bunch of things based on the platform name? That sounds
like entirely the wrong approach to me. It makes the build system too brittle
to changes in its environment: exactly the problem that autoconf was designed to
solve.
I spent a good amount of time writing the replacement library build
system in GNU Make (min. 3.8--the current min is 3.79.1) to blend
seamlessly with the current system. It does use a custom "configure"
script written in Python (more consistently portable, no temporary files
of any kind in $(srcdir))--John, that is where I originally used
Interscript: to bake configuration settings into the setup files. The
configuration determines what system it is on and the relative-path
location of certain requirements if they are not already available--for
testing the processor type and os support (when it can't read from
something cool like /proc/cpuinfo) it does build small programs but all
building is done in the build directory which may be located anywhere
you want. It then sets those parameters for configuration files that
already contain other presets for that platform; general guesses may go
into the main GHC autoconf and I will keep them very simple (new
architectures get the generic C library by default). I simply can't
convince myself that it is better to use a guess-based system for
architectures I already know, especially when it also makes
cross-compiling more complex than necessary. For Windows it uses a VS
project and calls that from a DOS-batch file (for setup parameters) so
you can run it from the command line.
Adding a dependency on Python is already something I want to avoid. One way we
try to keep the GHC build system sane is by keeping the external dependencies to
a minimum (yes I know the testsuite requires Python, but the build itself doesn't).
However, I admit I don't fully understand the problem you're trying to solve,
not having tried to do this myself. The GHC build system now uses Cabal to
build libraries (actually Cabal + make + a bit of autoconf for some libraries).
Why can't this method work for building libraries on Windows native? We must
port Cabal to Windows native anyway, and then you have a library build system.
Of course that would require some
maintenance when things change in the main system but I could write
update scripts for trivial changes; I believe anything more complex
should be carefully checked in any case. VS is troublesome (its project
files are written in XML, but that may be automated). If you would
rather use a Make-like system I could do it in Jam and then you would
add only a few extra Jamfiles to the current system.
If we were to use something other than GNU make, we should do it wholesale, or
not at all, IMO.
Cheers,
Simon
If I could change one feature of the current system ... for known
systems use autoconf only to determine what the $(build) system is
and to ensure those programs are available, then jump into make
which would call pre-set configuration makefiles for that system.
So you'd hard-wire a bunch of things based on the platform name?
That sounds like entirely the wrong approach to me. It makes the
build system too brittle to changes in its environment: exactly the
problem that autoconf was designed to solve.
Maybe this depends on the type of convenience you want to offer GHC-
developers. With the autoconf system they are required (for Windows)
to download and install: Mingw, perl, python (for the testsuite),
flex, happy, alex and some others I can't remember right now. Oh
yeah, GMP. When I did that, autoconf gave me the convenience of
having those programs somewhere in my $PATH and if autoconf found
them I was good to go. The way I would change things would not be so
different, except that the developer would have to set up the build
environment: run from under Mingw, etc. The benefit would be that
instead of testing all the extra things autoconf tests for--path
separator, shell variables, big/little endian, the type of ld and the
commands to execute compile--those would be hard-wired because the
system is known. When things break down you don't have to search
through the long lines of output because you know what the initial
settings are and can even rely on them to help debug Make. This is
the way it is done for several systems: NextStep (now Apple) project
makefiles, Jam, and many of the recent build systems, including
CMake, Scons and WAF.
Just last week I got an email from someone who had downloaded Cygwin
and installed GMP but ran into problems compiling GHC because
autoconf couldn't find the installed-GMP and tried to build the old
GMP located in the rts directory instead--note: it fails under Cygwin
for GMP_CHECK_ASM_W32, an autoconf macro. Clear directions on where
to put things would have solved this, or a build system that had
better search capabilities--but I see your point about changing
things too much (there is a very delicate balance to things as they
are).
It does use a custom "configure" script written in Python (more
consistently portable, no temporary files of any kind in $
(srcdir)) ...
Adding a dependency on Python is already something I want to
avoid. One way we try to keep the GHC build system sane is by
keeping the external dependencies to a minimum (yes I know the
testsuite requires Python, but the build itself doesn't).
I fought with myself about this for awhile--the end result will
integrate into the main GHC configure.ac (as it should) but it will
have to be much simpler. I initially had a much more complex
autoconf-based system but that created temporary files in $(srcdir)
and it would not port well to other autoconf systems--even including
extra .m4 code for the tests (I ran into trouble running autoreconf
and even using a simple shell script that would run aclocal -Im4,
autoheader, automake -a, and autoconf separately, when porting from
autoconf 2.59 (OS X) to autoconf 2.61 (Ubuntu Linux) and autoconf
2.52 (Mingw)--the point being that a developer would be able to run
autoreconf and get the same thing back again). The python script
works well on all systems I have tested it on and I wanted to use
python to help run the C-based testing framework and handle auto-
tuning. I can put all that into a shell script but for Windows it
should be a DOS batch file, at least.
Windows developers should not be forced to run everything under Mingw
or Cygwin. There are two problems with $(srcdir): someone who wishes
to work out of a darcs repository will have to do a full copy of the
source directory if they don't have the ability to create a shadow
copy; and, those environments are themselves quite brittle--depending
on how you invoke Mingw, you may have a completely different $PATH
setup and for a Windows-native build you would have to manually add
the Windows tools to your path (calling vsvars32.bat, the file that
sets up the path for the MS Tools, from under Mingw doesn't work--
maybe I've forgotten something).
However, I admit I don't fully understand the problem you're trying
to solve, not having tried to do this myself. The GHC build system
now uses Cabal to build libraries (actually Cabal + make + a bit of
autoconf for some libraries). Why can't this method work for
building libraries on Windows native? We must port Cabal to
Windows native anyway, and then you have a library build system.
It seems everyone has been looking for a replacement to autoconf for
Cabal for quite some time now. John suggested using a complete
scripting language to do it and I had toyed with that prospect a
bit. A basic requirement would be that it is already available on
the majority of architectures where GHC has not been ported. A more
practical requirement would be that it has a definite syntax for all
architectures--no special shell quirks. Things that fit this bill
are JVM-based applications and Python--JVM being more portable. I
have to confess, I hate Java and I consider the language itself
plebian, or at least parochial. Two systems that would do better
with this are Jaskell (http://jaskell.codehaus.org/) or SISC (the
Scheme-to-Java compiler, at http://sisc-scheme.org/). In any case, a
good project would be a decent replacement for autoconf based on one
of these. Perhaps port Cabal to Jaskell?
I may have digressed there a little. Sorry. In any case, for
Windows-GHC it seems impractical to require developers to have Mingw
installed--the ported compiler will run through native-asm only so
they won't need to have perl installed; only GHC and Yasm and VS
(which is a long download but requires far learning for a Windows
developer). I can fairly easily change the library requirements for
Windows because I am only porting to variations of the Windows platform.
What I hope you would agree on for Windows-GHC is a build that ran
parallel to the autoconf-make system.
What I hope is that we don't have to do this :-)
It's a really big job but it seems more complete in the end. I think
we talked about this before: the cross-compile to get the hc-files
must use the Windows headers, then the hc-build script must be
modified to use the windows toolset. Both of these are quite
invasive and messy to the current system--imagine the makefile code
for DLL-building x10. Once that is done the Windows-GHC will have to
operate as an independent fork: I doubt many people will wish to do
another hc-build of it though it would be a good idea to keep the
files around. What I propose is to start using a different build
system starting with the hc-build script. Whatever the end result
is, GHC must be able to operate without Mingw and the GNU toolset.
If we were to use something other than GNU make, we should do it
wholesale, or not at all, IMO.
Yeah. Otherwise one system sits around and gets a few too many bugs
for anyone to want to fix it up again, then it rots away, leaving a
great many maggot-files of forgotten purpose scattered through your
source code. I spent quite some time searching, learning about and
using different build systems. John is right: aside from tracking
dependancies, it would also be better to maintain the configuration
and build system in a single language, as well. The problem of build
systems seems to be reaching a head outside these walls--I am
beginning to see stuff on this all over the place. I suggested a
special build for Windows because Windows is a "special" system--
practically everything else (I haven't heard anyone talk of porting
GHC to OS/2) is posix-based. There is only one special system to
maintain independently of posix and using, say, VS tools you will
have a ready supply of experienced programmers who would be grateful
for the opportunity to work with something familiar.
Cheers,
Pete
So you'd hard-wire a bunch of things based on the platform name?
That sounds like entirely the wrong approach to me.
FYI: there is a rather nice set of platform data in the
ACE package.
--
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net
Maybe this depends on the type of convenience you want to offer
GHC-developers. With the autoconf system they are required (for
Windows) to download and install: Mingw, perl, python (for the
testsuite), flex, happy, alex and some others I can't remember right
now. Oh yeah, GMP.
In fact, to build a source distribution on Windows, there are only 3
dependencies: GHC, Mingw and (either MSYS or Cygwin).
To build from darcs, you also need: darcs, Happy, and Alex. To build docs, you
also need Haddock. To run the testsuite you need Python.
When I did that, autoconf gave me the convenience
of having those programs somewhere in my $PATH and if autoconf found
them I was good to go. The way I would change things would not be so
different, except that the developer would have to set up the build
environment: run from under Mingw, etc. The benefit would be that
instead of testing all the extra things autoconf tests for--path
separator, shell variables, big/little endian, the type of ld and the
commands to execute compile--those would be hard-wired because the
system is known. When things break down you don't have to search
through the long lines of output because you know what the initial
settings are and can even rely on them to help debug Make.
This is the
way it is done for several systems: NextStep (now Apple) project
makefiles, Jam, and many of the recent build systems, including CMake,
Scons and WAF.
Ok, you clearly have looked at a lot more build systems than I have. So you
think there's a shift from autoconf-style "figure out the configuration by
running tests" to having a database of configuration settings for various
platforms? I'm surprised - I thought conventional wisdom was that you should
write your build system to be as independent as possible from the name of the
build platform, so that the system is less sensitive to changes in its
environment, and easier to port. I can see how wiring-in the parameters can
make the system more concrete, transparent and predictable, and perhaps that
makes it easier to manage. It's hard to predict whether this would improve our
situation without actually doing it, though - it all comes down to the details.
On the other hand, we do hard-wire a lot of knowledge about Windows rather than
autoconfing it. This works because Windows is a fixed point; in contrast every
Linux system is different in various ways. So I guess I don't find it
problematic to wire-in what happens on Windows, but I do try to avoid it where
possible.
Getting back to the Windows native port, I'm pretty sure you're making more of a
meal of this than necessary. There's no need to port via HC files, unless I'm
missing something.
Whatever the end result is, GHC must be able to operate without Mingw
and the GNU toolset.
That's the whole point of doing the port!
However, what I'm saying is that we can continue to use Cygwin as a set of tools
for doing the build. I don't see any problems with this (except that Cygwin is
slow and clunky), and it keeps the changes to the current system to a minimum,
and means we can continue to share the build system with Posixy systems. Here's
the plan I had in mind:
1. modify GHC so that:
a) it can invoke CL instead of gcc to compile C files
b) its native code generator can be used to create native .obj files,
I think you kept the syntax the same and used YASM, the other
alternative is to generate Intel/MS syntax and use MASM.
c) it can link a binary using the MS linker
2. modify Cabal so that it can use this GHC, and MS tools
3. modify the build system where necessary to know about .obj .lib etc.
4. modify the core packages to use Win32 calls only (no mingw)
5. Use the stage 1 GHC to compile the RTS and libraries
6. Build a stage 2 compiler: it will be a native binary
7. Build a binary distribution
Regarding autoconf, for the time being, just supply ready-made output files
(mk/config.h, libraries/base/include/HsBaseConfig.h, etc.).
Cheers,
Simon
Ok, you clearly have looked at a lot more build systems than I have. So you
think there's a shift from autoconf-style "figure out the configuration by
running tests" to having a database of configuration settings for various
platforms? I'm surprised - I thought conventional wisdom was that you should
write your build system to be as independent as possible from the name of the
build platform, so that the system is less sensitive to changes in its
environment, and easier to port. I can see how wiring-in the parameters can
make the system more concrete, transparent and predictable, and perhaps that
makes it easier to manage. It's hard to predict whether this would improve our
situation without actually doing it, though - it all comes down to the details.
This misses the point. The 'suck it and see' idea fails totally for
cross-compilation. It's a special case.
The right way to do things is to separate the steps:
(a) make a configuration
(b) select a configuration
logically. This is particularly important for developers who are using
the same code base to build for multiple 'platforms' on the
same machine.
With the above design you can have your cake and eat it too .. :)
Thats the easy part.. the HARD part is: every 'system' comes with
optional 'add-on' facilities. These add-ons may need configuration
data. Often, you want to add the 'add-on' after the system is built.
So integrating the configuration data is an issue.
Felix build system allows add-on packages to have their own
configuration model. It happens to be executed on the fly,
and typically does your usual suck it and see testing
(eg .. where are the SDL headers? Hmm ..)
This design is wrong of course ;(
--
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net
Ok, you clearly have looked at a lot more build systems than I have. So you
think there's a shift from autoconf-style "figure out the configuration by
running tests" to having a database of configuration settings for various
platforms? I'm surprised - I thought conventional wisdom was that you should
write your build system to be as independent as possible from the name of the
build platform, so that the system is less sensitive to changes in its
environment, and easier to port. I can see how wiring-in the parameters can
make the system more concrete, transparent and predictable, and perhaps that
makes it easier to manage. It's hard to predict whether this would improve our
situation without actually doing it, though - it all comes down to the details.
This misses the point. The 'suck it and see' idea fails totally for
cross-compilation. It's a special case.
The right way to do things is to separate the steps:
(a) make a configuration
(b) select a configuration
logically.
Hmm, I don't see how the approach "fails totally" for cross-compilation. You
simply have to create the configuration on the target machine, which is exactly
what we do when we cross-compile GHC. Admittedly the process is a bit ad-hoc,
but it works.
Cheers,
Simon
This misses the point. The 'suck it and see' idea fails totally for
cross-compilation. It's a special case.
The right way to do things is to separate the steps:
(a) make a configuration
(b) select a configuration
logically.
Hmm, I don't see how the approach "fails totally" for cross-compilation. You
simply have to create the configuration on the target machine, which is exactly
what we do when we cross-compile GHC. Admittedly the process is a bit ad-hoc,
but it works.
But that consists of:
(a) make a configuration (on the target machine)
(b) select that configuration (on the host machine)
which is actually the model I suggest. To be more precise, the
idea is a 'database' of configurations, and building by selecting
one from that database as the parameter to the build process.
The database would perhaps consists of
(a) definitions for common architectures
(b) personalised definitions
You would need a tool to copy and edit existing definitions
(eg .. a text editor) and a tool to 'autogenerate' prototype
definitions (autoconf for example).
What I meant failed utterly was simply building the sole
configuration by inspection of the properties of a the
host machine (the one you will actually build on).
That does work if
(a) the auto-dectect build scripts are smart and
(b) the host and target machines are the same
BTW: Felix has a 4 platform build model:
* build
* host
* target
* run
The build machine is the one you build on. Example: Debian
autobuilder.
The host is the one you intend to translate Felix code to
C++ code on, typically your workstation. In Windows environment
this might be Cygwin.
The target is the one you actually compile the C++ code on.
In Windows environment, this might be WIN32 native (MSVC++).
The run machine is where you actually execute the code.
The 'extra' step here is because it is a two stage compiler.
Some code has to be built twice: for example the GLR parser
elkhound executable runs on the host machine to generate
C++ and it uses a library. The same library is required
at run time, but has to be recompiled for the target.
EG: Elkhound built on cygwin to translate grammar to C++,
and Elkhound on MSVC++ for the run time automaton.
I'm not sure GHC as such need cross-cross compilation model,
but bootstrapping a cross compiler version almost certainly does.
--
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net
Ok, you clearly have looked at a lot more build systems than I
have. So you think there's a shift from autoconf-style "figure
out the configuration by running tests" to having a database of
configuration settings for various platforms?
I shouldn't overstate the situation: the other "complete" build
systems, CMake and SCons do have autoconf capabilities in the way of
finding headers and programs and checking test-compiles, the basic
sanity checks--CMake has many more autoconf-like checks than SCons.
Where they differ from the automake system seems to be their setup,
which like Make has hard-coded settings for compilers, linkers, etc.
(Some standard cmake settings are wrong for certain targets.) I
don't know if you have any interest in pursuing or evaluating CMake
(certainly not now) but the "standard" setup is stored in a standard
directory on each platform, say, /usr/share/cmake-2.4/Modules/
Platform/$(platform).cmake and may be overridden by your own cmake
file in, say, $(srcdir)/cmake/UserOverride.cmake.
The preset-target-configuration build model I was referring to is a
scaled-down version of the commercial practice which allows you to
have a single system and simultaneously compile for many different
architecture-platform combinations--once you have tested each and
know how everything works. For the initial exploration, it is a
different (more anal) strategy: before invading, get all the
intelligence you can and prepare thoroughly. The GNU-Autoconf
strategy is to keep a few troops who have already invaded many other
places, adjust their all-purpose equipment a little for the mission
and let them have at it. My gripe is that their equipment isn't very
good.
Cheers,
Pete
In fact, to build a source distribution on Windows, there are only
3 dependencies: GHC, Mingw and (either MSYS or Cygwin).
To build from darcs, you also need: darcs, Happy, and Alex. To
build docs, you also need Haddock. To run the testsuite you need
Python.
True, Mingw does come standard with perl and a version of flex.
There are Windows-native versions of Perl and flex available (i.e.,
ActivePerl). Now you are familiar with Mingw. Imagine being a
standard Windows programmer, trying to choose which version of Mingw
to download--some are minimal installations--and going over the build
requirements: perl, flex, happy, alex and, haddock are listed. That
is quite a bit of preparation. There are minimal-effort ways to go
about this (I will look into updating the wiki.)
b) its native code generator can be used to create native .obj
files,
I think you kept the syntax the same and used YASM, the other
alternative is to generate Intel/MS syntax and use MASM.
This is as easy as simply using Yasm--also mostly done (not
completely tested). By the way, by "testing" I mean doing more than
a simple -optc... -optc... -optl... addition to the command line,
although an initial build using a current mingw version of GHC may
certainly do this.
c) it can link a binary using the MS linker
2. modify Cabal so that it can use this GHC, and MS tools
3. modify the build system where necessary to know about .obj .lib
etc.
A bit invasive (it involves modifying the make rules so they take an
object-suffix variable). Instead of the current suffix.mk:
$(odir_)%.$(way_)o : %.hc
it should be:
$(odir_)%.$(way_)$(obj_sfx) : %.hc
or some such. This may affect other builds, especially if for some
reason autoconf can't determine the object-suffix for a platform,
which is one reason I suggested a platform-specific settings file. I
could handle this by having autoconf set the target variable, put all
the windows-specific settings in a "settings.mk" file (including a
suffix.mk copy) and have make include that file.
4. modify the core packages to use Win32 calls only (no mingw)
That is where a lot of preparation is going. This is *much* harder
to do from mingw than from VS tools since you have to set up all the
paths manually.
5. Use the stage 1 GHC to compile the RTS and libraries
6. Build a stage 2 compiler: it will be a native binary
7. Build a binary distribution
I told Torkil I would have a version of the replacement library
available for him as soon as possible. I'll shut up now. It looks
like a long weekend.
Cheers,
Pete
A bit invasive (it involves modifying the make rules so they take an
object-suffix variable). Instead of the current suffix.mk:
$(odir_)%.$(way_)o : %.hc
it should be:
$(odir_)%.$(way_)$(obj_sfx) : %.hc
or some such. This may affect other builds, especially if for some
reason autoconf can't determine the object-suffix for a platform, which
is one reason I suggested a platform-specific settings file. I could
handle this by having autoconf set the target variable, put all the
windows-specific settings in a "settings.mk" file (including a suffix.mk
copy) and have make include that file.
Surely this isn't hard?
ifeq "$(TargetOS)" "windows"
osuf=obj
else
osuf=o
endif
and then use $(osuf) wherever necessary.
4. modify the core packages to use Win32 calls only (no mingw)
That is where a lot of preparation is going. This is *much* harder to
do from mingw than from VS tools since you have to set up all the paths
manually.
I don't understand the last sentence - what paths? Perhaps I wasn't clear here:
I'm talking about the foreign calls made by the base package and the other core
packages; we can't call any functions provided by the mingw C runtime, we can
only call Win32 functions. Similarly for the RTS. I have no idea how much
needs to change here, but I hope not much.
A bit invasive (it involves modifying the make rules so they take
an object-suffix variable). Instead of the current suffix.mk:
$(odir_)%.$(way_)o : %.hc
it should be:
$(odir_)%.$(way_)$(obj_sfx) : %.hc
or some such. This may affect other builds, especially if for
some reason autoconf can't determine the object-suffix for a
platform, which is one reason I suggested a platform-specific
settings file. I could handle this by having autoconf set the
target variable, put all the windows-specific settings in a
"settings.mk" file (including a suffix.mk copy) and have make
include that file.
Yes it is easy but now all Makefiles must be changed to use $(osuf),
such as this line in rts/Makefile:
378: %.$(way_)o : %.cmm $(H_FILES),
for what will be a (hopefully) temporary Windows build.
4. modify the core packages to use Win32 calls only (no mingw)
That is where a lot of preparation is going. This is *much*
harder to do from mingw than from VS tools since you have to set
up all the paths manually.
I don't understand the last sentence - what paths? Perhaps I
wasn't clear here: I'm talking about the foreign calls made by the
base package and the other core packages; we can't call any
functions provided by the mingw C runtime, we can only call Win32
functions. Similarly for the RTS. I have no idea how much needs
to change here, but I hope not much.
To use the MS tools with the standard C libraries and include
directories, I must either gather the environment variables
separately and pass them to cl/link on the command line or I must
manually add them to my system environment (i.e., modify msys.bat, or
the windows environment) so msys will use them in its environment.
The other problem is the old no-pathnames-with-spaces in Make, since
that must be made to quote all those environment variables when
passing them to cl. I could use the Make-trick of filling the spaces
with a character and removing that just before quoting but that is a
real hack and not very reliable--it breaks $(word ...).
Altogether it is a pain to get going and barely reproducible. That
is why I suggested simply producing .hc files and building from .hc
using VS.
Cheers,
Pete
A bit invasive (it involves modifying the make rules so they take an
object-suffix variable). Instead of the current suffix.mk:
$(odir_)%.$(way_)o : %.hc
it should be:
$(odir_)%.$(way_)$(obj_sfx) : %.hc
or some such. This may affect other builds, especially if for some
reason autoconf can't determine the object-suffix for a platform,
which is one reason I suggested a platform-specific settings file. I
could handle this by having autoconf set the target variable, put all
the windows-specific settings in a "settings.mk" file (including a
suffix.mk copy) and have make include that file.
Yes it is easy but now all Makefiles must be changed to use $(osuf),
such as this line in rts/Makefile:
378: %.$(way_)o : %.cmm $(H_FILES),
for what will be a (hopefully) temporary Windows build.
I bet there are only a few makefiles that explicitly refer to "o" as the
object-file suffix.
I don't understand why you see this as a temporary measure. Surely we'll need a
way to build GHC again for this platform? Unless you intend to replace the
whole build system? (which I strongly recommend *not* doing, at least not yet)
4. modify the core packages to use Win32 calls only (no mingw)
That is where a lot of preparation is going. This is *much* harder
to do from mingw than from VS tools since you have to set up all the
paths manually.
I don't understand the last sentence - what paths? Perhaps I wasn't
clear here: I'm talking about the foreign calls made by the base
package and the other core packages; we can't call any functions
provided by the mingw C runtime, we can only call Win32 functions.
Similarly for the RTS. I have no idea how much needs to change here,
but I hope not much.
To use the MS tools with the standard C libraries and include
directories, I must either gather the environment variables separately
and pass them to cl/link on the command line or I must manually add them
to my system environment (i.e., modify msys.bat, or the windows
environment) so msys will use them in its environment.
The other problem is the old no-pathnames-with-spaces in Make, since
that must be made to quote all those environment variables when passing
them to cl. I could use the Make-trick of filling the spaces with a
character and removing that just before quoting but that is a real hack
and not very reliable--it breaks $(word ...).
Use GHC as your C compiler, i.e. don't invoke CL directory from make, and add
the INCLUDE/LIB directories to the RTS's package.conf.
Yes it is easy but now all Makefiles must be changed to use $
(osuf), such as this line in rts/Makefile:
378: %.$(way_)o : %.cmm $(H_FILES),
for what will be a (hopefully) temporary Windows build.
I bet there are only a few makefiles that explicitly refer to "o"
as the object-file suffix.
After poking around I found that my fears were unfounded. Simply
pass cl the /TC (or -TC) option--same as the gcc option '-x c'.
Object files are also fine since cl assumes any file with an
unrecognised suffix is an object file.
The environment variables problem is also solved: either have the
environment set up automatically by placing a batch-script 'call' to
the MS PSDK 'SetEnv.Cmd' before the shell login in msys.bat or start
the DOS shell from the MS PSDK shortcut and log into the msys shell
manually--or run the whole thing from DOS. Shows how much I know of
msys. Passing flags to cl would be best in a command file (at least
I have done _that_ before).
I don't understand why you see this as a temporary measure. Surely
we'll need a way to build GHC again for this platform? Unless you
intend to replace the whole build system? (which I strongly
recommend *not* doing, at least not yet)
I keep on referring to this as temporary because there are two
different builds here:
(1) the build using the old mingw-GHC, without option support for CL;
and,
(2) the build using the new Windows-native GHC.
You will almost certainly keep mingw-GHC around but users should not
have to download a mingw-GHC to build Windows-native from source
(they can't start at a stage1 build), so the Windows-native requires
a separate setup. That might as well be Windows-native itself, in
other words, use VS--it is the quickest and easiest build to put
together. I do not suggest CMake because CMake is a sledgehammer
when it comes to managing projects and sub-projects: all paths are
absolute (you cannot move the source directories around), there is
only one major "Project" in a system--it only really builds 'all',
not sub-targets and build variants beyond the buitin Debug,
MinSizeRel, Release, etc., have to be custom-added; it would not
integrate well with the current $(way) system. If you are heavily
against using VS, maybe an Eclipse/Ant-based build would do. I might
use Bakefile.
It would be much better to have a single build system. I would
gladly replace the whole thing for three reasons:
(1) it is a source of many build bugs and it makes them much more
difficult to track down; and,
(2) it seems to be a serious hurdle for anyone who wants to build and
hack on GHC--this is true for most other compiler systems that use
the autoconf and Make; and,
(3) if GHC is ever going to have cross-compilation abilities itself,
the current build system must go, while cross-compiling GHC with the
current system requires access to the actual host-system hardware.
The reasons I don't are:
(1) time (parallel to money);
(2) I wouldn't undertake such an effort unless we were all pretty
sure what you want to change the build system to;
(3) an inevitable side-effect of the move would be loss of old (or
little-used) build settings, such as GranSim, and a change to the
build system would propagate to parallel projects; and,
(4) it is a huge project: both the compiler and libraries must change
and the change must integrate with the Cabal system.
Work on the mingw-make system is progressing fairly well.
The reason to make a special VS build are:
(1) Windows programmer familiarity;
(2) reduction in the number of build bugs;
(3) ease of extension or integration with other VS tools, such
as .NET; and,
(4) speed--VS builds are much faster than Make.
I should also add that when building the RTS it is simply much easier
to have a build problem reported in VS than search back through Make-
output and manually go to the offending line in a source file.
The reason not to make a special VS build is you would have to
support it--one more thing to check when new source files are added.
As I said before, this may be scripted and if Windows programmers
have something familiar to work with there may be more of them to
help. (You probably have better reasons than that one.)
Use GHC as your C compiler, i.e. don't invoke CL directory from
make, and add the INCLUDE/LIB directories to the RTS's package.conf.
Certainly doable but it does present a conundrum: for the old GHC
(without builtin cl-support) the order for compilation seems to be:
<output> <other flags>
while for cl running link.exe or link.exe, it is better to put all
the files at the end of the command line:
<output> <other flags>
It also adds one more layer of indirection a that delicate stage.
I am in the process of modifying and testing changes to some of the
build rules for cl. As an added bonus, unless you want to keep debug
information in each object file (instead of a separate .pdb file), by
putting compile options and sources into a compile/link file we:
(1) avoid problems of exceeding the command line; and,
(2) have the option to use /Gm, which does automatic dependancy
tracking for us.
While I am at it, back in November 2006 I mentioned removing the old
mingw/cygwin-dll stuff. I can leave that in for now until Windows-
native is done. I keep coming across greencard stuff and honestly
that is annoying since greencard hasn't been supported since GHC
6.2. I would really like to remove it entirely but I don't know
whether you want to keep it in.
Altogether it is a pain to get going and barely reproducible.
That is why I suggested simply producing .hc files and building
from .hc using VS.
Doing an unregisterised build, you mean? Sounds a bit scary!
It would be difficult but it would completely remove the mingw-ghc
from the build process and it would stress-test the resulting cl-
compiled haskell code.
Cheers,
Pete
(1) it is a source of many build bugs and it makes them much more
difficult to track down; and,
(2) it seems to be a serious hurdle for anyone who wants to build and
hack on GHC--this is true for most other compiler systems that use
the autoconf and Make; and,
(3) if GHC is ever going to have cross-compilation abilities itself,
the current build system must go, while cross-compiling GHC with the
current system requires access to the actual host-system hardware.
The reasons I don't are:
(1) time (parallel to money);
(2) I wouldn't undertake such an effort unless we were all pretty
sure what you want to change the build system to;
(3) an inevitable side-effect of the move would be loss of old (or
little-used) build settings, such as GranSim, and a change to the
build system would propagate to parallel projects; and,
(4) it is a huge project: both the compiler and libraries must change
and the change must integrate with the Cabal system.
I am thinking of starting a new project (possibly as sourceforge)
to implement a new build system. I think Erick Tryzelaar might
also be interested. The rule would be: it isn't just for GHC.
So any interested people would have to thrash out what to
implement it in, and the overall requirements and design ideas.
My basic idea is that it should be generic and package based,
that is, it does NOT include special purpose tools as might
be required to build, say, Haskell programs: these are
represented by 'plugin' components.
A rough model of this: think Debian package manager, but
for source code not binaries.
--
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net
My basic idea is that it should be generic and package based,
that is, it does NOT include special purpose tools as might
be required to build, say, Haskell programs: these are
represented by 'plugin' components.
It would be much better to have a single build system. I would
gladly replace the whole thing ...
I am thinking of starting a new project (possibly as sourceforge)
to implement a new build system. I think Erick Tryzelaar might
also be interested. The rule would be: it isn't just for GHC.
So any interested people would have to thrash out what to
implement it in, and the overall requirements and design ideas.
My basic idea is that it should be generic and package based,
that is, it does NOT include special purpose tools as might
be required to build, say, Haskell programs: these are
represented by 'plugin' components.
A rough model of this: think Debian package manager, but
for source code not binaries.
I have been considering the same thing for some time, partly because
the specification properties of most available build systems are
terrible: XML is not a language (it is better as a back-end for a gui-
based build system); current plug-in systems (similar to what WAF
uses) are object-oriented and require a deep knowledge of the build
system API; others are manual hacks. One big thing to avoid are
cache-files: CMake, SCons, WAF, autotools, all use cache-files and
all run into problems when the cache files aren't cleaned or include
errors. (WAF has the best cache-file system--they are designed to be
human-readable.) I will gladly lend support to this.
An idea I have been kicking around is a hybrid between the autoconf
strategy and the commercial-setup strategy: add support for a
specification of program requirements, i.e., , gcc/cl/icl,
things like that in a simple spec-document with dead-simple syntax,
then let the build system handle the rest--it would know what to do
for each architecture. That seems similar to the Debian package
maker, right?
What language are you thinking about using? Scheme seems good but
the build-language itself might be different; gui-support should be
available which says to me (horrors!) Java AWT--a cross-platform gui-
supported build system (not an IDE) would rock the world because it
doesn't exist. There are tons of Python packages out there (A-A-P,
hasn't been updated since 2003; SCons, WAF, Bakefile (uses Python)).
I don't know if this is possible in Felix.
Other requirements might be:
(1) never alter anything in the source directories--everything is
managed through the build directory
(2) ability to easily build separate targets from the command line,
similar to 'make test_shifts'--externally scriptable.
One thing other systems seem to fail at is building off the enormous
trove of information in autoconf--it's right there, open source, and
they reinvent the wheel when it comes to making configuration tests
or finding platform information (config.guess is essentially a
database of platform-specific information but is somewhat dated with
regard to newer systems, including OS X). On that note, a different
approach to configuration tests might be clearer knowledge about the
compilers: all these systems build small C programs to test certain
compiler characteristics and test for a 0 exit value. Well, CMake
does actually 'read' the output files for some things, such as
compiler version.
Cheers,
Pete
Certainly doable but it does present a conundrum: for the old GHC
(without builtin cl-support) the order for compilation seems to be:
<output> <other flags>
while for cl running link.exe or link.exe, it is better to put all the
files at the end of the command line:
<output> <other flags>
It also adds one more layer of indirection a that delicate stage.
Maybe some gcc mimicing cl wrapper tailored specifically for GHC
building system could help? One more layer of indirection, but could
leave ghc driver relatively intact.
Cheers,
Kyra
Certainly doable but it does present a conundrum: for the old GHC
(without builtin cl-support) the order for compilation seems to be:
<output> <other flags>
while for cl running link.exe or link.exe, it is better to put all
the files at the end of the command line:
<output> <other flags>
It also adds one more layer of indirection a that delicate stage.
Maybe some gcc mimicing cl wrapper tailored specifically for GHC
building system could help? One more layer of indirection, but
could leave ghc driver relatively intact.
That's a good idea! Do you know if or how the mingw-gcc is able to
do that? Does mingw-gcc wrap link.exe? It sounds silly that someone
relatively inexperienced with mingw should be doing this but it
_really_needs doing and no one else seems to want it (besides, from
my perspective, once I get through the build-system drudgery it lets
me handle the fun stuff like adding inline MASM to the RTS, such as
ghc/includes/SMP.h).
Cheers,
Pete
Maybe some gcc mimicing cl wrapper tailored specifically for GHC
building system could help? One more layer of indirection, but
could leave ghc driver relatively intact.
That's a good idea! Do you know if or how the mingw-gcc is able to
do that? Does mingw-gcc wrap link.exe?
There's more to portable building than the build system.
For example, for C code, you need a system of macros to support
void MYLIB_EXTERN f();
where MYLIB_EXTERN can be empty, say __declspec(dllexport)
on Windows when building a DLL, and __declspec(dllimport)
when using it. This is *mandatory*.
The build system controls the command line switches that
turn on "We're building a DLL" flag. A distinct macro is needed
for every DLL.
In Felix, there is another switch which tells the source
if the code is being built for static linkage or not:
some macros change when you're linking symbols statically
compared to using dlsym().. it's messy: the build system
manages that too.
Building Ocaml, you have a choice of native or bytecode,
and there are some differences. Probably many such things
for each and every language and variation of just about
anything .. eg OSX supports two kinds of dynamic libraries.
The point is that a 'Unix' oriented build script probably
can't be adapted: Unix is different to Windows. The best
way to adapt to Windows is to use Cygwin.. if you want
a Windows native system, you have to build in the Windows
way and make Windows choices. A silly example of that
is that (at least in the past) Unix lets you link at
link time against a shared library, whereas Windows
requires to link against a static thunk ..
so building a shared library produces TWO outputs
on Windows.
OTOH, Unix has this woeful habit of naming shared libraries
like libxxx.so.1.2 which really makes a complete mess
of build systems.
What I'm saying is you just can't wrap Windows tools
inside a Unix build script.
You have to write an abstract script, and implement
the abstractions for each platform.
--
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net
Maybe some gcc mimicing cl wrapper tailored specifically for GHC
building system could help? One more layer of indirection, but
could leave ghc driver relatively intact.
That's a good idea! Do you know if or how the mingw-gcc is able to
do that? Does mingw-gcc wrap link.exe?
There's more to portable building than the build system.
For example, for C code, you need a system of macros to support
void MYLIB_EXTERN f();
where MYLIB_EXTERN can be empty, say __declspec(dllexport)
on Windows when building a DLL, and __declspec(dllimport)
when using it. This is *mandatory*.
Of course--one thing I would add to a build system, instead of
compiling little C files and testing the return value to detect some
compiler functionality, is the ability to read builtin macros, say,
by telling the compiler to dump all macros like 'gcc -E -dM' and
then read through the macros.
As for the Windows-native build, I am pretty far long with that but
the idea was to hijack the "gcc" executable with a script that would
convert the gcc arguments to cl arguments. The one thing such a
script would not do is compile everything at once. So far that is
one thing I am adding to the Make system here: since dependancy
generation is good for Haskell files but is not necessary for C files
since I can bunch the C sources together with the compiler flags and
pass them cl all at once in a command file. This should be faster
than Make.
In Felix, there is another switch which tells the source
if the code is being built for static linkage or not:
some macros change when you're linking symbols statically
compared to using dlsym().. it's messy: the build system
manages that too.
Sometimes this is better in header files and change the macros with
defines the build system passes to the c compiler but Felix's system
is much more flexible than that (it builds the source files as
interscript extracts them, right?).
Building Ocaml, you have a choice of native or bytecode,
and there are some differences. Probably many such things
for each and every language and variation of just about
anything .. eg OSX supports two kinds of dynamic libraries.
GHC's interpreter (GHCi) does have to be built. I have not found a
libReadline DLL, but I am sure I can scrounge something--possibly
from Python since they had this same problem back around 2000.
The point is that a 'Unix' oriented build script probably
can't be adapted: Unix is different to Windows. The best
way to adapt to Windows is to use Cygwin.. if you want
a Windows native system, you have to build in the Windows
way and make Windows choices. A silly example of that
is that (at least in the past) Unix lets you link at
link time against a shared library, whereas Windows
requires to link against a static thunk ..
so building a shared library produces TWO outputs
on Windows.
I am building with Mingw because that is better supported by the GHC
build system (Cygwin is somewhat defunct); the end result should
build from source in Visual Studio/Visual C++ Express.
Cheers,
Pete
Maybe some gcc mimicing cl wrapper tailored specifically for GHC
building system could help? One more layer of indirection, but
could leave ghc driver relatively intact.
there is possibility that such driver was already implemented somewhere
i remember old good says when symantec c++ contained such drivers for
emulation of CL and BCC :)
--
Best regards,
Bulat mailto:Bulat.Ziganshin@gmail.com
I keep on referring to this as temporary because there are two different
builds here:
(1) the build using the old mingw-GHC, without option support for CL; and,
(2) the build using the new Windows-native GHC.
Yes. And what I'm suggesting is the following - what I've been suggesting all
along, but we keep getting sidetracked into sub-discussions:
- the initial bootstrap is done by hand. (your (1) above). All you need to
do is build stage 1 using an existing mingw-GHC. The stage 1 that you
build will be capable of creating native objects and binaries. Any hacks
you have to apply to do this part are certainly temporary.
- we adapt the current build system to use the native GHC. I really don't
think this is hard, and it's way quicker than replacing significant chunks
of the build system, as you seem to be suggesting.
So the result is a build system that can build a win-native GHC using another
win-native GHC, but not necessarily build a win-native GHC using a mingw GHC.
I'm not against VS in particular - I'm against duplication. Build systems rot
quickly. By all means discuss a wonderful replacement for the current build
system -- I'm aware that the current system is far from perfect -- but I'm not
at all convinced that it is a necessity for building win-native GHC.
I could be wrong. If I am wrong, then constructing a convincing argument might
be difficult, because it's a matter of details - the cumulative weight of things
you have to hack is too great. So go ahead and use VS or whatever; but please
think very carefully before doing so, because it's not cost-free. We're not
going to drop the mingw port of GHC, and it's hard to keep everything working as
it is. We'll have to import new hackers who understand VS builds, because none
of the current GHC maintainers do!
Use GHC as your C compiler, i.e. don't invoke CL directory from make,
and add the INCLUDE/LIB directories to the RTS's package.conf.
Certainly doable but it does present a conundrum: for the old GHC
(without builtin cl-support) the order for compilation seems to be:
<output> <other flags>
while for cl running link.exe or link.exe, it is better to put all the
files at the end of the command line:
<output> <other flags>
Why is that a "conundrum"? GHC can invoke CL with the arguments in whatever
order it likes. Sorry, but this just seems like a trivial detail to me.
I keep on referring to this as temporary because there are two
different builds here:
(1) the build using the old mingw-GHC, without option support for
CL; and,
(2) the build using the new Windows-native GHC.
Yes. And what I'm suggesting is the following - what I've been
suggesting all along, but we keep getting sidetracked into sub-
discussions:
- we adapt the current build system to use the native GHC. I
really don't
think this is hard, and it's way quicker than replacing
significant chunks
of the build system, as you seem to be suggesting.
I don't have to replace large chunks of the system, although I have
added several separate makefiles--an mk/msvc_tools.mk and mk/
build.mk.msvc (which configure will copy into mk/build.mk). It is
almost done (the current system, I mean)--although I do have one
detail question: Clemens Fruhwirth sent a patch to add shared library
support for all architectures, i.e., MkDLL -> MkDSO (in compiler/main/
DriverPipeline.hs). I haven't seen the patch after I did my last
pull, yesterday. So I assume it has not been applied yet. How do you
want autoconf to detect the shared library extension and libtool
support? AC_PROG_LIBTOOL does not seem to work well on OS X: OS X
libtool is Apple, not GNU (it is also a binary, not a driver-script
for libltdl); that macro failed the last time I built GMP and I had
to make shared libraries manually). This is precient because the
default build for Windows should be DLLs but I want the configuration
(at least) to mesh with the rest of the system: I wanted to add $
(libext) and $(shlibext); as it is, I vary them by a simple case in
*windows), *darwin*) or *) (unix) but this does not seem correct.
So the result is a build system that can build a win-native GHC
using another win-native GHC, but not necessarily build a win-
native GHC using a mingw GHC.
I could set it up so configure could detect which GHC is available
and build using that GHC (Mingw or Windows-native). (Just add a C-
compiler string to 'ghc -v' or 'ghc --version' and grep for it.)
I'm not against VS in particular - I'm against duplication. Build
systems rot quickly. By all means discuss a wonderful replacement
for the current build system -- I'm aware that the current system
is far from perfect -- but I'm not at all convinced that it is a
necessity for building win-native GHC.
VS is not necessary; it is aesthetic and may offer other benefits for
those who wish to hack on GHC. It would require many bits of glue
code and careful alignment of tasks so the entire build would not be
transparent to any but the most experienced VS programmers. It
would, however, be much easier to the more casual developer and it
may not be as brittle: shallow build settings, compiler settings,
source files included, a bureaucratic notion of ways, would be
available from a simple point and click. If I have time I will at
least do a prototype (base-compiler only) and see if people like it.
I could be wrong. If I am wrong, then constructing a convincing
argument might be difficult...
We'll have to import new hackers who understand VS builds, because
none of the current GHC maintainers do!
New blood! :) I'm joking--there have been forks of GHC in the past
but they generally don't last long because GHC moves too fast and
that's because the Architects are still at work. The only convincing
argument here would be a prototype that even the GHC maintainers
would be able to understand.
Certainly doable but it does present a conundrum: for the old GHC
(without builtin cl-support) the order for compilation seems to be:
<output> <other flags>
while for cl running link.exe or link.exe, it is better to put all
the files at the end of the command line:
<output> <other flags>
Why is that a "conundrum"? GHC can invoke CL with the arguments in
whatever order it likes. Sorry, but this just seems like a trivial
detail to me.
Mingw GHC can't do that. I simply added some conditional changes to
the rules in mk/suffix.mk.
Cheers,
Pete
Brian Hulley
Bulat Ziganshin
Gour
Greg Fitzgerald
kyra
Peter Tanski
Simon Marlow
skaller