- Glasgow-haskell-users

parallelizing ghc
by Evan Laforge 27 Feb '12

27 Feb '12

I recently switched from ghc --make to a parallelized build system. I was looking forward to faster builds, and while they are much faster at figuring out what has to be rebuilt (which is most of the time for a small rebuild, since ld dominates), compilation of the whole system is either the same or slightly slower than the single threaded ghc --make version. My guess is that the overhead of starting up lots of individual ghcs, each of which has to read all the .hi files all over again, just about cancels out the parallelism gains. So one way around that would be parallelizing --make, which has been a TODO for a long time. However, I believe that's never going to be satisfactory for a project involving various different languages, because ghc itself is never going to be a general purpose build system. So ghc --make provides two things: a dependency chaser and a way to keep the compiler resident as it compiles new files. Since the dependency chaser will never be as powerful as a real build system, it occurs to me that the only reasonable way forward is to split out the second part, by adding an --interactive flag to ghc. It would then read filenames on stdin, compiling each one in turn, only exiting when it sees EOF. Then a separate program, ghc-fe, can wrap ghc and acts like a drop-in replacement for ghc. It would be nice if ghc could atomically read one line from the input, then you could just start a bunch of ghcs behind a named pipe and each would steal its own work. But I don't think that's possible with unix pipes, and of course there are still a few non-unix systems out there. And I guess ghc-fe has to wait for the compilation to finish, so I guess ghc has to print a status line when it completes (or fails) a module. But it can still be done with an external distributor program that acts like a server: starts up n ghcs, distributes src files between them, and shuts them down then given the command: data Ghc = Ghc { status :: Free|Busy, in :: Handle, out :: Handle, pid :: Int } main = do origFlags <- getArgs ghcs <- mapM (startup origFlags) [0..cpus] socket <- accept while $ read socket >>= \case of Quit -> return False Compile ghcFlags src -> forkIO $ assert $ ghcFlags == origFlags result <- bracket (findFreeAndMarkBusy ghcs) markFree $ \ghc -> do tellGhc ghc src readResult ghc write socket result return True mapM_ shutdown ghcs The ghc-fe then starts a distributor if one is not running, sends a src file and waits for the response, acting like a drop-in replacement for the ghc cmdline. Build systems just call ghc-fe and have an extra responsibility to call ghc-fe --quit when they are done. And I guess if they know how many files they want to rebuild, it won't be worth it below a certain threshold. So I'm wondering, does this seem reasonable and feasible? Is there a better way to do it? Even if it could be done, would it be worth it? If the answers are "yes", "maybe not", and "maybe yes", then how hard would this be to do and where should I start looking? I'm assuming start at GhcMake.hs and work outwards from there... I'm not entirely sure it would be worth it to me even if it did make full builds, say 1.5x faster for my dual core i5, but it's interesting to think about all the same.

5 16

Removal of #include <HsFFI.h> from template-hsc.h breaks largefile support on 32bit Linux
by Eugene Crosser 27 Feb '12

27 Feb '12

Hello all, I am new here, but I want to report what I suspect may be a problem. I ran into it while using some third-party package from hackage on a 32bit Linux with ghc 7.4.1. I discovered that off_t fields in the .hsc files in the package where interpreted as 32bit words. I suspected that 64bit offsets should be used because even 32bit Linux has "largefile" support with 64bit offsets. I found that earlier versions of hsc2hs included HsFFI.h into the generated C code, and HsFFI.h in turn indirectly includes ghcautoconf.h which has #define _FILE_OFFSET_BITS 64 in it. So, if I build the .hsc files like this: hsc2hs -i HsFFI.h filename.hsc then off_t is 64bit and 64bit file manipulation syscalls are used. I did not check it but I think that earlier versions of hsc2hs where creating largefile-aware version of the code by default, because HsFFI.h was included in the code by default. This is a simple test program: ==== checktypes.hsc ==== -- run like this: hsc2hs checktypes.hsc && runhaskell checktypes.hs module Main where #include <sys/types.h> main = do putStrLn $ show (#size off_t) ======================== $ hsc2hs checktypes.hsc && runhaskell checktypes.hs 4 $ hsc2hs -i HsFFI.h checktypes.hsc && runhaskell checktypes.hs 8 As I understand, this situation means that while the ghc itself and haskell programs compiled by it are largefile-capable, any third party modules that contain .hsc files are not. If I am right, this is probably not a good thing. Please can some guru take a look at this issue? See also: http://www.haskell.org/pipermail/glasgow-haskell-users/2009-February/016606… http://www.haskell.org/pipermail/cvs-ghc/2011-September/065848.html Thanks, Eugene

4 10

Re: Records in Haskell
by Anthony Clayden 27 Feb '12

27 Feb '12

Wren/all Please remember SPJ's request on the Records wiki to stick to the namespace issue. We're trying to make something better that H98's name clash. We are not trying to build some ideal polymorphic record system. To take the field labelled "name": in H98 you have to declare each record in a different module and import every module into your application and always refer to "name" prefixed by the module. DORF doesn't stop you doing any of that. So if you think of each "name" being a different meaning, carry on using multiple modules and module prefixes. That's as easy (or difficult) as under H98. You can declare fieldLabel "name" in one module, import it unqualified into another and declare more records with a "name" label -- contrary to what somebody was claiming. Or you can import fieldLabel "name" qualified, and use it as a selector function on all record types declared using it. It's just a function like any other imported/qualified function, for crying out loud! So if there's 'your' "name" label and 'my' "name", then use the module/qualification system as you would for any other scoped name. Then trying to apply My.name to Your.record will get an instance failure, as usual. (And by the way, there's no "DORFistas", let's avoid personalising this. There are people who don't seem to understand DORF -- both those criticising and those supporting.) AntC ----- Original Message Follows ----- > On 2/25/12 10:18 AM, Gábor Lehel wrote: > > On Sat, Feb 25, 2012 at 3:54 PM, Barney > Hilken<b.hilken(a)ntlworld.com> wrote: >> After more > pondering, I finally think I understand what the DORFistas > want. Here is an example: >> > >> You want to define records which describe people, and > include (among other things) a field called "name". There > might be several different record types with a name field, > depending on whether the record refers to a customer, an > employee, a business contact etc., but in each case "name" > is the name of the person to which the record refers. You > then write various functions which assume this, such as >> > >>> spam :: Has r "name" String => r -> String > >>> spam r = "Dear " ++ r.name ++ "\nHave you > heard..." >> > >> Now I want to define records which describe products, > and I also use a field "name" in the same way, except that > it is the brand name of the product. I also define > functions such as >> > >>> offer :: Has r "name" String => r -> String > >>> offer r = "Reduced! " ++ r.name ++ " 50% off!" > >> > >> It doesn't make any sense to apply your functions to my > records or vice-versa, but because we both chose the same > label, the compiler allows it. Putting the code in > separate modules makes no difference, since labels are > global. > > > Exactly! > > FWIW, this is the concern I alluded to earlier. Namely > that we may want to have two (or more), er, 'classes' of > records--- where a field is polymorphic over an > individual class, but we don't want those classes to > merge simply because they happened to choose the same name > and type for the field. > > I'm not sure it's a good proposal, but it seems like the > only way to handle this issue is to (1) introduce a new > kind for semantically-oriented field names, and (2) make > the Has class use that kind rather than a type-level > string. By (1), what I mean is that rather than referring > to the field as "name", we would declare PersonalName and > BrandName and then use those in lieu of the string. And if > we do that, then (2) demands that we must somehow make > explicit which one we mean, should we want the `name` > field to be polymorphic for some given record > declaration. > > -- > Live well, > ~wren > > _______________________________________________ > Glasgow-haskell-users mailing list > Glasgow-haskell-users(a)haskell.org > http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

3 2

Using 'git bisect' on the GHC tree
by Erik de Castro Lopo 27 Feb '12

27 Feb '12

Hi all, I am trying to track down a build failure on PowerPC that was introduced some time this year. Usually, the easiest way to do this is by using 'git bisect'. Unfortunately, this doesn't work with the GHC tree since its easy to go to a GHC revision which is incompatible with one of the many GHC sub-modules (eg Cabal). Given a GHC git commit hash, is there a way to get the various libraries into a state where I can build the GHC version specified by the hash? Regards, Erik -- ---------------------------------------------------------------------- Erik de Castro Lopo http://www.mega-nerd.com/

2 2

Re: [Haskell-cafe] ANN: hp2html, a tool for viewing GHC heap-profiles
by Johan Tibell 22 Feb '12

22 Feb '12

On Tue, Feb 21, 2012 at 7:36 PM, Iavor Diatchki <iavor.diatchki(a)gmail.com> wrote: > Hello, > > I made a new version of `hp2html` (0.2), which shows the legend by default > (take a look at at the attached screen shot). If the legend is in the way, > you can turn it off (and on) by clicking on the button at the top of the > screen. Nice! -- Johan

1 0

Re: [Haskell-cafe] ANN: hp2html, a tool for viewing GHC heap-profiles
by Johan Tibell 21 Feb '12

21 Feb '12

Hi Iavor, On Mon, Feb 20, 2012 at 6:45 PM, Iavor Diatchki <iavor.diatchki(a)gmail.com> wrote: > Hello, > > I am happy to announce the availability of a little tool that I wrote while > I was doing some Haskell profiling. It converts GHC's heap-profiles into > HTML, and renders them nicely using the flot library. Its functionality is > similar to `hp2ps`. I wrote it because I find the HTML output easier to > work with and, also, because it can cope with partial profiles, so one can > refresh the profile while the program is running. The tool is a very short > Haskell program, so it should be quite easy to modify and improve (and there > is a lot that can be improved in it! :-). Looks really nice. The hovering behavior is nice, but I'd like to see the legend as well. It makes it quicker when you want to get a quick overview of what types there are, as the eye can travel back-and-forth between the graph and the legend. -- Johan

2 2

simple extension to ghc's record disambiguation rules
by John Meacham 20 Feb '12

20 Feb '12

Hi, I'd like to propose an extremely simple extension to ghc's record disambiguation rules, my motivation is that I often have record types with multiple constructors but common fields. so the handy idiom of f Rec { .. } = do blah return Rec { .. } won't work, because I don't know the specific constructor. so, my proposal is that when you come across something like (e::RecType) { blah = foo } (with an explicit type signature like shown) You interpret 'blah' as if it is a field of the record of type 'Rec'. This gives the advantages of record field names being scoped by type but without you having to specify the precise constructor. It is also backwards compatible for expressions, but would be a new thing for patterns which generally don't allow type signatures there. It sidesteps type checker interactions by only being triggered when an explicit type annotation is included. ideally it would be combined with the 'update' and 'label-based pattern-matching' extensions from this page http://hackage.haskell.org/trac/haskell-prime/wiki/ExistingRecords John

2 3

Holes in GHC
by Thijs Alkemade 19 Feb '12

19 Feb '12

Hello! As mentioned in our previous mail [1], we've been working on introducing Agda's holes into GHC [2]. Our general approach is as follows: we've added a hole to the syntax, denoted here by two underscores: "__". We've introduced a new HsExpr for this, which stores the hole's source span (as this is the only thing that a user needs to identify the hole). Then, we extended the local environment of the typechecker to include a mutable Map, mapping a SrcSpan to a Type and a WantedConstraints (we need these later to produce the correct class constraints on the type). In TcExpr.lhs, whenever the tcExpr function encounters a hole, it adds the source position with the res_ty and the current tcl_lie to the map. After type checking has finished, these types can be taken out of the map and shown to the user, however, we are not sure yet where to do this. Currently the map is read in tcRnModule and tcRnExpr, so loading modules and evaluating expressions with ":t" will show the types of the holes in that module or expression. There, we call mkPiTypes, mkForAllTys (with the quantified type variables we obtained from the WantedConstraints), we zonk and tidy them all (most of this imitates how tcRnExpr modifies a type before returning it, except the tidying, which needs to pass the updated TidyEnv to the tidying of the next hole). Examples: *Main> :t [__, ()] tcRnExpr2: [(<interactive>:1:2-3, ())] [__, ()] :: [()] *Main> :t map __ __ tcRnExpr2: [(<interactive>:1:5-6, a0 -> b), (<interactive>:1:8-9, [a0])] map __ __ :: [b] Any feedback on this design would be appreciated. We would like to know if this is something that could be considered to be added to GHC, and what your requirements for that are. Also, we have a confusing problem when type checking a module. Above, we showed the result of ":t map __ __" in ghci. However, if we put "f = map __ __" in a module, we get: tcRnModule: [(f.hs:1:9-10, GHC.Prim.Any * -> b), (f.hs:1:12-13, [GHC.Prim.Any *])] If I read GHC.Prim.Any * as forall a. a, then this is not correct: the GHC.Prim.Any * in both holes should have the same type. We assume it shows up because the type that should be there does not occur in the type signature of f (as it's just [b]), but so far we've been unable to figure out why this behavior is different in interactive mode. Does someone have an idea about what to do to avoid the introduction of these GHC.Prim.Any * types? Best regards, Thijs Alkemade [1] http://www.haskell.org/pipermail/glasgow-haskell-users/2012-January/021453.… [2] https://github.com/xnyhps/ghc/commits/master

15 26

Re: simple extension to ghc's record disambiguation rules
by Anthony Clayden 18 Feb '12

18 Feb '12

> Hi, I'd like to propose an extremely simple extension to > ghc's record disambiguation rules, > John, I've just posted a proposal on the 'Records in Haskell' wiki that I think will do the job for you. "Declared Overloaded Record Fields" (DORF). I'd appreciate feedback. > my motivation is that I often have record types with > multiple constructors but common fields. > This is exactly the use case I'm aiming at. > > so, my proposal is that when you come across something > like > > (e::RecType) { blah = foo } > > (with an explicit type signature like shown) > ... My proposal is the same, except that you don't need a type sig (or not always). That record syntax desugars to a call to `set', which is a method of class `Has' (which also has method `get'). Has/get/set are overloaded for each record/field combination. The instance is generated at the record declaration, instead of the H98 record selector function. > > It is also backwards compatible for expressions, but would > be a new thing for patterns which generally don't allow > type signatures there. > DORF is backwards compatible with H98 record selectors, in the sense that polymorphic selectors (using `get') are just functions. > It sidesteps type checker interactions by only being > triggered when an explicit type annotation is included. > (DORF uses usual instance resolution by record type and field.) > ideally it would be combined with the 'update' and > 'label-based pattern-matching' extensions from this page > http://hackage.haskell.org/trac/haskell-prime/wiki/ExistingRecords > > John > DORF supports update and pattern matching by label, per -XDismabiguateRecordFields and friends. Anthony

1 0

ghc 7.0.4 with the new Xcode 4.3
by Luca Ciciriello 17 Feb '12

17 Feb '12

Hi All. If you have ghc 7.0.4 on Mac OS X 10.7.3 and you have recently installed the new Xcode 4.3, here a workaround to continue using ghc without the /Developer folder (removed with Xcode 4.3). Workaround: from terminal type: sudo emacs /Library/Frameworks/GHC.framework/Versions/7.0.4-x86_64/usr/bin/ghc-7.0.4 then insert your psw. in emacs change the line pgmgcc="/Developer/usr/bin/gcc" in pgmgcc="gcc" save all. Now Your ghc should be working properly. Bye. Luca.

1 0