- Glasgow-haskell-users

Re: [Haskell-cafe] Records in Haskell
by Evan Laforge 01 Mar '12

01 Mar '12

> Thanks Evan, I've had a quick read through. Thanks for reading and commenting! > It's a bit difficult to compare to the other proposals. > > I can't see discussion of extracting higher-ranked functions and applying them > in polymorphic contexts. (This is SPJ's `rev` example.) > > Putting h-r fields into records is the standard way of emulating object- > oriented style. SPJ's view is that requirement is "very important in practice". > > (No proposal has a good answer to updating h-r's, which you do discuss.) Yeah, I've never wanted that kind of thing. I've written in object-oriented languages so it's not just that I'm not used to the feature so I don't feel its lack. And if I did want it, I would probably not mind falling back to the traditional record syntax, though I can see how people might find that unsatisfying. But my suggestion is meant to solve only the problem of composed record updates and redundant "thing"s in 'Thing.thing_field thing'. Not supporting higher-ranked function record fields *only* means that you can't use this particular convenience to compose updates to a higher-ranked field. If you happen to have that particular intersection of requirements then you'll have to fall back to typing more "thing"s for that particular update. My motivation is to solve an awkward thing about writing in haskell as it is, not add a new programming style. > Re the cons 1. "Still can't have two records with the same field name in the > same module since it relies on modules for namespacing." > > Did you see the DORF precursor page ? > http://hackage.haskell.org/trac/ghc/wiki/Records/DeclaredOverloadedRecordFi… > /NoMonoRecordFields > > I tried to figure out if that would help, but I suspect not. (Looking at the > desugar for `deriving (Lens)`, you need the H98 field selector functions.) > Then for me, cons 1. is a show-stopper. (I know you think the opposite.) Yeah, I don't think the DORF precursor stuff is related, because it's all based on typeclasses. I think there are two places where people get annoyed about name clashes. One is where they really want to have two records with the same field name defined in one module. The other is where they are using unqualified imports to shorten names and get a clash from records in different modules. Only the former is a problem, the latter should work just fine with my proposal because ghc lets you import clashing names as long as you don't call them unqualified, and SDNR qualifies them for you. So about the former... I've never had this problem, though the point about circular imports forcing lots of things into the same module is well taken, I have experienced that. In that case: nested modules. It's an orthogonal feature that can be implemented and enabled separately, and can be useful in other ways too, and can be implemented separately. If we are to retain modules as *the* way to organize namespaces and visibility then we should think about fancying-up modules when a namespacing problem comes up. Otherwise you're talking about putting more than one function into one symbol, and that's typeclasses, and now you have to think of something clever to counteract typeclasses' desire to be global (e.g. type proxies). Maybe that's forcing typeclasses too far beyond their power/weight compromise design? > I also don't see whether you can 'hide' or make abstract the representation of > a record type, but still allow read-access to (some of) its fields. If you want a read-only field, then don't export the lens for 'a', export a normal function for it. However, it would mean you'd use it as a normal function, and couldn't pass it to 'get' because it's not a lens, and couldn't be composed together with lenses. I'd think it would be possible to put 'get' and 'set' into different typeclasses and give ReadLenses only the ReadLens dictionary. But effectively we'd need subtyping, so a Lens could be "casted" automatically to a ReadLens. I'm sure it's possible to encode with clever rank2 and existentials and whatnot, but at that point I'm inclined to say it's too complicated and not worth it. Use plain functions. Since 'get' turns a lens into a plain function, you can still compose with '#roField . get (#rwField1 . #rwField2)'. We could easily support 'get (#roField1 . #roField2)' by doing the ReadLens thing and putting (->) into ReadLens, it's just combining rw fields and ro fields into the same composition that would require type gymnastics. > Suppose a > malicious client declares a record with field #a. Can you stop them reading > and/or updating your field #a whilst still letting them see field #b of your > record type? I don't think it's worth designing to support malicious clients, but if you don't want to allow access to a function or lens or any value, then don't export it. #a can't resolve to M.a if M doesn't export 'a'. > With SDNR, is it possibly to define a polymorphic field selector function? I > suspect no looking at the desugar for `deriving (Lens)`, but perhaps I've mis- > understood. I mean: > get_a r = ?? #a r -- gets the #a field from any record r Nope, 'r' has to be monomorphic. That's the "no structural polymorphism" thing I was talking about. I don't mind not having it. If we do that I believe we're moving to a fundamentally different type of polymorphism, and, I think, a less principled one than typeclasses. > This mechanism then supports the idea of 'virtual' fields -- SPJ's example of > fullName, built from polymorphic firstName and lastName. The read-only version is a plain function: fullName r = firstName r ++ " " ++ lastName r Because we are using plain functions we don't need to get all fancy with "virtual" things, they're just normal functions. If you want to update fullName as well, then you could write a lens version: fullName r = lens (firstName r ++ " " ++ lastName r) (\name -> let [first, last] = unwords name in set #firstName first . set #lastName last) Also, fclabels has a notion of views where you can express one type in terms of another (say extract two fields as a pair) and then if you update the first element of the pair it updates the record that came from. It seemed complicated, but maybe it has uses I haven't thought of. I suppose if you wanted to have a record with some internal and some external ones, you could then define the external view as another record, and then they can pass around the whole thing but only get to fiddle with the external fields. I would try to solve the problem without views, e.g. nest the externally visible record inside the internal one instead of trying to put all the fields into one record. > [By the way, did you mean to post to the cafe only? Most of the discussion is > going on on ghc-users.] Oops, no I hadn't noticed which list the threads were on, thanks for pointing it out. Ccing ghc-users. Anyway, one thing I think is different about this proposal is what I said before: it aims to solve an awkward thing about haskell as it is, not add a new programming style. So it adds no power whatsoever. It's more limited than the existing record notation, so it's not even a replacement (but from what I understand, the other proposals aren't either). But it takes what is, for me, the 90% use case and eliminates the redundant typing. Maybe it's not nearly 90% for other people, I can only speak for myself.

1 0

Unpack primitive types by default in data
by Johan Tibell 01 Mar '12

01 Mar '12

Hi all, I've been thinking about this some more and I think we should definitely unpack primitive types (e.g. Int, Word, Float, Double, Char) by default. The worry is that reboxing will cost us, but I realized today that at least one other language, Java, does this already today and even though it hurts performance in some cases, it seems to be a win on average. In Java all primitive fields get auto-boxed/unboxed when stored in polymorphic fields (e.g. in a HashMap which stores keys and fields as Object pointers.) This seems analogous to our case, except we might also unbox when calling lazy functions. Here's an idea of how to test this hypothesis: 1. Get a bunch of benchmarks. 2. Change GHC to make UNPACK a no-op for primitive types (as library authors have already worked around the lack of unpacking by using this pragma.) 3. Run the benchmarks. 4. Change GHC to always unpack primitive types (regardless of the presence of an UNPACK pragma.) 5. Run the benchmarks. 6. Compare the results. Number (1) might be what's keeping us back right now, as we feel that we don't have a good benchmark set. I suggest we try with nofib first and see if there's a different and then move on to e.g. the shootout benchmarks. I imagine that ignoring UNPACK pragmas selectively wouldn't be too hard. Where the relevant code? Cheers, Johan

8 29

New STM?
by wren ng thornton 01 Mar '12

01 Mar '12

Hello all, I had some patches committed to STM a while back[1], and for some reason I was expecting them in the new GHC release. Does STM only ship with the Platform? If not, then any idea when the next version of STM will be released? [1] http://hackage.haskell.org/trac/ghc/ticket/5104 http://www.haskell.org/pipermail/cvs-libraries/2011-April/012914.html -- Live well, ~wren

2 2

Understanding the -A and the -H flags
by Johan Tibell 29 Feb '12

29 Feb '12

Hi! I'm trying to understand the interaction between the -A and -H RTS flags. The documentation at http://www.haskell.org/ghc/docs/7.4.1/html/users_guide/runtime-control.html says that if you use -H (with or without an argument) it implicitly implies some value of -A. However, it's not clear to me what value -A will get and how that value is related to the value of -H. For example, if I set the suggested heap size to 1G, using -H1G, surely the size of the nursery (-A) won't be "whatever is left over," but something more reasonable e.g. the size of the L2 cache? Perhaps it would make sense to document the actual algorithm used to set -A given -H (with and without argument.) Cheers, Johan

3 6

Re: Records in Haskell
by Anthony Clayden 29 Feb '12

29 Feb '12

Whoa! suddenly a deluge over the DORF proposal. I don't have time to reply fully now, but I must say: Barney you have got it all wrong. No, DORF does not attach one class to each label. There is only one class 'Has', with methods get and set. Each record decl generates an instance for the combination of record/field. You can't mix declared and free-standing labels in the same record. The switch for DORF is at the module level: in a module either all records and labels use DORF, or none do (that is, they use H98 style with each field name being unique). AntC ----- Original Message Follows ----- > > My objection is that I'm not sure if there is ever a > > case where "you really want things to be polymorphic > over all records". > > Well, I don't have a simple, really convincing example, > but there are certainly things I want to play with. More > importantly, DORF automatically attaches one class to each > label, but this is often not what you want. For example, > if you have two fields "firstname" and "lastname" the > associated classes are less useful: what you really want > is > > > class (Has r "firstname" String, Has r "lastname" > String) => HasPersonalName r > > so that you can define > > > fullname :: HasPersonalName r => r -> String > > fullname r = r.firstname ++ " " ++ r.lastname > > You may also want to define subclasses to express more > specific conditions. In general, the compiler cannot > automatically deduce what is semantically important: you > need to define it yourself. The Has class is the base on > which you can build. > > > It doesn't seem like the > > Haskell way to have the less safe thing as the one > > that's default and convenient, and to allow the > > programmer to layer a more-safe thing on top of it if he > > or she wants to. It seems more like the Haskell way to > have the safer thing be the default and to require extra > > work if you want to do something less safe*. > > I think you are using the word "safe" in a slightly > misleading way. None of this is mathematically unsafe, > because projections are natural (truly polymorphic). The > "safety" that is broken here is nothing to do with the > semantics of the language, it is to do with the semantics > of the system being implemented, and that is something the > compiler cannot infer. As my example above shows, it > doesn't always correspond one to one with the labels. > > The Haskel way is to make things as polymorphic as is > mathematically safe, even when this goes beyond the > programmers original intention. You can then restrict this > polymorphism by giving explicit less general types in the > same way as in my examples. I think my approach is more > Haskel like. > > Another important Haskel design consideration is to reuse > parts of the language where possible, rather than > introduce new structures. Type classes were originally > introduced to deal with equality and numeric functions, > but were reused for many things including monads. My > approach achieves the same as DORF (and more), but using > existing language features instead of introducing new > ones. > > Barney. > > > _______________________________________________ > Glasgow-haskell-users mailing list > Glasgow-haskell-users(a)haskell.org > http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

6 6

Increasing number of worker tasks in RTS (GHC 7.4.1) - how to debug?
by Sanket Agrawal 28 Feb '12

28 Feb '12

I wrote a program that uses a timed thread to collect data from a C producer (using FFI). The number of threads in C producer are fixed (and created at init). One haskell timer thread uses threadDelay to run itself on timed interval. When I look at RTS output after killing the program after couple of timer iterations, I see number of worker tasks increasing with time. For example, below is an output after 20 iterations of timer event: MUT time (elapsed) GC time (elapsed) Task 0 (worker) : 0.00s ( 0.00s) 0.00s ( 0.00s) Task 1 (worker) : 0.00s ( 0.00s) 0.00s ( 0.00s) .......output until task 37 snipped as it is same as task 1....... Task 38 (worker) : 0.07s ( 0.09s) 0.00s ( 0.00s) Task 39 (worker) : 0.07s ( 0.09s) 0.00s ( 0.00s) Task 40 (worker) : 0.18s ( 10.20s) 0.00s ( 0.00s) Task 41 (worker) : 0.18s ( 10.20s) 0.00s ( 0.00s) Task 42 (worker) : 0.18s ( 10.20s) 0.00s ( 0.00s) Task 43 (worker) : 0.18s ( 10.20s) 0.00s ( 0.00s) Task 44 (worker) : 0.52s ( 10.74s) 0.00s ( 0.00s) Task 45 (worker) : 0.52s ( 10.75s) 0.00s ( 0.00s) Task 46 (worker) : 0.52s ( 10.75s) 0.00s ( 0.00s) Task 47 (bound) : 0.00s ( 0.00s) 0.00s ( 0.00s) After two iterations of timer event: MUT time (elapsed) GC time (elapsed) Task 0 (worker) : 0.00s ( 0.00s) 0.00s ( 0.00s) Task 1 (worker) : 0.00s ( 0.00s) 0.00s ( 0.00s) Task 2 (worker) : 0.07s ( 0.09s) 0.00s ( 0.00s) Task 3 (worker) : 0.07s ( 0.09s) 0.00s ( 0.00s) Task 4 (worker) : 0.16s ( 1.21s) 0.00s ( 0.00s) Task 5 (worker) : 0.16s ( 1.21s) 0.00s ( 0.00s) Task 6 (worker) : 0.16s ( 1.21s) 0.00s ( 0.00s) Task 7 (worker) : 0.16s ( 1.21s) 0.00s ( 0.00s) Task 8 (worker) : 0.48s ( 1.80s) 0.00s ( 0.00s) Task 9 (worker) : 0.48s ( 1.81s) 0.00s ( 0.00s) Task 10 (worker) : 0.48s ( 1.81s) 0.00s ( 0.00s) Task 11 (bound) : 0.00s ( 0.00s) 0.00s ( 0.00s) Haskell code has one forkIO call to kick off C FFI - C FFI creates 8 threads. Runtime options are "-N3 +RTS -s". timer event is kicked off after forkIO. It is for the form (pseudo-code): timerevent <other arguments> time = run where run = do threadDelay time >> do some work >> run where <other variables defined for run function> I also wrote a simpler code using just timer event (fork one timer event, and run another timer event after that), but didn't see any tasks in RTS output. I tried searching GHC page for documentation on RTS output, but didn't find anything that could help me debug above issue. I suspect that timer event is the root cause of increasing number of tasks (with all but last 9 tasks idle - I guess 8 tasks belong to C FFI, and one task to timerevent thread), and hence, memory leak. I will appreciate pointers on how to debug it. The timerevent does forkIO a call to send collected data from C FFI to a db server, but disabling that fork still results in the issue of increasing number of tasks. So, it seems strongly correlated with timer event though I am unable to reproduce it with a simpler version of timer event (which removes mvar sync/callback from C FFI).

2 4

Enormous FFI speedup observed
by Joachim Breitner 28 Feb '12

28 Feb '12

Hi, as I wrote on http://www.joachim-breitner.de/blog/archives/546-GHC-7.4.1-speeds-up-arbtt-… I observe an enormous speed up (22×) in arbtt, and it seems to be related to FFI function calls. The release notes do not give a hint what has changed to result in this speed up, so I’m curious whether someone here knows the reason, or can make a good guess? Thanks, Joachim -- Joachim "nomeata" Breitner mail(a)joachim-breitner.de | nomeata(a)debian.org | GPG: 0x4743206C xmpp: nomeata(a)joachim-breitner.de | http://www.joachim-breitner.de/

2 1

parallel build fixes for 7.4.2
by Conrad Parker 28 Feb '12

28 Feb '12

Hi, recently we've been tweaking our internal build system at Tsuru to handle parallel builds of both cabal packages via 'cabal-sort --makefile' and our local code tree via 'ghc -M'. In addition to the recompilation checker fixes of #5878, the following would be great to have in 7.4.2: 1) http://hackage.haskell.org/trac/ghc/ticket/5891 -- the patch fixes a race condition in creating parent directories for built object files 2) master commit b6f94b5 "Compile link .note section separately from main.c" -- I think this is the patch that fixes link errors we've seen during parallel builds (via ghc -M) with 7.4.1, such as: /x/y/z.o: file not recognized: File truncated and: /usr/bin/ld: BFD (GNU Binutils for Ubuntu) 2.20.51-system.20100908 internal error, aborting at ../../bfd/merge.c line 872 in _bfd_merged_section_offset Will everything currently in master already be included in the next release or is it a separate branch? (If it's a separate branch I'll do some more testing to confirm that b6f94b5 is the patch that fixes the link error). Conrad.

3 2

Re: Records in Haskell
by Anthony Clayden 28 Feb '12

28 Feb '12

> Actually, I looked at the SORF page again: > http://hackage.haskell.org/trac/ghc/wiki/Records/OverloadedRecordFields > There is now an 'Alternative Proposal' section that claims > to solve polymorphic update. > If anyone has comments on this please let us know! Yes, Greg the quasifunctor stuff has been there for some time. You seem to be confused between polymorphic fields vs type-changing update. Quasifunctor is talking about type-changing update. (Look at the example given for `data R a`: there's no polymorphic field.) DORF already has a working approach to type-changing update. I make reference to quasifunctor in the DORF proposal. Essentially, if quasifunctor can be made to work for SORF, it can also be made to work for DORF. (Because what underlies both is to use a Has class with methods get/set.) No, I don't think anybody has a satisfactory approach to updating polymorphic/higher-ranked fields. (DORF mentions one, but it's a ghastly hack. In addition to higher-ranked types, SPJ is also concerned about h-r's with constraints. I've added a very speculative piece in DORF's comparison to SORF that considers using Constraint Kinds. (It's probably piling ghastliness upon ghastliness.) > ... and not automatically > abstract over fields. This leaves all of our future > options open while satisfying the narrow issue at hand. > What on earth do you mean by "not automatically abstract over fields"? AntC

4 4

parallelizing ghc
by Evan Laforge 27 Feb '12

27 Feb '12

I recently switched from ghc --make to a parallelized build system. I was looking forward to faster builds, and while they are much faster at figuring out what has to be rebuilt (which is most of the time for a small rebuild, since ld dominates), compilation of the whole system is either the same or slightly slower than the single threaded ghc --make version. My guess is that the overhead of starting up lots of individual ghcs, each of which has to read all the .hi files all over again, just about cancels out the parallelism gains. So one way around that would be parallelizing --make, which has been a TODO for a long time. However, I believe that's never going to be satisfactory for a project involving various different languages, because ghc itself is never going to be a general purpose build system. So ghc --make provides two things: a dependency chaser and a way to keep the compiler resident as it compiles new files. Since the dependency chaser will never be as powerful as a real build system, it occurs to me that the only reasonable way forward is to split out the second part, by adding an --interactive flag to ghc. It would then read filenames on stdin, compiling each one in turn, only exiting when it sees EOF. Then a separate program, ghc-fe, can wrap ghc and acts like a drop-in replacement for ghc. It would be nice if ghc could atomically read one line from the input, then you could just start a bunch of ghcs behind a named pipe and each would steal its own work. But I don't think that's possible with unix pipes, and of course there are still a few non-unix systems out there. And I guess ghc-fe has to wait for the compilation to finish, so I guess ghc has to print a status line when it completes (or fails) a module. But it can still be done with an external distributor program that acts like a server: starts up n ghcs, distributes src files between them, and shuts them down then given the command: data Ghc = Ghc { status :: Free|Busy, in :: Handle, out :: Handle, pid :: Int } main = do origFlags <- getArgs ghcs <- mapM (startup origFlags) [0..cpus] socket <- accept while $ read socket >>= \case of Quit -> return False Compile ghcFlags src -> forkIO $ assert $ ghcFlags == origFlags result <- bracket (findFreeAndMarkBusy ghcs) markFree $ \ghc -> do tellGhc ghc src readResult ghc write socket result return True mapM_ shutdown ghcs The ghc-fe then starts a distributor if one is not running, sends a src file and waits for the response, acting like a drop-in replacement for the ghc cmdline. Build systems just call ghc-fe and have an extra responsibility to call ghc-fe --quit when they are done. And I guess if they know how many files they want to rebuild, it won't be worth it below a certain threshold. So I'm wondering, does this seem reasonable and feasible? Is there a better way to do it? Even if it could be done, would it be worth it? If the answers are "yes", "maybe not", and "maybe yes", then how hard would this be to do and where should I start looking? I'm assuming start at GhcMake.hs and work outwards from there... I'm not entirely sure it would be worth it to me even if it did make full builds, say 1.5x faster for my dual core i5, but it's interesting to think about all the same.

5 16