January 2006 - Haskell-Cafe

darcs: the first haskell tool more popular than Haskell itself?
by Isaac Jones 25 Jan '06

25 Jan '06

Greetings, Debian has a system called popularity-contest, which is an opt-in survey of package use. I was curious to see the ranking of darcs among Haskell implementations themselves. The results: Darcs ranks higher than ghc and hugs. #<name> is the package name; #<inst> is the number of people who installed this package; #<vote> is the number of people who use this package regularly; #<old> is the number of people who installed, but don't use this package # regularly; #<recent> is the number of people who upgraded this package recently; #<no-files> is the number of people whose entry didn't contain enough # information (atime and ctime were 0). #rank name inst vote old recent no-files (maintainer) 138 darcs 563 159 280 124 0 (Isaac Jones) (51) hugs 304 119 157 28 0 (Isaac Jones) 321 ghc6 194 81 80 33 0 (Ian Lynagh) Hugs is listed in a different category, so the ranking is off. Ian Lynagh pointed me at this nice graph showing the historical installation of the three packages: http://people.debian.org/~igloo/popcon-graphs/index.php?packages=ghc6,darcs… peace, isaac

5 6

Guess what ... Tutorial uploaded! :)
by Dmitry Astapov 24 Jan '06

24 Jan '06

Hi all, Recend thread "Tutorial upload" reminded me of my intentions to write haskell tutorial which I suppresend from the times when Hal Daume began to write YAHT. Unusually low temperature here (-22C) freed me a lot of time this weekend, and the result it here: http://www.haskell.org/hawiki/HitchhickersGuideToTheHaskell This is a work in progress, plus I'm trying it on C++/Java folk around me, so it's bound to be heavily corrected, but main direction could already be seen (I hope). If you want - jump on the wagon, and provide me with (constructive) criticism, edits on hawiki or anything else you see fit. -- Dmitry Astapov //ADEpt GPG KeyID/fprint: F5D7639D/CA36 E6C4 815D 434D 0498 2B08 7867 4860 F5D7 639D

3 4

Performance Tuning & darcs (a real shootout?)
by Jason Dagit 24 Jan '06

24 Jan '06

Hello, This will be a long email so I've tried to give it sections. = Background = I've been using Haskell a while now (close to a year) and I'm still very much a newbie (but I've used other FP langs). I'm also very interested in darcs which, as many of you know, is written in Haskell. I've found a case which darcs does not handle efficiently and I'm trying to get the performance in that case to a satisfactory level. If you create a new repository, create a single large file (say 300mb) and record that file darcs will take a very long time and uses a lot of memory (usually at least 900mb for the 300mb case). Before I begin telling you what I've tried so far, let me give a brief history of previous efforts by others to optimize darcs. As far as I know, the main person to work on optimizing darcs in the past is Ian Lynagh. He was able to convert darcs from stricter algorithms to lazy ones and this made a huge improvement in performance. In theory darcs should be able to lazily read from a file, compute the patch that needs to be written, write that patch and then lazily read the patch while it is applied to the repository. This mostly works as expected, but I've discovered that somewhere near the end of the application the memory needed spikes dramatically. After almost two weeks of poking at darcs doing various benchmarks and profiles I've realized that optimizing Haskell programs is no easy task. I've been following the advice of numerous people from the haskell irc channel and learned a lot about darcs in the process. I've also been using this nifty library that Ian created for this purpose to get a measure for the non-mmap memory usage: http://urchin.earth.li/darcs/ian/memory Potentially useful information about darcs; 1) Uses a slightly modified version of FastPackedStrings. 2) Can use mmap or not to read files (compile time option). =Experiments and Findings= I have a summary of some of my experimentation with darcs here: http://codersbase.com/index.php/Darcs_performance Basically what I have found is that the read of the original file does not cause a spike in memory usage, nor does writing the patch. This would seem to imply that it's during application of the patch that the memory spikes. Modifying darcs to read the patch file and print just the first line of the patch causes some interesting results. The memory usage according to Ian's memory tool stays very low, at about 150kb max, but requesting the first line of the patch appears to make darcs read the entire patch! Darcs will literally grind away for, say, 30 minutes to just print the first line. On a side note, I've tried turing off mmap and running some of the above experiments. Ian's tool reports the same memory usage, and top still reports large amounts of memory used. Does ghc use mmap to allocate memory instead of malloc? Even if it does this shouldn't be a problem for Ian's tool as long as it maps it anonymously. =Questions= So far I've been tracking this performance problem by reading the output of ghc --show-iface and --ddump-simpl for strictness information, using the ghc profiler (although that makes already bad performance much worse), Ian's memory tool, and a lot of experiments and guess work with program modifications. Is there a better way? Are there tools or techniques that can help me understand why the memory consumption peaks when applying a patch? Is it foolish to think that lazy evaluation is the right approach? I've been thinking that perhaps darcs should be modified to use bounded buffers so that we have tight control over the amount of memory consumed during a run. This solution would require a lot of reworking of the existing code, and sounds frightful from a maintenance point of view. I'm also wondering if using mmap is a bad idea. Given the way files are currently mmap'd I think we are limiting darcs to handling files which are small enough to fit in the address space at once (eg., < 4GB on i386). Additionally, it would seem that using mmap does not really reduce memory stress. I'm looking for advice or help in optimizing darcs in this case. I guess this could be viewed as a challenge for people that felt like the micro benchmarks of the shootout were unfair to Haskell. Can we demonstrate that Haskell provides good performance in the real-world when working with large files? Ideally, darcs could easily work with a patch that is 10GB in size using only a few megs of ram if need be and doing so in about the time it takes read the file once or twice and gzip it. If anyone wants to look at the darcs code themselves the unstable version is available via: darcs get --partial http://abridgegame.org/repos/darcs-unstable Just to recap, I'm looking for 1) advice, 2) tips, 3) design ideas, 4) tools, 5) libraries and just about anything else :) Thanks, Jason

4 5

SDL on Mac
by Jake Luck 23 Jan '06

23 Jan '06

Anyone else here using the SDL binding on their Macs? (If memory serves me, at one point, I had gotten libSDL to work with 6.2.2). Now using GHC 6.4.1, while SDLstable-0.2.0 built without any problem for me on Linux, hsc2hs fails on the Mac: % runhaskell Setup.lhs build --verbose Reading parameters from /SDLstable/src/Core/SDL.buildinfo Preprocessing library SDL-0.2.0... /usr/local/bin/hsc2hs -I/fp/include/SDL -D_THREAD_SAFE -oGraphics/UI/SDL/General.hs Graphics/UI/SDL/General.hsc ld: Undefined symbols: _ZCMain_main_closure ___stginit_ZCMain % cat config.mk SDL_CFLAGS=-I/fp/include/SDL -D_THREAD_SAFE SDL_LIBS=-L/fp/lib -lSDLmain -lSDL -framework Cocoa -framework OpenGL PACKAGE=SDL VERSION=0.1.0 MAINTAINER=lemmih(a)gmail.com % cat SDL.buildinfo cc-options: -I/fp/include/SDL -D_THREAD_SAFE ld-options: -L/fp/lib -lSDLmain -lSDL -framework Cocoa -framework OpenGL I am using the Mac OS X binary ghc 6.4.1 from the GHC download page on 10.4.3, linking to the latest libSDL 1.2.9 handbuilt with gcc 3.3. jake

1 0

Can I use Haskell for web programming
by Maurício 23 Jan '06

23 Jan '06

Hi, Can I use Haskell to do what people do with, say, PHP? More and more I have the need for that, and I've been looking into Ruby on Rails. Do you thing Haskell could be a choice? Of course, I don't need something exactly like PHP (for instance, I don't care if I can't insert code in the middle of xhtml pages. If I have to generate everything from Haskell code, I would probably like it. Also, CGI can be a choice). But I need reasonable efficiency and to be able to find someone to host my site. What solutions do you suggest me? Thanks, Maurício

10 14

RE: [Haskell-cafe] Re: Hashtable woes
by Simon Marlow 23 Jan '06

23 Jan '06

On 23 January 2006 15:09, Chris Kuklewicz wrote: > That is good to hear. The benchmark's tests take 1,250,000 > pre-generated > strings as the keys. At the end, the string keys are 18 characters > long, drawn randomly from a set of 4 characters. So the hash > computations are a nontrivial hit. > > Using -A400m I get 39s down from 55s. That is the best > Data.HashTable time I > have seen. (Using -A10m and -A100m were a little slower). > > Using my over optimized c-code hashtable I get 12.3 seconds. The > associative arrays in OCaml and D are still faster. So you see why I > long for GHC 6.6. > > Is Jan-Willem Maessen's Hash available anywhere? I could benchmark > it. Jan-Willem's HashTable attached. It uses unsafeThaw/unsafeFreeze tricks to avoid the GC overheads, for this you need an up to date GHC due to a bug in the garbage collector: grab a STABLE snapshot (6.4.1 won't work). Or remove the unsafeThaw/unsafeFreeze to use it with 6.4.1, and be prepared to bump the heap size. In GHC 6.6 the unsafeThaw/unsafeFreeze tricks aren't required, because the GC is essentially doing it for you - we put a write barrier in the IOArray implementation. Cheers, Simon

1 0

RE: [Haskell-cafe] GHC integers pre-allocation
by Simon Peyton-Jones 23 Jan '06

23 Jan '06

I believe that we statically allocate all ASCII chars, and Ints between -16 and +16. Have a look in ghc/rts/StgMiscClosures.cmm Simon ________________________________ From: haskell-cafe-bounces(a)haskell.org [mailto:haskell-cafe-bounces@haskell.org] On Behalf Of Monique Monteiro Sent: 22 January 2006 23:26 To: haskell-cafe(a)haskell.org Subject: [Haskell-cafe] GHC integers pre-allocation Hi all, does anyone know if GHC implements any kind of pre-allocation of integers/caracteres ? If yes, which range of values does it use? Thanks in advance, Monique Monteiro

1 0

First steps in Haskell
by Daniel Carrera 22 Jan '06

22 Jan '06

Hello all, I'm trying to write the simplest possible Haskell program, and I'm not getting anywhere. I have installed Hugs, GHC and GHCI. I want to run the following program: fac :: Integer -> Integer fac 0 = 1 fac n | n > 0 = n * fac (n-1) This is what I see: $ hugs Hugs.Base> fac :: Integer -> Integer ERROR - Undefined variable "fac" Hugs.Base> fac 0 = 1 ERROR - Syntax error in input (unexpected `=') $ ghci Prelude> fac :: Integer -> Integer <interactive>:1:0: Not in scope: `fac' Prelude> fac 0 = 1 <interactive>:1:6: parse error on input `=' $ # Write the program to fac.hs $ ghc fac.hs fac.hs:1:0: The main function `main' is not defined in module `Main' When checking the type of the main function `main' This is a real problem for Haskell. I expect that a lot of people try Haskell and give up because they can't even write the simplest function. It's hard not to be put off by this. I love the theory behind Haskell, but the practice of it seems to be a real problem. I hope someone will show me how to make this program work. Even better, I hope someone will fix the compilers and interpreters if they need fixing, or fix the documentation if that's what needs fixing. Best, Daniel. -- /\/`) http://oooauthors.org /\/_/ http://opendocumentfellowship.org /\/_/ \/_/ I am not over-weight, I am under-tall. /

14 27

GHC integers pre-allocation
by Monique Monteiro 22 Jan '06

22 Jan '06

Hi all, does anyone know if GHC implements any kind of pre-allocation of integers/caracteres ? If yes, which range of values does it use? Thanks in advance, Monique Monteiro

1 0

Haskell': Kicking off the process of defining the next Haskell standard.
by Isaac Jones 21 Jan '06

21 Jan '06

let haskell' = succ haskell98 in Announcing the Haskell' ("Haskell-Prime") process. A short time ago, I asked for volunteers to help with the next Haskell standard. A brave group has spoken up, and we've organized ourselves into a committee in order to coordinate the community's work. It will be the committee's task to bring together the very best ideas and work of the broader community in an "open-source" way, and to fill in any gaps in order to make Haskell' as coherent and elegant as Haskell 98. Our task is broadly defined by our mission statement: The Haskell programming language is more-or-less divided into two "branches". The Haskell 98 standard is the "stable" branch of the language, and that has been a big success. A lot of progress has been made over the last few years in the "research" branch of the Haskell language. It is constantly advancing, and we feel that it is time for a new standard which reflects those advancements. Haskell' will be a conservative refinement of Haskell 98. It will be the work of this committee to adopt a set of language extensions and modifications and to standardize a new set of libraries. We will strive to only include tried-and-true language features, and to define them at least as rigorously as Haskell 98 was defined. This standard will reflect the realities of developing practical applications in the Haskell language. We will work closely with the rest of the Haskell community to create this standard. Your Haskell' Committee is as follows (slightly munged email addresses follow): * Manuel M T Chakravarty <chak at cse.unsw.edu.au> * John Goerzen <jgoerzen at complete.org> * Bastiaan Heeren <bastiaan at cs.uu.nl> * Isaac Jones <ijones at galois.com> * John Launchbury <john at galois.com> * Andres Loeh <loeh at iai.uni-bonn.de> * Simon Marlow <simonmar at microsoft.com> * John Meacham <john at repetae.net> * Ravi Nanavati <ravi at bluespec.com> * Henrik Nilsson <nhn at cs.nott.ac.uk> * Ross Paterson <ross at soi.city.ac.uk> * Simon Peyton-Jones <simonpj at microsoft.com> * Don Stewart <dons at cse.unsw.edu.au> * Audrey Tang <autrijus at gmail.com> * Simon J. Thompson <S.J.Thompson at kent.ac.uk> * Malcolm Wallace <Malcolm.Wallace at cs.york.ac.uk> * Stephanie Weirich <sweirich at cis.upenn.edu> The editors are Isaac Jones and John Launchbury. Feel free to contact any of us with any concerns or questions. If you don't know who to direct your questions to, email Isaac Jones ijones(a)syntaxpolice.org. Community involvement is vital to our task, and there will be a way for members of the community to make formal proposals. In the opening phases, please use these more informal resources to help us coordinate Haskell': * The haskell-prime mailing list. All technical discussion will take place here, or (if other meetings take place) be reported here. Anyone can subscribe, and any subscriber can post questions and comments, and participate in discussions. Anyone can read the list archives. http://haskell.org/mailman/listinfo/haskell-prime * A wiki / issue tracking system to document consensus and to track ongoing tasks. This system is publicly readable, but only committee writable so that we may present it as the "official" output of the committee. If you ever feel that the wiki is not accurate as to the consensus, please alert the committee! http://hackage.haskell.org/trac/haskell-prime * A darcs code repository for experiments, proposed libraries,and complex examples. darcs is a decentralized system, so anyone can use it, but patches should be sent to Isaac Jones: http://hackage.haskell.org/trac/haskell-prime/wiki/SourceCode Please join us in making Haskell' a success. your, Haskell' Committee p.s. Please feel free to forward this announcement to appropriate forums.

1 0