- Glasgow-haskell-users

Foreign.Marshal.Alloc questions
by Chris Kuklewicz 10 Aug '06

10 Aug '06

For the allocation functions in Foreign.Marshall.Alloc, are the function contracts that return new pointers such that the pointers are never nullPtr, or is it prudent to always check the new ptr? I can see on GHC than allocaBytes uses newPinnedByteArray# but I have not tracked down the source file for that to read it. -- Chris Kuklewicz

2 1

fork and ghc
by John Meacham 10 Aug '06

10 Aug '06

I want to modify jhc to take advantage of mutiple CPUs to help mitigate its prodigious computational requirements and was curious how well ghc compiled programs deal with forking? my initial plan is that once jhc determines which modules need to be recompiled, it will fork(2) off processes down each pass that can be done in parallel. however, I have no idea what the effect of just calling fork on the ghc runtime will be. both parent and child will continue on compiling code, so it won't be immediatly followed by an exec. John -- John Meacham - ⑆repetae.net⑆john⑈

3 3

RE: GHC 6.4.3 on FreeBSD (fwd)
by Simon Marlow 09 Aug '06

09 Aug '06

Thanks Robert. I don't believe that particular bug is on our critical path, but I'm glad they have a fix. I haven't tried the patch myself, not having a FreeBSD build handy. The current status w.r.t. GHC is: - I merged a fix to the timeout program in our testsuite that fixes occasional hangs on FreeBSD (issue was calling fork() from the child of fork(), not allowed according to POSIX). - I did a complete build and testsuite run with -lthr instead of -pthread, this completes without problems. - When GHC itself is linked with the threaded RTS and -pthread, it occasionally hangs (I saw about 5 hangs during a testsuite run of 2000 tests). When it hangs, it is consuming 100% CPU. I tried attached to the hung GHC process using gdb, and got this: ~ > gdb --pid=53978 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd". Attaching to process 53978 /usr/src/gnu/usr.bin/gdb/libgdb/../../../../contrib/gdb/gdb/solib-svr4.c :1443: internal-error: legacy_fetch_link_map_offsets called without legacy link_map support enabled. A problem internal to GDB has been detected, further debugging may prove unreliable. Quit this debugging session? (y or n) y /usr/src/gnu/usr.bin/gdb/libgdb/../../../../contrib/gdb/gdb/solib-svr4.c :1443: internal-error: legacy_fetch_link_map_offsets called without legacy link_map support enabled. A problem internal to GDB has been detected, further debugging may prove unreliable. Create a core file of GDB? (y or n) y Abort trap: 6 (core dumped) At this stage I'm tempted to default to using -lthr, since this appears to work. I can't make any further progress on the -pthread hangs without gdb working, unfortunately. If you think I should report the gdb crash, where is the best place? Cheers, Simon On 07 August 2006 18:31, Robert Watson wrote: > Dear many, > > I pinged the FreeBSD threads list about the previously mentioned > problem, and was sent this patch by Dan Eischen. The FreeBSD threads > mailing list is extremely responsive to bug reports; my > recommendation is that we direct future specific bug reports, as well > as any feedback on this patch, to that list. > > Thanks, > > Robert N M Watson > Computer Laboratory > University of Cambridge > > ---------- Forwarded message ---------- > Date: Mon, 7 Aug 2006 13:15:21 -0400 (EDT) > From: Daniel Eischen <deischen(a)freebsd.org> > To: Robert Watson <rwatson(a)freebsd.org> > Cc: threads(a)freebsd.org > Subject: Re: GHC 6.4.3 on FreeBSD (fwd) > > On Mon, 7 Aug 2006, Robert Watson wrote: > >> >> I've been talking to the Haskell folk about problems they've had >> getting their threaded runtime running on FreeBSD. Here's one of >> their problem reports. Since I'm not up on how threads and signals >> are supposed to act, I can't confirm the below is a bug, of course, >> but figured you (experts) would be able to respond better. > > Try this patch.

1 0

Building GHC on OS X
by David Kirkman 09 Aug '06

09 Aug '06

Hi, I've been using ghc on Mac OS X for a while now, so when I noticed Simon Marlow's call (made last month, but I just noticed it) for people to help with OS X, I thought, why not? Well, it turns out that I can't even build ghc, so I'm not likely to end up being much help. But it would be nice to at least be able to build the thing, so if anybody can help me... I'm trying to build GHC (6.5.?? ... version from darcs.haskell.org as of yesterday) on Mac OS X (10.4.7 ppc) bootstrapping from ghc 6.4.2 and gcc 4.0.1. "./configure; make" does not work. On my initial try, I ran into the UNDO_DELETE, etc problems associated with Darwin's partial readline support described in ticket #766. The fix suggested in that ticket seems to work fine, but not until I installed the GNU readline library. (If the change to detect the broken Darwin readline is in the repository, it did not work for me). After fixing the readline problem, it looks like I can build at least part of the stage1 compiler -- but the build fails just after starting to build stage2. The final error is: > ../compiler/ghc-inplace -H32m -O0 -W -fno-warn-unused-matches - fwarn-unused-imports -istage2/utils -istage2/basicTypes -istage2/ types -istage2/hsSyn -istage2/prelude -istage2/rename -istage2/ typecheck -istage2/deSugar -istage2/coreSyn -istage2/specialise - istage2/simplCore -istage2/stranal -istage2/stgSyn -istage2/ simplStg -istage2/codeGen -istage2/main -istage2/profiling - istage2/parser -istage2/cprAnalysis -istage2/ndpFlatten -istage2/ iface -istage2/cmm -istage2/nativeGen -istage2/ghci -Istage2 - DGHCI -DBREAKPOINT -package template-haskell -threaded -cpp -fglasgow- exts -fno-generics -Rghc-timing -I. -IcodeGen -InativeGen -Iparser - package unix -package Cabal -ignore-package lang -recomp -Rghc-timing -DDEBUG -H16M '-#include "hschooks.h"' -package-name ghc-6.5 - fgenerics -O -c utils/FastString.lhs -o stage2/utils/FastString.o - ohi stage2/utils/FastString.hi > > utils/FastString.lhs:73:0: > Failed to load interface for `GHC.Exts': > Could not find module `GHC.Exts': > use -v to see a list of the files searched for > <<ghc: 52487092 bytes, 5 GCs, 19236/19236 avg/max bytes residency (1 samples), 16M in use, 0.00 INIT (0.00 elapsed), 0.29 MUT (0.38 elapsed), 0.03 GC (0.04 elapsed) :ghc>> > make[2]: *** [stage2/utils/FastString.o] Error 1 > make[1]: *** [stage2] Error 2 > make: *** [bootstrap2] Error 2 The first error that I can find comes when building ghc/libraries/ base (in stage1) > ../../compiler/ghc-inplace -H32m -O0 -W -fno-warn-unused-matches - fwarn-unused-imports -fglasgow-exts -cpp -Iinclude -"#include" HsBase.h -funbox-strict-fields -package-name base-1.0 -O -dcore-lint -W -fno-warn-unused-matches -fwarn-unused-imports -keep-hc-files - fgenerics -c Data/Generics/Schemes.hs -o Data/Generics/Schemes.o - ohi Data/Generics/Schemes.hi > Data/Generics/Schemes.hs:108:15: > Couldn't match expected type `r' (a rigid variable) > against inferred type `[a]' > `r' is bound by the type signature for `listify' > at Data/Generics/Schemes.hs:106:20 > Expected type: r -> r -> r > Inferred type: [a] -> [a] -> [a] > In the first argument of `everything', namely `(++)' > In the call (everything (++) > ([] `mkQ` (\ x -> if p x then [x] else []))) > make[2]: *** [Data/Generics/Schemes.o] Error 1 After that, there are a handful of errors complaining about things that can't be found (e.g.) > CPUTime.hs:5:0: > Failed to load interface for `System.CPUTime': > Could not find module `System.CPUTime': > use -v to see a list of the files searched for > make[2]: *** [CPUTime.o] Error 1 and > Language/Haskell/TH/Syntax.hs:46:0: > Failed to load interface for `Data.PackedString': > Could not find module `Data.PackedString': > use -v to see a list of the files searched for > make[2]: *** [Language/Haskell/TH/Syntax.o] Error 1 There are six or seven more messages like those, but nothing severe enough to stop make until the problem where it can not find 'GHC.Exts'. I tried to brute force it with 'make -k', which went a lot further, but in the end build nothing useful. Anybody have any ideas? Or build instructions for OS X? Thanks, -david k.

2 1

ANN: TextRegexLazy-0.66 devel release
by Chris Kuklewicz 08 Aug '06

08 Aug '06

(when (compilesOk) (announce newVersion)) This is a development release announcement of "TextRegexLazy" The "TextRegexLazy" package is dead. Long Live [regex-base -- interfaces ,regex-compat -- replace Text.Regex ,regex-posix -- PosixRE backend ,regex-pcre -- PCRE backend ,regex-parsec -- My lazy Haskell backend ,regex-dfa -- DFA based on modified CTKLight (LGPL) ] Note: Everything is BSD licensed except the regex-dfa engine which is LGPL. The darcs repository at http://evenmere.org/~chrisk/trl/head/ has the new version. The regex-compat package is a single Text.Regex.New module that uses regex-posix to replace Text.Regex (I cut and pasted from the Text.Regex module and re-implemented some of the functions). The old Text.Regex.Posix module is bypassed (so it could be removed in the future). Changing the dependency to a different backend is as simply as changing the import statement. Changing from String to ByteString would be a simple search-and-replace (and import Data.ByteString). The regex-posix package does not have a personal copy of the posix regex library like the copy GHC brings to Windows: http://darcs.haskell.org/packages/base/cbits/regex/ If someone can get regex-posix working on Windows, then [regex-base,regex-posix,regex-compat] will be functionally ready to be included along with GHC. TODO: * Much Haddock fixing for the new module hierarchy * Point the test cases at the new packages * Benchmark -- Chris Kuklewicz

1 0

Lookup module and package defining <<x>?
by Marc Weber 08 Aug '06

08 Aug '06

Has somene already implemented something like: $ghc-pkg --where-from ParseError package parsec: defining modules: Text.ParserCombinators.Parsec.Error, Text.ParserCombinators.Parsec ? lambdabot @index does what I want, but not with my libs installed only locally.. This might be used by any developement system for easily importing modules.. If not I'll start working on this and I'll post again after needing help ;) or finishing It can't be that hard because there is already ghc-pkg list and ghc --show-iface ... Marc

6 7

wislist item: more control over branch prediction
by Duncan Coutts 08 Aug '06

08 Aug '06

So now that GHC is so good at producing really fast low level code (see ByteString benchmarks) we start to encounter low level gubbins where to get the fastest possible code we need slightly more influence over branch prediction. In the code for ByteString's 'writeStrUp/Dn' functions we're doing a tight loop writing bytes to memory. The first version looks much like this: loop fp n off s = case next s of Done -> return $! PS fp 0 off Skip s' -> loop fp n off s' Yield x s' -> do withForeignPtr fp $ \p -> pokeByteOff p off x loop fp n (off+1) s' When we get to the end of a memory block we need to double the size of the memory block and carry on. So this adds an additional conditional test into this loop. loop fp n off s = case next s of Done -> trimUp fp n off Skip s' -> loop fp n off s' Yield x s' | n == off -> realloc fp n off s' x | otherwise -> do withForeignPtr fp $ \p -> pokeByteOff p off x loop fp n (off+1) s' There are dramatic differences between equivalent forms of code. Just by reversing the order of the (n==off) test one form I can process 50Mb of data in 0.20 seconds and in the other it takes 0.34 seconds. That is: | n == off -> realloc fp n off s' x | otherwise -> do ... vs | n /= off -> do ... | otherwise -> realloc fp n off s' x I think that this is all down to branch prediction and the arrangement of basic blocks. On a modern CPU correctly predicted branches are nearly free while mis-predicted branches are very costly due to stalled pipelines etc. In gcc they have a branch prediction framework. It annotates conditionals with prediction information. It then uses that during code generation to do things like arranging basic blocks so that unlikely blocks are moved out of line. It gets the prediction information from a few sources. One is a simple static branch predictor, for example branches back to to the beginning of a loop are likely to be taken and branches to error functions are not. gcc even has a profile feedback system to find the real probabilities of branches from a sample run of the program. It also has a user annotation __builtin_expect() which many C projects use in the form of a macro: if (unlikely(x==0)) { ... The Linux kernel uses this fairly heavily to move error handling code out of the main 'fast path' code. Anyway, so what I wish is that I could write something like: loop fp n off s = case next s of Done -> {-# UNLIKELY #-} trimUp fp n off Skip s' -> loop fp n off s' Yield x s' | n == off -> {-# UNLIKELY #-} realloc fp n off s' x | otherwise -> do withForeignPtr fp $ \p -> pokeByteOff p off x loop fp n (off+1) s' The best approximating effect that I can use at the moment is to make the unlikely realloc branch a local function in a where clause but mark it with {-# NOINLINE #-} so that the code for the realloc case doesn't pollute the tight loop for the normal fast case. Then the other approximation is fiddling with the logic of the binary test and looking at the benchmarks. The combination of these two techniques makes a dramatic difference to the speed. However I do need more control to do it reliably, while I was able to make it work for writeStrUp, I could not find a combination to work for writeStrDn - we loose about 50% performance when we add the realloc branch. So a slightly more reliable method to hint at the likelihood of conditionals would make this kind of low level code faster and easier to write. Some time ago as a quick hack I did add a branch prediction annotation to the CMM conditional constructor. I only used it in the C backend to turn it into uses of gcc's __builtin_expect(). I also didn't connect it to the front end so I didn't have any high level static branch prediction (which could be done by looking for branches with recursive calls or calls to error etc). The only place I did actually use it was in the heap check and stack check. I assumed that it would be advantageous to predict heap and stack overflow branches as not taken. It was only a quick hack - I didn't do any benchmarks to see if it had any effect. Anyway, just a thought, and of course not something we should be thinking about 'til post 6.6. Duncan

1 0

Evaluating expressions in Visual Haskell
by Vyacheslav Akhmechet 08 Aug '06

08 Aug '06

Hi, I apologize if this question has been asked before, but I couldn't find an FAQ or the answer online. Does Visual Haskell have support for evaluating expressions? I tried a few things (like highlighting an expression, rightclicking, and looking for "evaluate" option) but couldn't find anything. If this feature is missing, is it planned? Also, what is a good way to do this together with Visual Haskell? I could, of course, open an interpreter in a command window, load appropriate modules, and evaluate expressions from there but is there a more "integrated" way? Thanks, - Slava Akhmechet

2 1

Graphics library very slow
by Vyacheslav Akhmechet 08 Aug '06

08 Aug '06

The latest stable release seems to have some sort of a problem with the graphics library (the general one as well as the wrapper used in SOE). Opening a window takes more than a couple of minutes (on Windows XP). When I run an identical version of the code through Hugs, the window opens immediately. Am I doing something wrong? Thanks, - Slava.

4 4

default declarations
by Daniel Fischer 07 Aug '06

07 Aug '06

Hi, apparently ghci doesn't take default declarations into account. Inspired by a question on the hugs-users list I wrote module BoolNum where default (Bool, Rational) instance Num Bool where (+) = (/=) (-) = (/=) (*) = (&&) negate x = x abs x = x signum x = x fromInteger = odd and in hugs, as expected I get BoolNum> 1 True but in ghci, it's *BoolNum> 1 1, no defaulting apparently takes pkace. Why? Bug or feature ? Cheers, Daniel -- "In My Egotistical Opinion, most people's C programs should be indented six feet downward and covered with dirt." -- Blair P. Houghton

2 1