- Glasgow-haskell-users

GHC Trac says database is locked
by Sean Leather 20 Nov '09

20 Nov '09

I tried to submit a comment on #2530 and got this error. Sean ------- Oops... *Trac detected an internal error:* If you think this really should work and you can reproduce it. Then you should consider to report this problem to the Trac team. Go to http://trac.edgewall.org/ and create a new ticket where you describe the problem, how to reproduce it. Don't forget to include the python traceback found below. TracGuide <http://hackage.haskell.org/trac/ghc/wiki/TracGuide> — The Trac User and Administration Guide Python traceback Traceback (most recent call last): File "/var/lib/python-support/python2.4/trac/web/main.py", line 387, in dispatch_request dispatcher.dispatch(req) File "/var/lib/python-support/python2.4/trac/web/main.py", line 237, in dispatch resp = chosen_handler.process_request(req) File "/var/lib/python-support/python2.4/trac/ticket/web_ui.py", line 279, in process_request self._do_save(req, db, ticket) File "/var/lib/python-support/python2.4/trac/ticket/web_ui.py", line 546, in _do_save cnum=internal_cnum): File "/var/lib/python-support/python2.4/trac/ticket/model.py", line 250, in save_changes (self.id, when, author, cnum, comment)) File "/var/lib/python-support/python2.4/trac/db/util.py", line 50, in execute return self.cursor.execute(sql_escape_percent(sql), args) File "/usr/lib/python2.4/site-packages/sqlite/main.py", line 237, in execute self.con._begin() File "/usr/lib/python2.4/site-packages/sqlite/main.py", line 503, in _begin self.db.execute("BEGIN") OperationalError: database is locked

3 4

ANNOUNCE: parallel-2.0.0.0
by Simon Marlow 18 Nov '09

18 Nov '09

I've just uploaded parallel-2.0.0.0 to Hackage. If you're using Strategies at all, I'd advise updating to this version of the parallel package. It's not completely API compatible, but if you're just using the supplied Strategies such as parList, the changes should be relatively minor. The Haddock docs include a full description of the changes, reproduced below. Haddock docs are also here, until Hackage catches up: http://www.haskell.org/~simonmar/parallel/index.html The Strategy-using programs I've tried so far go faster. Enjoy! Cheers, Simon --------------------------------------------------------------------- Version 1.x The original Strategies design is described in <http://www.macs.hw.ac.uk/~dsg/gph/papers/html/Strategies/strategies.html> and the code was written by Phil Trinder, Hans-Wolfgang Loidl, Kevin Hammond et al. Version 2.x Later, during work on the shared-memory implementation of parallelism in GHC, we discovered that the original formulation of Strategies had some problems, in particular it lead to space leaks and difficulties expressing speculative parallelism. Details are in the paper \"Runtime Support for Multicore Haskell\" <http://www.haskell.org/~simonmar/papers/multicore-ghc.pdf>. This module has been rewritten in version 2. The main change is to the 'Strategy a' type synonym, which was previously @a -> Done@ and is now @a -> a@. This change helps to fix the space leak described in \"Runtime Support for Multicore Haskell\". The problem is that the runtime will currently retain the memory referenced by all sparks, until they are evaluated. Hence, we must arrange to evaluate all the sparks eventually, just in case they aren't evaluated in parallel, so that they don't cause a space leak. This is why we must return a \"new\" value after applying a 'Strategy', so that the application can evaluate each spark created by the 'Strategy'. The simple rule is this: you /must/ use the result of applying a 'Strategy' if the strategy creates parallel sparks, and you should probably discard the the original value. If you don't do this, currently it may result in a space leak. In the future (GHC 6.14), it will probably result in lost parallelism instead, as we plan to change GHC so that unreferenced sparks are discarded rather than retained (we can't make this change until most code is switched over to this new version of Strategies, because code using the old verison of Strategies would be broken by the change in policy). The other changes in version 2.x are: * Strategies can now be defined using a convenient Applicative type Eval. e.g. parList s = unEval $ traverse (Par . s) * 'parList' has been generalised to 'parTraverse', which works on any 'Traversable' type. * 'parList' and 'parBuffer' have versions specialised to 'rwhnf', and there are transformation rules that automatically translate e.g. @parList rwnhf@ into a call to the optimised version. * 'NFData' is deprecated; please use the @DeepSeq@ class in the @deepseq@ package instead. Note that since the 'Strategy' type changed, 'rnf' is no longer a 'Strategy': use 'rdeepseq' instead.

1 0

Announcing the GHC Bug Sweep
by Simon Marlow 18 Nov '09

18 Nov '09

Help us weed the GHC ticket database, and get a warm fuzzy feeling from contributing to Haskell core technology! There are currently ~750 tickets against GHC. Many of them have not been looked at in months or years. Often when I go through old tickets I find easy targets: bugs that have already been fixed, duplicates, bugs that are not reproducible and the submitter has gone away. So the idea we have is this: do an incremental sweep of the whole database, starting from the oldest tickets. Check each one, and try to make some progress on it. If we get enough momentum going we can make sure every ticket gets looked at every few months at the least. This is a game for the whole family! We don't care how much progress you make on each ticket, just as long as someone has taken a look and moved the ticket forward in some way. For example, you might check for duplicates, update the metadata, ask for more information from the submitter, try to reproduce the bug against the latest version of GHC. To claim a ticket all you have to do is remove it from the list on the wiki. Full instructions are here http://hackage.haskell.org/trac/ghc/wiki/BugSweep including a list of suggestions for ways to make progress on a ticket. Cheers! Simon & the GHC team

2 3

Inliner behaviour - tiny changes lead to huge performance differences
by Bryan O'Sullivan 17 Nov '09

17 Nov '09

I'm working on measuring and improving the performance of the text library at the moment, and the very first test I tried demonstrated a piece of behaviour that I'm not completely able to understand. Actually, I'm not able to understand what's going on at all, beyond a very shallow level. All the comments below pertain to GHC 6.10.4. The text library uses stream fusion, and I want to measure the performance of UTF-8 decoding. The code I'm measuring is very simple: import qualified Data.ByteString as B import Data.Text.Encoding as T import qualified Data.Text as T import System.Environment (getArgs) import Control.Monad (forM_) main = do args <- getArgs forM_ args $ \a -> do s <- B.readFile a let t = T.decodeUtf8 s print (T.length t) The streamUtf8 function looks roughly like this: streamUtf8 :: OnDecodeError -> ByteString -> Stream Char streamUtf8 onErr bs = Stream next 0 (maxSize l) where l = B.length bs next i | i >= l = Done | U8.validate1 x1 = Yield (unsafeChr8 x1) (i+1) | {- etc. -} {-# INLINE [0] streamUtf8 #-} The values being Yielded from the inner function are, as you can see, themselves constructed by functions. Originally, with the inner next function manually marked as INLINE, I found that functions like unsafeChr8 were not being inlined by GHC, and performance was terrible due to the amount of boxing and unboxing happening in the inner loop. I somehow stumbled on the idea of removing the INLINE annotation from next, and performance suddenly improved by a significant integer multiple. This caused the body of streamUtf8 to be inlined into my test program, as I hoped. However, I wasn't yet out of the woods. The length function is defined as follows: length :: Text -> Int length t = Stream.length (Stream.stream t) {-# INLINE length #-} And the streaming length is: length :: Stream Char -> Int length = S.lengthI {-# INLINE[1] length #-} And the lengthI function is defined more generally, in the hope that I could use it for both Int and Int64 lengths: lengthI :: Integral a => Stream Char -> a lengthI (Stream next s0 _len) = loop_length 0 s0 where loop_length !z s = case next s of Done -> z Skip s' -> loop_length z s' Yield _ s' -> loop_length (z + 1) s' {-# INLINE[0] lengthI #-} Unfortunately, although lengthI is inlined into the Int-typed streaming length function, that function is not in turn marked with __inline_me in simplifier output, so the length/decodeUtf8 loops do not fuse. The code is pretty fast, but there's still a lot of boxing and unboxing happening for all the Yields. So. I am quite baffled by this, and I confess to having no idea what to do to get the remaining functions to fuse. But that's not quite confusing enough! Here's a one-byte change to my test code: main = do args <- getArgs forM_ args $ \a -> do s <- B.readFile a let !t = decodeUtf8 s *{- <-- notice the strictness annotation -}* print (T.length t) In principle, this should make the code a little slower, because I'm deliberately forcing a Text value to be created, instead of allowing stream/unstream fusion to occur. Now the length function seems to get inlined properly, but while the decodeUtf8 function is inlined, the functions in its inner loop that must be inlined for performance purposes are not. The result is very slow code. I found another site for this one test where removing a single INLINEannotation makes the strictified code above 2x faster, but that change causes the stream/unstream fusion rule to fail to fire entirely, so the strictness annotation no longer makes a difference to performance. All of these flip-flops in inliner behaviour are very difficult to understand, and they seem to be exceedingly fragile. Should I expect the situation to be better with the new inliner in 6.12? Thanks for bearing with that rather long narrative, Bryan.

9 13

Changes to the ticket system
by Simon Marlow 17 Nov '09

17 Nov '09

I've made a few changes to the ticket system today, in preparation for a push on cleaning up the ticket database (announcement to be made shortly). The changes are: 1. There's a new field "Type of Failure", with values None/Unknown Build failure Install failure GHC doesn't work at all Compile-time crash Runtime crash GHCi crash Incorrect result at runtime Compile-time performance bug Runtime performance bug Hopefully the values are self-explanatory. Setting this correctly will help us to focus on important tickets, and may help us automatically prioritize tickets in the future. 2. The "Severity" field has gone away. This field was intended to be used as a way for ticket submitters to communicate to us the importance of the bug to them, but in practice it wasn't clear how to use it, it hasn't been used consistently, and as a result we generally ignore it. The new "Type of Failure" is a much less ambiguous way to communicate some of the same information. 3. The "Difficulty" field has more sensible values representing a range of times, rather than just "1 hour" or "1 week": Unknown Easy (less than 1 hour) Moderate (less than a day) Difficult (2-5 days) Project (more than a week) Rocket Science Cheers, Simon

3 4

Ubuntu ghc6 package (german translation)
by Heiko Studt 16 Nov '09

16 Nov '09

To: glasgow-haskell-users Cc: ubuntu-devel-discuss Hi, the German translation of the ghc6 package description is some kind of wrong in the current Ubuntu 9.10: the technical term "lazy" is translated as "träge" which means (slow, lazy) with an accent on slow. I don't know whether there is a default translation in the haskell universum (like "faul"). In my opinion the best would be to stay at the technical term "lazy". Possibly this mistake was done in other translations as well. Mit freundlichen Grüßen (With kindly regards) Heiko Studt

2 1

3 questions regarding profiling in ghc
by Daniil Elovkov 13 Nov '09

13 Nov '09

Hello Sorry for the uninformative subject, but I've got no less than 3 questions about profiling in ghc. Here we go 1) Can I profile my program if I don't have all the libraries it depends on compiled with profiling? ghc says it can't find profiling variants of this and that module in other packages, when I try ghc -prof. I managed to build all the dependencies with profiling, but that being a necessity doesn't look good to me. Maybe there's a way to avoid that? Moreover, as far as I could tell functions from those packages didn't appear in the call graph after +RTS -p profiling. 2) Can I exclude a function from profiling? That probably means not assigning a cost centre to it. Typical case, I think. Database connect function is rather heavyweight (regarding time) compared to the rest of the code, and it takes up to 98% of time. So the rest of the picture is less informative than it could be. 3) Isn't it possible to have -p profiling data of the interrupted (ctrl-c) program? Thanks a lot! -- Daniil Elovkov

4 4

Re: [Haskell-cafe] Fwd: (Solved) cabal install with external gcc tool chain not the ghc-bundled one
by Duncan Coutts 12 Nov '09

12 Nov '09

On Thu, 2009-11-12 at 10:46 +0100, Daniel Kahlenberg wrote: > to answer this question myself how the use of another gcc is specified > with effect, I used the following options with the 'cabal install' call: > > --ghc-options="-pgmc e:/programme/ghc/mingw-gcc4/bin/gcc.exe -pgml > e:/programme/ghc/mingw-gcc4/bin/gcc.exe" > > See > http://www.haskell.org/ghc/docs/latest/html/users_guide/options-phases.html… > (searched in the wrong direction, too many trees...). Slightly tuned, > this should be the way to go in all similar cases. > > One thing I haven't considered yet is if the '--with-ld' and > '--with-gcc' options (if curious too, see logs in my previous mail - > Subject "[Haskell-cafe] caba install with external gcc toolchain not the > ghc-bundled one") only effect what gets written into the > setup-config/package.conf file or what other effects these have. Feel free to file a ticket about this. What makes me somewhat nervous is that the gcc you want to use for say .c files is not necessarily the same as the one ghc wants to use to compile .hc files or link stuff. This is particularly the case on Windows where ghc includes its own copy of gcc. Similarly on Solaris 10, ghc cannot use the /usr/bin/gcc because it's a hacked-up gcc that uses the Sun CC backend (which doesn't grok some of the crazy GNU C stuff that ghc uses). So it'd certainly be possible to have cabal's --with-gcc/ld override the ones that ghc uses by default, but the question is should it do so? I think it's worth asking the ghc hackers about this. Duncan

1 0

GHCI :reload module without erasing previous data
by Michael Serdar 11 Nov '09

11 Nov '09

I have a module that loads several dozen megabytes of static data into memory. This module is working properly. I have another module that I'm developing to analyze that data. As I make iterations on the analysis module, I would like it if the data loaded by the data module could stay in memory. In GHCI, every time I reload a module, it clears whatever data I have set with the "let" command. Is there a way to persist that data across calls to :reload, or is there an altogether different workflow I should be using for this?

2 1

Three patches for cabal
by Niklas Broberg 09 Nov '09

09 Nov '09

(Trying again since my previous patches were too big for the list.) While doing a survey[1] of the extensions registered with Cabal, I came across two warts in the list of constructors, and one constructor that should be deprecated. First there's the constructor called TransformListComp, which should really be named GeneralizedListComp, since the constructor should describe the extension and not the implementation scheme. Second there's the constructor NoMonoPatBinds, which actually describes the default Haskell 98 behavior, even if GHC has a different default. It's GHC's behavior that is the extension, so the constructor in cabal should really be named MonoPatBinds. Also, the PatternSignatures constructor has been deprecated in GHC and superceded by ScopedTypeVariables. The attached patches (three in one file) adds the proposed new constructors, deprecates the old ones, and adds documentation. Cheers, /Niklas

8 15