- Libraries - Haskell.org

Re: #562: cabal-install update fails going through a HTTP proxy
by Valery V. Vorotyntsev 29 Nov '09

29 Nov '09

Sigbjorn Finne <sigbjorn.finne(a)gmail.com> wrote: > > Despite repeated attempts from this end, I am unable to reproduce > this via local proxies. > > Hence chasing down whatever problem is biting you (and others) is > tricky without a repro case. I'll try this change tomorrow: -----BEGIN DIFF----- diff --git a/Network/TCP.hs b/Network/TCP.hs index 7d3dbe7..04b8e2a 100644 --- a/Network/TCP.hs +++ b/Network/TCP.hs @@ -34,7 +34,7 @@ module Network.TCP import Network.BSD (getHostByName, hostAddresses) import Network.Socket - ( Socket, SockAddr(SockAddrInet), SocketOption(KeepAlive) + ( Socket, SockAddr(SockAddrInet), SocketOption(KeepAlive, Linger) , SocketType(Stream), inet_addr, connect , shutdown, ShutdownCmd(..) , sClose, setSocketOption, getPeerName @@ -189,6 +189,7 @@ openTCPConnection_ :: BufferType ty => String -> Int -> Bool - openTCPConnection_ uri port stashInput = do s <- socket AF_INET Stream 6 setSocketOption s KeepAlive 1 + setSocketOption s Linger 5 hostA <- getHostAddr uri let a = SockAddrInet (toEnum port) hostA catchIO (connect s a) (\e -> sClose s >> ioError e) -----END DIFF----- | If there is still data waiting to be transmitted over the | connection, normally `close' tries to complete this transmission. | You can control this behavior using the `SO_LINGER' socket option to | specify a timeout period; see *note _Socket Options_. [http://www.gnu.org/s/libc/manual/html_node/Closing-a-Socket.html] -- vvv

1 0

ANNOUNCE: deepseq-1.0.0.0
by Simon Marlow 28 Nov '09

28 Nov '09

I've just uploaded deepseq-1.0.0.0 to Hackage http://hackage.haskell.org/package/deepseq This provides a DeepSeq class with a deepseq method, equivalent to the existing NFData/rnf in the parallel package. I'll be using this in a newly revamped parallel package, which I hope to upload shortly. Cheers, Simon

13 23

Advance warning to package authors of breakage
by Duncan Coutts 28 Nov '09

28 Nov '09

package authors, I'm preparing to release two new versions of cabal-install. The following packages are broken, and the new cabal-install releases will mean that more users run into the breakage than do so currently. All these packages claim to work with base 3 *and* with base 4, however they actually only work with base 3. The new cabal-install will build them against base 4 and so users will notice that they are broken. I'm emailing the whole libs list because while I was able to build most of hackage, I could not build all of it, so there may be other packages with this same problem. FileManipCompat-0.14 type-level-0.2.3 compact-string-0.3.1 ghc-core-0.5 hsmtpclient-1.0 pointful-1.0.1 show-0.3.3 These packages all use "build-depends: base >= 3 && < 4" (except for ghc-core which cheekily uses "build-depends: base < 10" --- a practise we will ban if it becomes any more popular). Current versions of cabal-install choose base version 3 in this case, the new cabal-install will choose base version 4 in this case. The reason for the somewhat odd current behaviour was to help with packages that only said "build-depends: >=3". Now that we require an upper bound too, the new behaviour is to only apply the backwards compat preferences for version ranges that are unbounded above. For ones that are bounded above we go back to picking the highest version by default. Duncan

2 4

Re: Cabal and Hsc2hs Build Phase Options
by Duncan Coutts 25 Nov '09

25 Nov '09

On Fri, 2009-11-20 at 09:44 -0500, Matthew William Cox wrote: > On Fri, Nov 20, 2009 at 10:00:10AM +0000, Duncan Coutts wrote: > > The builder is in a position to know if special flags are needed. Also, > > since the builder controls the build environment they could use special > > flags anyway by changing the $PATH and using wrapper scripts, so there's > > no point in not letting the builder do what they want. > > The author is also in a position to know that the same flags need to be > used in two different build phases. In this case, I'm using > autoconfUserHooks to find out how to use the system's MPI library. A > builder should direct the autoconf script to the correct library (if there > are multiple) using an environment variable. In either case, autoconf > hands me a compiler wrapper which needs to be used to compile and link all C > code (and the FFI using code). There's no other portable way to use MPI, > like pkgconfig or whathaveyou to suss out the magic CFLAGS and LDFLAGS > needed. Wow, that's fairly horrible. I wonder what the MPI people were thinking. You can't have two dependencies that use this trick since they would fight over which cc to use. Does it let you tell it which gcc to wrap or does it try and pick the gcc itself? > Anyways, from my perspective as an author that wants to provide the > simplist building experience for someone else, I'd rather be able to > tell them in the docs to run > runhaskell Setup.hs configure && runhaskell Setup.hs build > rather than to 1) runhaskell Setup.hs configure, 2) read the generated > buildinfo script to determine the compiler and linker passed to ghc, 3) > call runhaskell Setup.hs build --hsc2hs-options="-cc=$COMPILER > --ld=$COMPILER" where $COMPILER is the exact compiler wrapper used by > Ghc, and failing to do this may result in failure or having a > schizophrenic build compiled against one library and linked against > another. Right, so you'll need a custom Setup.hs. It will collect the info from your ./configure script. It will need to update the ProgramConfiguration environment that Cabal uses. The ProgramConfiguration tells Cabal where various named programs live and if they need any extra flags (this is the information store that the --with-prog= and --prog-options= flags modify). You'll want to set the gcc to be used. That will propagate automatically to the hsc2hs call, though not to ghc (though arguably it should). You'll need to modify the program configuration to add in the appropriate flags to the ghc calls. > The user arguments at the command line should always override the ones > in a configuration file of course. That gets you the best of both > worlds. I'm not quite sure how that should work. The initial ProgramConfiguration you get will contain the info from the user command line. Can you tell the MPI thing which gcc it should wrap? If so then you can respect the user arguments, otherwise I don't really see how you can and also make it work. > tl;dr: you can have your cake and eat it too if the user's command line > arguments always override the equivalent configuration file directive. > > > Now I take it that you really want to use your mpicc just for hsc2hs and > > not for all cases where cabal (or programs that cabal calls) call gcc > > right? I ask because we've got another open feature request ticket that > > asks that cabal --with-gcc=blah should pass that gcc on to ghc, hsc2hs, > > etc etc. > > I need it for all phases of the build (GHC, hsc2hs, etc.) Right now, > it's override GHC using ghc-options: -pgmc ... -pgml ... but it would be > even better if there was one central way. We've not quite decided if cabal blah --with-gcc= should set just the gcc that cabal calls directly, or if cabal should also instruct all programs that call gcc to use that same one. What makes me most nervous is that ghc has a fairly intimate relationship with gcc and especially when it comes to gcc flags, not all gcc flags are compatible with ghc. > Again, a --with-linker would be nice to add, and especially, a > configuration file directive to set it statically based on an autoconf > check with the option to be overridden by a user's command line > arguments. Would it be enough to have a central setting for which gcc to use, as mentioned above? > > BTW, as a short-term workaround you can always use the wrapper script > > trick: --with-hsc2hs=hsc2hs.sh > > I take it there's no way of doing this statically? Not in the .cabal file. You can do what you like in the Setup.hs though. Duncan

2 1

Proposal (Trac ticket #3671): Add takeRec, genericTakeRec and spanRec to Data.List
by Philip K.F. Hölzenspies 25 Nov '09

25 Nov '09

Dear all, After defining these for the umpeenth time, I would like to add the functions takeRec, genericTakeRec and spanRec to Data.List, as per http://hackage.haskell.org/trac/ghc/ticket/3671 Suggested discussion time: 2 weeks. Regards, Philip

8 21

export toDescList from Data.Map
by Evan Laforge 25 Nov '09

25 Nov '09

So, I mentioned this a long time ago but didn't get any responses and then I got distracted. So this time I added a ticket and patch and everything: 2580 Here's the text: It's even implemented, but not exported. Without this, there's apparently no way to iterate over a map from high to low, since foldl is also not exported. Also, I implemented a toDescList for IntMap, and could do the same for Set, and could add those patches, if people agree it's a good idea. In addition, is there any problem with exporting foldl aka descending for the three modules? I implemented a foldl for IntMap and it seems to work, but I'd want someone smarter to look for problems with it. In addition, Set and Map are inconsistent with folds: Set exports 'fold' and says "no order is guaranteed", but it's just defined to foldr, which is in ascending order. Set.foldr itself is not exported. Meanwhile, Map also exports fold, but heavily implies from the doc that it's guaranteed to be ascending. Would it restrict implementation changes too much to export a descending fold for all three? Other languages that have tree oriented general purpose maps all let you start anywhere and iterate in either direction. Anyway, is 2-3 weeks ok for discussion period? The library submission wiki doesn't say how long.

6 13

Could iteratee depend on mtl instead of transformers?
by Valery V. Vorotyntsev 24 Nov '09

24 Nov '09

Hi, Iteratee [1] library depends on transformers [2]. The latter exposes `Control.Monad.Trans' module. [1] http://hackage.haskell.org/package/iteratee-0.3.1 [2] http://hackage.haskell.org/package/transformers-0.1.4.0 So does mtl [3] which is also installed on my system. [3] http://hackage.haskell.org/package/mtl-1.1.0.2 As a result, I cannot compile a program using Data.Iteratee, unless one of the packages is hidden. $ ghci GHCi, version 6.10.4: http://www.haskell.org/ghc/ :? for help Loading package ghc-prim ... linking ... done. Loading package integer ... linking ... done. Loading package base ... linking ... done. Prelude> :m + Control.Monad.Trans Could not find module `Control.Monad.Trans': it was found in multiple packages: mtl-1.1.0.2 transformers-0.1.4.0 $ ghci hide transformers-0.1.4.0 Having transformers hidden, I need to copy-paste MonadTrans and MonadIO instances [4] from Data.Iteratee.Base to my code, otherwise I get errors: Prelude> :load "/home/vvv/job/cdr.mscp/Codec/Binary/MSCP.hs" [1 of 1] Compiling Codec.Binary.MSCP ( /home/vvv/job/cdr.mscp/Codec/Binary/MSCP.hs, interpreted ) /home/vvv/job/cdr.mscp/Codec/Binary/MSCP.hs:88:7: Could not deduce (MonadIO (IterateeG [] Word8 m)) from the context (MonadIO m) arising from a use of `liftIO' at /home/vvv/job/cdr.mscp/Codec/Binary/MSCP.hs:88:7-12 Possible fix: add (MonadIO (IterateeG [] Word8 m)) to the context of the type signature for `warn' or add an instance declaration for (MonadIO (IterateeG [] Word8 m)) In the first argument of `(.)', namely `liftIO' In the expression: liftIO . hPutStrLn stderr . ("*WARNING* " ++) In the definition of `warn': warn = liftIO . hPutStrLn stderr . ("*WARNING* " ++) Failed, modules loaded: none. [4] http://bit.ly/1TT37E With MonadTrans & MonadIO instances duplicated, I need to use `-fno-warn-orphans' GHC option in order not to see "orphan instance" warnings. Or I can just update iteratee.cabal and make it depend on mtl instead of transformers: --- iteratee.cabal.orig 2009-11-09 20:40:29.000000000 +0200 +++ iteratee.cabal 2009-11-16 18:32:36.637288144 +0200 @@ -59,7 +59,7 @@ containers >= 0.2 && < 0.4, extensible-exceptions >= 0.1 && < 0.2, haskell98 >= 1 && < 2, - transformers >= 0.1.4 && < 0.2 + mtl >= 1.1 && < 1.2 exposed-modules: Data.Iteratee This allows me to remove Monad{Trans,IO} instances from my code and forget about `-fno-warn-orphans' flag. But now my package cannot claim its dependency on iteratee-0.3.1 any more. Thus, the question is: are there any transformers-specific features iteratee package needs, or can it just go with mtl? * * * I suspect mtl to be more popular... And actual "popularity" can be measured. I'll try to do that using cabal-query [5] tool; it allows to list "reverse dependencies" [6]. [5] http://hackage.haskell.org/package/cabal-query [6] http://hackage.haskell.org/trac/hackage/ticket/576#comment:2 Cheers. -- vvv

14 26

Re: [patch] #562: cabal-install update fails going through a HTTP proxy
by Valery V. Vorotyntsev 23 Nov '09

23 Nov '09

> Valery V. Vorotyntsev wrote: >> Sorry, Sigbjorn, it didn't work. Sigbjorn Finne wrote: > Thanks - that's good, now we know :) I'm not so sure any more. >> I've rolled back to patched HTTP-4000.0.8, the one with connection >> closing commented out in `sendHTTP_notify'. I need this proxy >> downloading thing to just work... Cabal client reinstalled with "patched" HTTP-4000.0.8 was still having occasional failures today. My proxy-POC.hs always succeeded, while cabal failed quite often with familiar cabal: Codec.Compression.Zlib: premature end of compressed stream This means that proxy-POC.hs is not trustworthy: it doesn't expose the bug in 100% cases. I am going to roll back even further tomorrow: I'll patch [1] cabal-install to use strict ByteStrings and see if /that/ does the job. (I'm not sure in anything any more.) [1] http://hpaste.org/fastcgi/hpaste.fcgi/view?id=9447#a9504 > I suspect tempering the notion of close and doing a socket shutdown > is the way forward here. So do I... If my suspicions are of any significance after several delusive "Wolf!" cries [2,3]. [2] http://hackage.haskell.org/trac/hackage/ticket/562#comment:9 [3] http://trac.haskell.org/http/ticket/8#comment:7 > I'll see if I can play with this some tonight. I've installed squid proxy server. Gotta make this bug exposable... -- Regards, vvv

1 1

Cabal and Hsc2hs Build Phase Options
by Matthew William Cox 20 Nov '09

20 Nov '09

Hello, Firstly, is there any way to specify (statically) options to be passed to Hsc2hs to be used during the build phase? Ideally, extending the --$PROGNAME-options command line argument to a cabal directive would be ideal, so that hsc2hs-options: would be a valid directory for use in .buildinfo/.cabal files. Is there a way to specificy the compiler and linker to be used in the invocation of hsc2hs? Secondly, there is a problem with how command line arguments are passed to hsc2hs. Running build with the command line argument: --hsc2hs-options="--cc=mpicc --ld=mpicc" -v yields the following call to hsc2hs: > Creating dist/build/dllg/dllg-tmp/System (and its parents) > /usr/bin/hsc2hs --cc=mpicc -ld=mpicc --cc=/usr/bin/gcc --ld=/usr/bin/gcc --cflag=-D__G... Two compilers and two linkers are passed to Hsc2hs, but the later arguments (the default compilers) take precedence, as can be seen if the /bin/false is passed instead of mpicc. Cheers, Matt

2 2

ANNOUNCE: parallel-2.0.0.0
by Simon Marlow 19 Nov '09

19 Nov '09

I've just uploaded parallel-2.0.0.0 to Hackage. If you're using Strategies at all, I'd advise updating to this version of the parallel package. It's not completely API compatible, but if you're just using the supplied Strategies such as parList, the changes should be relatively minor. The Haddock docs include a full description of the changes, reproduced below. Haddock docs are also here, until Hackage catches up: http://www.haskell.org/~simonmar/parallel/index.html The Strategy-using programs I've tried so far go faster. Enjoy! Cheers, Simon --------------------------------------------------------------------- Version 1.x The original Strategies design is described in <http://www.macs.hw.ac.uk/~dsg/gph/papers/html/Strategies/strategies.html> and the code was written by Phil Trinder, Hans-Wolfgang Loidl, Kevin Hammond et al. Version 2.x Later, during work on the shared-memory implementation of parallelism in GHC, we discovered that the original formulation of Strategies had some problems, in particular it lead to space leaks and difficulties expressing speculative parallelism. Details are in the paper \"Runtime Support for Multicore Haskell\" <http://www.haskell.org/~simonmar/papers/multicore-ghc.pdf>. This module has been rewritten in version 2. The main change is to the 'Strategy a' type synonym, which was previously @a -> Done@ and is now @a -> a@. This change helps to fix the space leak described in \"Runtime Support for Multicore Haskell\". The problem is that the runtime will currently retain the memory referenced by all sparks, until they are evaluated. Hence, we must arrange to evaluate all the sparks eventually, just in case they aren't evaluated in parallel, so that they don't cause a space leak. This is why we must return a \"new\" value after applying a 'Strategy', so that the application can evaluate each spark created by the 'Strategy'. The simple rule is this: you /must/ use the result of applying a 'Strategy' if the strategy creates parallel sparks, and you should probably discard the the original value. If you don't do this, currently it may result in a space leak. In the future (GHC 6.14), it will probably result in lost parallelism instead, as we plan to change GHC so that unreferenced sparks are discarded rather than retained (we can't make this change until most code is switched over to this new version of Strategies, because code using the old verison of Strategies would be broken by the change in policy). The other changes in version 2.x are: * Strategies can now be defined using a convenient Applicative type Eval. e.g. parList s = unEval $ traverse (Par . s) * 'parList' has been generalised to 'parTraverse', which works on any 'Traversable' type. * 'parList' and 'parBuffer' have versions specialised to 'rwhnf', and there are transformation rules that automatically translate e.g. @parList rwnhf@ into a call to the optimised version. * 'NFData' is deprecated; please use the @DeepSeq@ class in the @deepseq@ package instead. Note that since the 'Strategy' type changed, 'rnf' is no longer a 'Strategy': use 'rdeepseq' instead.

3 4