- Haskell-Cafe - Haskell.org

Re: [Haskell-cafe] Data.Binary suboptimal instance
by Henning Thielemann 24 May '09

24 May '09

Khudyakov Alexey schrieb: > On Saturday 23 May 2009 02:55:17 Antoine Latter wrote: >> Or you could go for the compromise position, where the list can be >> part of a complex data structure so you're not relying on EOF to find >> the end. >> > Interesting solution however it does not perform very nice. I wrote > microbenchmark > >> xs :: [Word32] >> xs = [1..(10^6)] > > Writing chunked list of Word32 >> B.writeFile "chunked" . toLazyByteString . putList putWord32be $ xs > real 0m4.311s > user 0m3.272s > sys 0m0.096s > > Reading chunked list of Word32 >> print . last . runGet (getList getWord32be) =<< B.readFile "chunked" > real 0m0.634s > user 0m0.496s > sys 0m0.012s > > > Writing stream of Word32 >> B.writeFile "stream" . encodeStream $ xs > real 0m0.391s > user 0m0.252s > sys 0m0.020s > > Reading stream of Word32 >> print . (last :: [Word32] -> Word32) . decodeStream =<< B.readFile "stream" > real 0m0.376s > user 0m0.248s > sys 0m0.020s > > > I didn'd do any profiling so I have no idea why writing is so slow. If you use top-level definition 'xs' the program might cache the list and second write is faster. You may change the order of tests in order check, whether that is the cause.

3 2

ANN: bsd-sysctl 1.0.3
by Maxime Henrion 24 May '09

24 May '09

I'm pleased to announce the release of bsd-sysctl 1.0.3, a package that provides a System.BSD.Sysctl module allowing access to the C sysctl(3) API. It should fully work on FreeBSD, NetBSD and Mac OS X platforms; it should also work on OpenBSD and Linux, although looking up sysctl's by name isn't supported there, and I believe the sysctl API doesn't return anything useful under Linux. The package is available here: http://hackage.haskell.org/cgi-bin/hackage-scripts/package/bsd-sysctl-1.0.3 As this is my first package on hackage, I'm eager to hear about any comments on this code. Cheers, Maxime

1 0

Defining a CPP macro when building documentation
by Maxime Henrion 24 May '09

24 May '09

Hello all, I have been exposed to a problem that have happened to others too, and since I have found a (scary) solution with the help of Duncan Coutts, I'm now sharing it with you. The reason I wanted to pass specific CPP flags to haddock was to allow the documentation of the full module to get built, even when only parts of the module would end up getting compiled because of restrictions on the build platform. In this mail, Duncan Coutts gives a very helpful bit of code to override the haddockHook and pass additional options to haddock : http://www.haskell.org/pipermail/haskell-cafe/2008-December/051785.html In my case, I needed something very similar, except that I needed to pass those additional options to hsc2hs which my code uses. This worked fine as long as the cabal commands sequence was ["configure","haddock"] and not ["configure","build","haddock"]. The reason for this is that in the latter case, the library wasn't being processed again, but output of previous runs of hsc2hs were reused. So I worked around this with an ugly hack that removes the .hs files before running haddock, in order to force re-processing! Here's the final code I ended up with : import Distribution.Simple import Distribution.Simple.LocalBuildInfo (LocalBuildInfo(withPrograms), buildDir) import Distribution.Simple.Program (userSpecifyArgs) import System.Directory import System.FilePath -- Define __HADDOCK__ when building documentation. main = defaultMainWithHooks simpleUserHooks { haddockHook = \pkg lbi h f -> do let progs = userSpecifyArgs "hsc2hs" ["-D__HADDOCK__"] (withPrograms lbi) removePreProcessedFiles (buildDir lbi) haddockHook simpleUserHooks pkg lbi { withPrograms = progs } h f } -- Horrible hack to force re-processing of the .hsc file. Otherwise -- the __HADDOCK__ macro doesn't end up being defined. removePreProcessedFiles :: FilePath -> IO () removePreProcessedFiles dir = removeFile (dir </> "System/BSD/Sysctl.hs") `catch` \_ -> return () Cheers, Maxime

1 0

`seq` and categories
by Jason Dusek 24 May '09

24 May '09

On the IRC channel a few days ago, it was said that, as long as we allow `seq`, Hask is not a valid category. Doesn't this basically mean that a very large amount of Haskell -- anything with strictness annotations -- can not be described in a category Hask? -- Jason Dusek

2 1

Re: Re: Re: Re: [Haskell-cafe] Re: A problem with par and modules boundaries...
by Mario Blažević 24 May '09

24 May '09

> I recommend using -ddump-simpl, as it produces more readable output. > >> Actually, I can't see any effect of that pragma in the >> core files whatsoever, but it certainly has effect on >> run time. > > How about diffing the whole core output (and using -ddump-simpl). If > there's a performance difference then there must be a difference in the > core code too. Ok. $ ghc-6.11.20090421 --make primes-test.hs -threaded -O2 -ddump-simpl &> main.simpl $ time ./primes-test +RTS -N2 4001 real 0m9.636s user 0m18.201s sys 0m0.088s $ ghc-6.11.20090421 --make primes-test.hs -threaded -O2 -ddump-simpl &> imported.simpl $ time ./primes-test +RTS -N24001 real 0m17.547s user 0m17.453s sys 0m0.052s I can't exactly use diff because the generated identifier names are not the same, but after poring over with Emacs ediff I have found only one difference that's not attributable to identifiers: $diff main.simpl imported.simpl ... 223c232 < a_s1rs [ALWAYS Just L] :: GHC.Integer.Internals.Integer --- > a_s1sV [ALWAYS Just S] :: GHC.Integer.Internals.Integer ... Does this S vs. L difference have anything to do with strictness and laziness? That line is a part of the `Main.test' definition: $ diff -C 3 main.simpl imported.simpl *** 217,244 **** [Arity 2 Str: DmdType LL] Main.test = ! \ (n1_ahR :: GHC.Integer.Internals.Integer) ! (n2_ahT :: GHC.Integer.Internals.Integer) -> let { ! a_s1rs [ALWAYS Just L] :: GHC.Integer.Internals.Integer LclId [Str: DmdType] ! a_s1rs = Data.List.prod1 (Main.factors2 (Data.List.prod1 ! (GHC.Num.up_list Main.lvl Main.lvl n1_ahR) Data.List.lvl1)) Data.List.lvl1 } in ! case GHC.Prim.par# @ GHC.Integer.Internals.Integer a_s1rs of _ { __DEFAULT -> case Data.List.prod1 (Main.factors2 (Data.List.prod1 ! (GHC.Num.up_list Main.lvl Main.lvl n2_ahT) Data.List.lvl1)) Data.List.lvl1 ! of x1_aUS { __DEFAULT -> ! case GHC.Real.$wdivMod x1_aUS a_s1rs of _ { (# ww1_aUo, _ #) -> ! ww1_aUo } } } >> Or do you mean to say that *your* installation of GHC >> behaves the same when the function `parallelize' is defined in >> the same module and when it's imported? > > Yes, exactly. With both ghc-6.10.1 and a very recent build of ghc-6.11 As you can see, I'm using the 6.11.20090421 build. I looked at recent ones, but the Linux builds seem to be less stable in May. I have the same results (though I didn't use -ddump-simpl) with 6.8.2. Can you recommend a different version to try?

1 0

HPC Website & FAQ
by Dominic Steinitz 24 May '09

24 May '09

There's a nice website for HPC but it looks a bit out of date. http://projects.unsafeperformio.com/hpc/ I wanted to send a patch to the FAQ for using HPC with .lhs files (you have to run ghc -E to generate .hs files and strip some of the the lines ghc generates: "{-# LINE 1 "ASNTYPE.lhs" #-}" & "#line 1 "ASNTYPE.lhs""). Would it be possible for the author of HPC to make the website into a darcs repo so that we send in patches? Apparently the Cabal website is managed like that and it sounds like a good way of doing things in general to me. Thanks, Dominic.

1 0

Re: Re: Re: [Haskell-cafe] Re: A problem with par and modules boundaries...
by Duncan Coutts 24 May '09

24 May '09

On Sat, 2009-05-23 at 20:42 -0400, Mario Blažević wrote: > On Sat 23/05/09 2:51 PM , Duncan Coutts duncan.coutts(a)worc.ox.ac.uk sent: > > On Sat, 2009-05-23 at 13:31 -0400, Mario Blažević wrote: > > ... > >> So the function is not strict, and I don't understand > >> why GHC should evaluate the arguments before the call. > > > > Right, it's lazy in the first and strict in the second argument. As far > > as I can see we have no evidence that is is evaluating anything before > > the call. > > > When I look at the Core definition of `test', it begins with > > > \ (n1axl::integer:GHCziIntegerziInternals.Integer) > (n2axn::integer:GHCziIntegerziInternals.Integer) -> > %let as1sU :: integer:GHCziIntegerziInternals.Integer = > base:DataziList.prod1 > (main:Main.factors2 > (base:DataziList.prod1 > (base:GHCziNum.upzulist main:Main.lvl main:Main.lvl n1axl) > base:DataziList.lvl1)) > base:DataziList.lvl1 > %in %case integer:GHCziIntegerziInternals.Integer (ghczmprim:GHCziPrim.parzh > @ > integer:GHCziIntegerziInternals.Integer > as1sU) > %of (dsapq::ghczmprim:GHCziPrim.Intzh) I recommend using -ddump-simpl, as it produces more readable output. > To my untrained eyes, this looks like it's evaluating > > > product $ factors $ product [1..n1]) > > which is the first argument to `parallelize'. That core code is doing: let blah = product $ factors $ product thelist in case par# blah of _ -> ... So it's calling the primitive par# but nothing is forced to WHNF (except the unused dummy return value of par#). > I assume that %case in Core evaluates the argument to WHNF, just like > case in Haskell. Yes. (In fact case in core always reduces to WHNF, where as in Haskell case (...) of bar -> (...) does not force anything.) > Then again, I could be completely misinterpreting what Core is, because > I can't find any call to `parallelize' before or after that. It appears > to be inlined in Core, regardless of whether the pragma > > > {-# INLINE parallelize #-} > > is there or not. Yes, because even without the pragma, ghc decides it's profitable to inline. > Actually, I can't see any effect of that pragma in the > core files whatsoever, but it certainly has effect on run time. How about diffing the whole core output (and using -ddump-simpl). If there's a performance difference then there must be a difference in the core code too. > Or do you mean to say that *your* installation of GHC behaves > the same when the function `parallelize' is defined in the same module and when > it's imported? Yes, exactly. With both ghc-6.10.1 and a very recent build of ghc-6.11 Duncan

1 0

hackage version scheme survey
by brian＠lorf.org 24 May '09

24 May '09

It's not good to specify open version ranges in cabal build-depends like foo >= 1.1 because the foo maintainer will eventually release 2.0, containing API-breaking changes, and the build will fail. If you depend on foo 1.1, you can specify == 1.*, no problem. But if you depend on a package that uses a version scheme like 1.2.3 or 1.2.3.4, it's not clear (at least to me) where the major/minor division is. I did a survey of version schemes in 'cabal list': 484 packages use a.b. 630 packages use a.b.c. 180 packages use a.b.c.d. The rest: two packages use just one field for their versions; one uses five; one uses six. Maybe most of the a.b people are thinking major.minor, and most of the a.b.c people are thinking breaking.feature.implementation like the "rational" RubyGems scheme described in http://rubygems.org/read/chapter/7#page24 , but I don't know. It makes it hard to describe dependencies. Will there be a standard versioning scheme sometime?

6 6

Cabal, Time & GHC 6.10.2
by Dominic Steinitz 24 May '09

24 May '09

I get > dom@linux-6ofq:~/asn1> runghc Setup.hs configure > Configuring PER-0.0.20... > Setup.hs: At least the following dependencies are missing: > time -any && -any but I have time > dom@linux-6ofq:~/asn1> ghc-pkg list | grep time > old-locale-1.0.0.1, old-time-1.0.0.2, packedstring-0.1.0.1, > time-1.1.2.4 I think I can see why cabal isn't finding it: > ghc-pkg dump --global | grep time-1.1.2.4 finds nothing and I believe that is what cabal uses to find things. What's not clear is what I do to fix things. Go back to 6.10.1? Thanks, Dominic. > dom@linux-6ofq:~/asn1> ghc --version > The Glorious Glasgow Haskell Compilation System, version 6.10.2 > dom@linux-6ofq:~/asn1> ghc-pkg --version > GHC package manager version 6.10.2 Here's my .cabal file. > Name: PER > Version: 0.0.20 > License: BSD3 > Author: Dominic Steinitz > Maintainer: dominic.steinitz(a)blueyonder.co.uk > Copyright: Dominic Steinitz 2003 - 2009 > Stability: Alpha > Category: Language > Homepage: http://www.haskell.org/asn1 > Synopsis: ASN.1 PER support for Haskell > Description: A formal and executable specification of the Packed Encoding Rules (PER) > for ASN.1 > build-depends: binary-strict == 0.4.2, > bytestring, > mtl, > containers, > time, > pretty, > base > build-type: custom > > Exposed-Modules: Language.ASN1.PER.Integer > Language.ASN1.PER.GenerateC > > Executable: PERTest > Main-Is: PERTest.hs

2 3

Re: Re: Re: [Haskell-cafe] Re: A problem with par and modules boundaries...
by Mario Blažević 24 May '09

24 May '09

On Sat 23/05/09 2:51 PM , Duncan Coutts duncan.coutts(a)worc.ox.ac.uk sent: > On Sat, 2009-05-23 at 13:31 -0400, Mario Blažević wrote: > ... >> So the function is not strict, and I don't understand >> why GHC should evaluate the arguments before the call. > > Right, it's lazy in the first and strict in the second argument. As far > as I can see we have no evidence that is is evaluating anything before > the call. When I look at the Core definition of `test', it begins with \ (n1axl::integer:GHCziIntegerziInternals.Integer) (n2axn::integer:GHCziIntegerziInternals.Integer) -> %let as1sU :: integer:GHCziIntegerziInternals.Integer = base:DataziList.prod1 (main:Main.factors2 (base:DataziList.prod1 (base:GHCziNum.upzulist main:Main.lvl main:Main.lvl n1axl) base:DataziList.lvl1)) base:DataziList.lvl1 %in %case integer:GHCziIntegerziInternals.Integer (ghczmprim:GHCziPrim.parzh @ integer:GHCziIntegerziInternals.Integer as1sU) %of (dsapq::ghczmprim:GHCziPrim.Intzh) To my untrained eyes, this looks like it's evaluating > product $ factors $ product [1..n1]) which is the first argument to `parallelize'. I assume that %case in Core evaluates the argument to WHNF, just like case in Haskell. Then again, I could be completely misinterpreting what Core is, because I can't find any call to `parallelize' before or after that. It appears to be inlined in Core, regardless of whether the pragma > {-# INLINE parallelize #-} is there or not. Actually, I can't see any effect of that pragma in the core files whatsoever, but it certainly has effect on run time. >> Does anybody know of a pragma or another way to make a >> function *non-strict* even if it does always evaluate its argument? >> In other words, is there a way to selectively disable the strictness >> optimization? > > Yes, which is what pseq and par already do. > > If there's a bug, we need to reproduce it and report it. I cannot > reproduce it. If you mean that you can't reproduce anything that's contrary to the specification, that's not saying much: there are practically no guarantees on what `par' is supposed to accomplish. If you mean you can't reproduce anything you wouldn't expect, pray explain what is going on, because everybody else seems to be surprised. Or do you mean to say that *your* installation of GHC behaves the same when the function `parallelize' is defined in the same module and when it's imported?

1 0