- Libraries - Haskell.org

Testing containers outside of ghc source tree
by Gracjan Polak 10 Nov '11

10 Nov '11

Hi all, Recent developments in containers made me to give it a spin, too. I'd like to compile and test containers package. I got the sources, compiled. So far so good. Then the testsuite inside source code seems to require GHC testing framework. This seems a bit heavy. Anyway I got recent GHC sources, placed containers under libraries/containers-ex. Running 'make TEST_HC=ghc' in tests directory fails with: *** unexpected failure for sequence001(threaded1) =====> sequence001(threaded2) 4 of 4 [0, 30, 0] cd . && '/usr/bin/ghc' -fforce-recomp -dcore-lint -dcmm-lint -dno-debug-output -no-user-package-conf -rtsopts -fno-ghci-history -o sequence001 sequence001.hs -O -threaded -eventlog -package containers >sequence001.comp.stderr 2>&1 Compile failed (status 256) errors were: ghc: unrecognised flags: -fno-ghci-history Usage: For basic information, try the `--help' option. What I aim for is to have a fast/short way of doing some experimental changes and running only the containers testsuite with my own changes included. How do I get to doing this? -- Gracjan

5 5

Removing lazy IO? (Was: Re: Proposal: Stop enforcing single-writer-multi-reader file access)
by Jason Dagit 10 Nov '11

10 Nov '11

On Tue, Nov 8, 2011 at 7:02 PM, Ian Lynagh <igloo(a)earth.li> wrote: > On Tue, Nov 08, 2011 at 01:07:23PM +0000, Duncan Coutts wrote: >> >> We should take the same approach with openFile. Currently we have: >> >> openFile :: FilePath -> IOMode -> IO Handle >> >> I suggest changing the IOMode for a record of options which would >> include the IO mode (read/write/read-write/etc), and other things like >> file locking, default create permissions, create exclusive etc. Again, >> we'd use a default value and override extra options as necessary. > > I've just reread the thread, and I think the best way forwards would be > to make the original proposed change now, and for the above change to be > proposed separately. The other separate proposal that I would like to propose (at some time in the future) is a thorough removal of "lazy IO" from base. Or at least make it not available in the Prelude (ie., people have to ask for it if they want it). Perhaps this would require the support of Haskell'? Jason

7 8

Proposal: Stop enforcing single-writer-multi-reader file access
by Ian Lynagh 10 Nov '11

10 Nov '11

Hi all, Haskell currently requires that multiple-reader single-writer locking is used on files. I understand the motivation for this was to try to protect people from running into problems due to lazy IO, but in practice I think it just causes more problems than it solves. The decision may also have been influenced by the belief that this behaviour is enforced for all programs on Windows, but I am told that it is possible to get lock-free behaviour on Windows too. I propose that we remove all the automatic locking from the libraries, and let the user do any locking that they wish to do themselves. Attached are patches for ghc, base, process and unix. (not yet tested on Windows). Suggested discussion period: Until 9 Nov 2011 Related GHC ticket: http://hackage.haskell.org/trac/ghc/ticket/4363 Thanks Ian

16 29

[PATCH] haskell-src build fix for ghc-7.2.1
by Conrad Parker 09 Nov '11

09 Nov '11

Hi, the attached patch for haskell-src conditionally depends on haskell98 only for ghc < 7.2.1. A similar patch was sent to libraries@ by Adam Sampson on Aug 14, removing the haskell98 dependency entirely as "Recent versions of Happy don't require it any more", which may be better. (There has been no response to Adam's mail, perhaps listing libraries@ as maintainer is not so useful). Conrad.

2 3

Proposal #3339: Add (+>) as a synonym for mappend
by Bryan O'Sullivan 08 Nov '11

08 Nov '11

Ticket: http://hackage.haskell.org/trac/ghc/ticket/3339 >From the ticket: This proposal was, I think, originally suggested by Jules Bean. The idea is to add two functions to the Data.Monoid module, (+>) and (<+), corresponding to different uses of mappend. These should not be methods of the Monoidtypeclass, but top-level functions. I hope (but slightly doubt) that the visual nature of the two operators might help to counter the thought that monoids are just for gluing things together. (+>) :: (Monoid a) => a -> a -> a a +> b = a `mappend` b (<+) :: (Monoid a) => a -> a -> a a <+ b = b `mappend` a infixl 4 +> infixl 4 <+ Proposed deadline: two weeks. If this looks reasonable, I'll attach darcs patches.

30 75

Proposal: Add {to,from}Strict conversion functions between strict and lazy ByteStrings
by Herbert Valerio Riedel 07 Nov '11

07 Nov '11

I propose to add optimized {to,from}Strict conversion functions between strict and lazy ByteStrings to the Data.ByteString.Lazy API. Discussion deadline: 2 weeks from now (12 November) = Current State = The current Data.ByteString.Lazy API doesn't provide direct conversion functions to/from single strict ByteStrings Currently, there are only `fromChunks` and `toChunks`, by which convert to/from a list of strict ByteStrings. A possible reference implementation of the missing conversion functions is: fromStrict = BL.fromChunks . (:[]) and toStrict = B.concat . BL.toChunks == The Issues == The lack of `fromStrict`/`toStrict` in the Data.ByteString.Lazy API has the following issues: - Convenience: If the single-strict-bytestring conversion is often needed, one tends to define module- or package-local helper functions for convenience/readability to perform the desired conversion. This violates the DRY principle. - Principle of least suprise: Might be confusing to users new to `Data.ByteString.Lazy` why there is no direct conversion. - Symmetry with `Data.Text.Lazy` API which does provide such single-strict-text conversion functions (`fromStrict`/`toStrict`) - Performance: The above provided "naive" `toStrict` definition has a roughly 2 to 4 times higher overhead than a manually fused version (which was kindly provided by Bas van Dijk -- whom I'd like to thank for providing me with the optimized versions of toStrict and fromStrict) -- see end of this mail for criterion benchmark code and results = Proposed Enhancement = Enhance the Data.ByteString.Lazy API by adding the following conversion functions (suggestions for improvements are highly welcome): -- see benchmark code at end of mail for the qualified imports -- |/O(n)/ Convert a strict ByteString into a lazy ByteString. fromStrict :: B.ByteString -> BL.ByteString fromStrict = flip BLI.chunk BLI.Empty -- |/O(n)/ Convert a lazy ByteString into a strict ByteString. toStrict :: BL.ByteString -> B.ByteString toStrict lb = BI.unsafeCreate len $ go lb where len = BLI.foldlChunks (\l sb -> l + B.length sb) 0 lb go BLI.Empty _ = return () go (BLI.Chunk (BI.PS fp s l) r) ptr = withForeignPtr fp $ \p -> do BI.memcpy ptr (p `plusPtr` s) (fromIntegral l) go r (ptr `plusPtr` l) == Benchmark Code & Results == ------------------------------------------------------------------------ {-# LANGUAGE OverloadedStrings #-} import Criterion import Criterion.Main import qualified Data.ByteString as B import qualified Data.ByteString.Internal as BI import qualified Data.ByteString.Lazy as BL import qualified Data.ByteString.Lazy.Internal as BLI import Foreign.ForeignPtr import Foreign.Ptr toStrict1 :: BL.ByteString -> B.ByteString toStrict1 = B.concat . BL.toChunks toStrict2 :: BL.ByteString -> B.ByteString toStrict2 lb = BI.unsafeCreate len $ go lb where len = BLI.foldlChunks (\l sb -> l + B.length sb) 0 lb go BLI.Empty _ = return () go (BLI.Chunk (BI.PS fp s l) r) ptr = withForeignPtr fp $ \p -> do BI.memcpy ptr (p `plusPtr` s) (fromIntegral l) go r (ptr `plusPtr` l) main :: IO () main = do let lbs1 = "abcdefghij" lbs2 = BL.fromChunks (replicate 10 "abcdefghij") lbs3 = BL.fromChunks (replicate 1000 "abcdefghij") -- force evaluation of lbs{1,2,3} and verify validity print $ toStrict1 lbs1 == toStrict2 lbs1 print $ toStrict1 lbs2 == toStrict2 lbs2 print $ toStrict1 lbs3 == toStrict2 lbs3 defaultMain [ bgroup "toStrict" [ bench "simple #1" $ whnf toStrict1 lbs1 , bench "simple #2" $ whnf toStrict1 lbs2 , bench "simple #3" $ whnf toStrict1 lbs3 , bench "optimized #1" $ whnf toStrict2 lbs1 , bench "optimized #2" $ whnf toStrict2 lbs2 , bench "optimized #3" $ whnf toStrict2 lbs3 ] ] {- True True True warming up estimating clock resolution... mean is 2.302557 us (320001 iterations) found 2039 outliers among 319999 samples (0.6%) 1658 (0.5%) high severe estimating cost of a clock call... mean is 54.99870 ns (14 iterations) found 1 outliers among 14 samples (7.1%) 1 (7.1%) low mild benchmarking toStrict/simple #1 mean: 28.96077 ns, lb 28.89527 ns, ub 29.01562 ns, ci 0.950 std dev: 305.8466 ps, lb 262.1008 ps, ub 345.6136 ps, ci 0.950 benchmarking toStrict/simple #2 mean: 487.0739 ns, lb 486.7939 ns, ub 487.4713 ns, ci 0.950 std dev: 1.699232 ns, lb 1.262363 ns, ub 2.457099 ns, ci 0.950 benchmarking toStrict/simple #3 mean: 55.06322 us, lb 54.91370 us, ub 55.20236 us, ci 0.950 std dev: 741.6239 ns, lb 656.3273 ns, ub 846.6403 ns, ci 0.950 benchmarking toStrict/optimized #1 mean: 48.67522 ns, lb 48.65188 ns, ub 48.70237 ns, ci 0.950 std dev: 129.3192 ps, lb 111.3761 ps, ub 165.4819 ps, ci 0.950 benchmarking toStrict/optimized #2 mean: 178.6342 ns, lb 178.5480 ns, ub 178.7276 ns, ci 0.950 std dev: 457.4436 ps, lb 409.2746 ps, ub 519.8267 ps, ci 0.950 benchmarking toStrict/optimized #3 mean: 13.01866 us, lb 13.00734 us, ub 13.03549 us, ci 0.950 std dev: 70.09916 ns, lb 52.18012 ns, ub 97.77226 ns, ci 0.950 -}

14 16

Implement foldMap on Data.Sequence.Seq?
by Sjoerd Visscher 05 Nov '11

05 Nov '11

When experimenting with the code here: http://stackoverflow.com/questions/7869359/why-is-it-faster-to-sum-a-data-s… it turned out that foldMapDefault from traversable was more than 2x faster than foldl', and more than 6x faster than foldMap. foldMap is not implemented by Data.Seq, so it defaults to foldr (mappend . f) mempty. Does anybody have a good explanation for the StackOverflow question, and would it be a good idea to add a foldMap implementation to Seq? greetings, Sjoerd Visscher

1 0

RFC: Should Data.IntMap.Strict be value-strict in the function arguments or the map itself
by Johan Tibell 04 Nov '11

04 Nov '11

Hi all, As discussed in an earlier (accepted) proposal we're adding Data.[Int]Map.{Strict,Lazy} APIs to make it easier for users to get the desired strictness. There are two subtly different ways these APIs can be strict: 1. All the functions are strict in the value argument, or 2. The structure is strict in the value field. The difference shows up in functions like insertWith. For example, given insertWith (\ old _new -> old + 1) k v someMap should 'v' be evaluated is the key 'k' is already in the map. (Note that the value isn't used in the update function.) If we go with option 2 it is more difficult for the user to reason about if a given value passed to the API is evaluated or not (as in the above example). We currently use option 1 for keys i.e. insert k v someMap is strict in the key even if the map is empty (which wouldn't require us to look at the key). Thoughts? I'd love to see some design pros/cons of both approaches. -- Johan

6 17

Proposal: Add NFData instances for strict and lazy ByteStrings
by Bas van Dijk 04 Nov '11

04 Nov '11

While I was hunting for a space leak in the new hackage-server (still haven't found it) I noticed strict and lazy ByteStrings don't have NFData instances. I propose to add them because it enables users to force ByteStrings in the principled way (using deepseq) so they don't need to use tricks like: B.length b `seq` ... instance NFData Strict.ByteString instance NFData Lazy.ByteString where rnf Empty = () rnf (Chunk _ b) = rnf b Note that the containers dependency of deepseq was recently dropped so depending on that package is not a big issue anymore. Discussion deadline: 2 weeks from now (9 November) Bas

9 10

Proposal: Remove Show and Eq superclasses of Num
by Ian Lynagh 04 Nov '11

04 Nov '11

Hi all, I would like to propose that we remove the Show and Eq superclasses from Num, i.e. change class (Eq a, Show a) => Num a where [...] to class Num a where [...] The first 2 attached patches (for base and ghc respectively) remove the Show constraint. I'm not aware of any justification for why this superclass makes sense. The next 2 patches (for base and unix respectively) remove the Eq constraint. Here's there's some justification in the superclass, as it makes f 5 = ... work for any Num type, rather than also needing an Eq constraint, but personally I would be in favour of removing this superclass too. Noteworthy is that Bits now needs an Eq superclass for the default methods for 'testBit' and 'popCount'. The fifth patch (for base) is what prompted me to get around to sending this proposal. It lets us de-orphan the Show Integer instance. Any opinions? Suggested discussion deadline: 12th October Thanks Ian

21 50