- Haskell-Cafe - Haskell.org

Weird multi-threading runtime behaviour of single-threaded program with GHC-7.0.3
by Herbert Valerio Riedel 24 Jun '11

24 Jun '11

Hello, I'm experiencing an unexpected behaviour with GHC-7.0.3/x86_64's multithreading runtime when running some more demanding single-thread computations. I've isolated the part of the larger application which exposes the problem down to the parsing of a ~8MiB JSON file which causes quite a bit of memory allocations (source at end of email). The program is compiled with ghc --make -O2 -threaded -eventlog -rtsopts parse_aeson.hs and I ran it with and w/o the -N12 option on a 12-core machine (the problem is visible on a 4-core machine as well): $ time ./parse_aeson test.json 0.000392s after readFile 0.001016s parse was a success... 6.050836s NFed parse-tree... 6.050968s NFed parse-tree... no exception real 0m6.100s user 0m5.680s sys 0m0.419s $ time ./parse_aeson test.json +RTS -N12 0.001249s after readFile 0.001777s parse was a success... 10.048607s NFed parse-tree... 10.048738s NFed parse-tree... no exception real 0m10.125s user 0m57.496s sys 0m39.927s When observing top(1)'s output, I saw that all HECs consumed a substantial amount of cpu cycle, so I loaded up threadscope to see what the remaining 11 HECs were doing all the time. I've put the resulting eventlog files and screenshots of threadscope's visualization online at http://www.narf.at/~hvr/Haskell/parse_aeson.N0.eventlog.png http://www.narf.at/~hvr/Haskell/parse_aeson.N0.eventlog.xz http://www.narf.at/~hvr/Haskell/parse_aeson.N12.eventlog.png http://www.narf.at/~hvr/Haskell/parse_aeson.N12.eventlog.xz What's happening there? The actual processing work seems to be done in a single HEC... but what are the remaining 11 HECs doing exactly? Am I doing something wrong? What does it mean when the GC-bars in threadscope becomes green as in the middle of the screenshot below? http://www.narf.at/~hvr/Haskell/parse_aeson.N4.eventlog.pdf ------------------------------------------------------------------------- -- also available at http://www.narf.at/~hvr/Haskell/parse_aeson.hs module Main where import Control.DeepSeq import Control.Exception import Data.Aeson import Data.Time import System.Environment import qualified Data.Attoparsec.Lazy as APL import qualified Data.ByteString.Lazy.Char8 as LB parseIt :: FilePath -> IO () parseIt fn = do t0 <- getCurrentTime let putStrLn' !s = do t0' <- getCurrentTime putStrLn $ show (t0' `diffUTCTime` t0) ++ " " ++ s json_text <- LB.readFile fn putStrLn' "after readFile" let Success p = decode json_text :: Result Value putStrLn' "parse was a success..." putStrLn' $ p `deepseq` "NFed parse-tree..." putStrLn' $ p `deepseq` "NFed parse-tree..." decode :: FromJSON a => LB.ByteString -> Result a decode = either Error fromJSON . APL.eitherResult . APL.parse json main :: IO () main = do fns <- getArgs case fns of [fn] -> do res <- try $ parseIt fn case res of Left e -> putStrLn $ "caught: " ++ show (e :: SomeException) Right () -> putStrLn "no exception" _ -> putStrLn "usage: parse_aeson <json-file>"

2 3

Re: [Haskell-cafe] Haskell Weekly News: Issue 187
by Jack Henahan 24 Jun '11

24 Jun '11

You could always just subscribe to the HWN feed on Contemplating Code. I just load up my reader when I don't want to read the text dispatches. Try http://contemplatecode.blogspot.com/feeds/posts/default?alt=rss That's the feed URL I use. Then the one on the mailing list is your plaintext fallback. :D On Jun 23, 2011, at 8:48 PM, Conrad Parker wrote: > On 24 June 2011 02:24, Rogan Creswick <creswick(a)gmail.com> wrote: >> On Thu, Jun 23, 2011 at 11:16 AM, Simon Michael <simon(a)joyful.com> wrote: >>> On 6/23/11 10:49 AM, Iustin Pop wrote: >>>> >>>> FYI, a regular link (though longer) seems more appropriate to me. >>>> Don't know if other people feel the same though. >> >> I prefer the short links, since it is much easier to keep track of >> what's going on when reading on a small screen (much of my email >> reading these days is done on my phone.) > > I'd prefer to just read an HTML version of HWN on my phone. How about > simply sending an HTML email with a plaintext fallback? > > In the plaintext email I'd prefer full URLs to cut/copy on my computer terminal. > > Conrad. > > _______________________________________________ > Haskell-Cafe mailing list > Haskell-Cafe(a)haskell.org > http://www.haskell.org/mailman/listinfo/haskell-cafe

1 0

Data.Enumerator question
by David Place 24 Jun '11

24 Jun '11

I can see that an Enumeratee can produce fewer outputs than it receives as inputs. For example, this GroupBy Enumeratee. > http://hpaste.org/47916 Can an Enumeratee produce more outputs than inputs? For instance, take a Data.Text and send each character to the inner Iteratee individually. I feels this can be done, but don't see how to do it. Thanks. ____________________ David Place Owner, Panpipes Ho! LLC http://panpipesho.com d(a)vidplace.com

2 1

Re: [Haskell-cafe] Category theory as a design tool
by wren ng thornton 23 Jun '11

23 Jun '11

On 6/23/11 1:39 AM, Arnaud Bailly wrote: > Of course, we can always say that each > system is a language of its own (rather than *has* a language...) which is > what Eric Evans coined with its "Ubiquitous language" term. But I find it > difficult to connect that particular dots. That's certainly my take on things (which I argued for at a workshop last week). Then again, I'm a linguist and type theorist, so viewing systems as languages[1] comes naturally. Do you have a link to Evans' coinage and argument? [1] Or rather, APIs as languages, their implementations as (defining their) semantics, and their execution as parsing. >> As an example of this approach, see my comments in the thread about >> anonymous sum types. For something to be a product means that a particular >> diagram commutes: namely, a product is the limit of the two point category >> (and coproducts are the colimit). One of the many things this tells us is >> that if we posit that A*A ~ A in a category that "has products", we must >> infer that the category is a meet-(pre)semilattice[1]. > > I certainly won't argue with you on this given my limited knowledge of what > a meet-(pre)semilattice could be :-) A preorder is just a generalization of a partial order, which we can gloss over here. A meet-semilattice is just a partial order with a "meet" operation, i.e. every pair of elements has a greatest lower bound. (A join-semilattice is the dual, with LUB instead of GLB.) > But then, if I happen to understand > what it is, how could that help me design and code "better" systems? I have > the intuition it could help me but again I do not (yet) see how. Well, for this example, there was the proposed idea to have anonymous union types with the injectors given by the types of the components (e.g., for the type A|B we'd have data constructors A :: A -> A|B and B :: B -> A|B). But this raises the issue of how to deal with A|A. Someone suggested that we could just say that A|A is the same as A. However, doing so means that the (|) type operator cannot be a coproduct. That's not a game-killer per se ---afterall, Either isn't a corpoduct--- but it does suggest that something fishy is going on. In this case, the better system is the one we already have, where we use disjoint unions (Either) instead of non-disjoint unions (|). To put a different spin on it, in the dual case we can show that Haskell's (,) is not a categorical product. There's a good deal of historical debate about why it works the way it does, but if we're looking to make a better system then the fact that tuples are not well-behaved suggests a place for improvement. The current (,) is a compromise between two different notions of product in domain theory. On the one hand we have domain products (which do not produce domains) and on the other hand we have smash products (which can "lose" information). I've had in mind for a while to make a Haskell-like language with a total functional core. In a total language (even a lazy one), we don't have bottom so we don't need domain theory to reason about the language. So in a language which is only partially total, it's reasonable to have two different kinds of products--- which means we can get away from Haskell's compromise, but do so in a way that's harmonious with the rest of the language. Would this new system be "better"? I dunno. It would behave in a more lawful manner, so it should be easier to reason about; but then we're making programmers keep that reasoning in mind, and maybe that's too much work. A decent system has to be both correct but also usable, and human factors are messy and hard to predict. -- Live well, ~wren

3 2

Category theory as a design tool
by Arnaud Bailly 23 Jun '11

23 Jun '11

(2nd try, took my gloves off...) Hello Café, I have been fascinated by Cat. theory for quite a few years now, as most people who get close to it I think. I am a developer, working mostly in Java for my living and dabbling with haskell and scala in my spare time and assuming the frustration of having to live in an imperative word. More often than not, I find myself trying to use constructs from FP in my code, mostly simple closures and typical data types (eg. Maybe, Either...). I have read with a lot of interest FPS (http://homepages.mcs.vuw.ac.nz/~tk/fps/) which exposes a number of OO patterns inspired by FP. Are there works/thesis/books/articles/blogs that try to use Cat. theory explicitly as a tool/language for designing software (not as an underlying formalisation or semantics)? Is the question even meaningful? Thanks in advance, Arnaud

6 11

Re: [Haskell-cafe] Problem with GLUT - no visuals found
by Jason Dagit 23 Jun '11

23 Jun '11

On Thu, Jun 23, 2011 at 11:57 AM, Chris Smith <cdsmith(a)gmail.com> wrote: > On Thu, 2011-06-23 at 11:50 -0700, Jason Dagit wrote: >> What is your deadline for making sure you have a working library? >> > > I don't know... August? In any case, I do now have a working library, > so long as I remember not to use GHCi for it! > > While GLFW would certainly be far preferable to GLUT in the long run, > the fact is that Gloss is already implemented using GLUT, and I can make > GLUT work. So I'll plan on using that unless someone else volunteers to > port Gloss to GLFW. > > (To be honest, I did consider it when I first ran into this problem a > week ago; but after a few hours of hacking at it, it became clear that > it's too large a job, and I'm far too unfamiliar with GLUT, GLFW, or > OpenGL, for that to be feasible as yet another side project for me.) Yes, I've suggested to the author of Gloss that he should switch to GLFW-b but he basically said what you said. He considered it when I mentioned it, but since Gloss already works with GLUT he wouldn't do it unless someone sent him the patches. Which is fair from a maintenance point of view. The downside is that all the windows users of Gloss have to get/install the glut32.dll from somewhere before Gloss will work for them. GLFW-b doesn't have that restriction. Also, when I first looked at GLFW I thought it didn't call atexit() (GLUT uses atexit(), which I think is kind of an evil C function, without having a specific gripe other than it being global mutable state), but it does actually use it. This line of thought always leads me to think that we should have a purely Haskell implementation of GLFW/GLUT on top of the platform specific gui bindings. At any rate, I'm glad to hear you have what you need for your class. Jason

1 0

Problem with GLUT - no visuals found
by Chris Smith 23 Jun '11

23 Jun '11

So I'm running into an embarrassing problem trying to prepare for the computer programming class that I intend to 12-13 year olds in Haskell this coming fall. I had decided to use gloss, an excellent package by Ben Lippmeier that makes it really easy to build images, animations, simulations, and games in a straight-forward and functional way. Excellent... but suddenly broken on my system since installing Ubuntu 11.4, and it looks like the cause traces back to GLUT. Prelude Graphics.Gloss> displayInWindow "Test" (100,100) (100,100) white (Circle 80) freeglut (<interactive>): ERROR: Internal error <Visual with necessary capabilities not found> in function fgOpenWindow X Error of failed request: BadWindow (invalid Window parameter) Major opcode of failed request: 4 (X_DestroyWindow) Resource id in failed request: 0x0 Serial number of failed request: 25 Current serial number in output stream: 28 and Prelude Graphics.UI.GLUT> initialize "foo" [] [] Prelude Graphics.UI.GLUT> get displayModePossible False Nothing I can find to change in GLUT causes displayModePossible to be True, and I'm at the end of my knowledge here. A quick web search turns up the occasional person complaining about the freeglut error, but in all cases they have trouble running any GLUT applications, and it turns out their configuration doesn't do direct rendering. In my case, glxgears and any other non-Haskell GLUT applications I can find work great, but Haskell's GLUT bindings are giving me no hint of success. Any ideas? -- Chris Smith

3 5

OSX, ghci, dylib, what is the correct way?
by Jason Dagit 23 Jun '11

23 Jun '11

Hello, I also asked this question on SO: http://stackoverflow.com/questions/6323755/osx-ghci-dylib-what-is-the-corre… I wonder where I'll get more/better answers? Read on for the question. Thanks! Jason I need to build some C code and then reference that C code via the FFI. I would like to use my binding from inside ghci on osx. On of my constraints is that I cannot just hand the C sources to ghc in the .cabal file. This is due to a limitation with ghc/cabal that may be fixed in the next release of ghc (but I want my code to work now and on older releases). See this [bug for details][1]. The gist of that bug is that the C code needs to be compiled with some Objective-C modules and ghc misinterprets those as linker scripts. I've tried many things and building the files myself with a makefile is the only thing that has worked. Really, this shouldn't be an issue though because it should be the same as if I decided to use a external C library that I didn't build myself. For the sake of this issue, let's pretend it's a separate C library that I can easily rebuild with different options. If I build the C library as a .a, then ghci comlains that it cannot open the .dylib. My first question is: Why does ghci need a .dylib and does it really use it? When I build a dylib I get a [segfault when loading the code into ghci][2]. Keep in mind, this binding works already on other platforms, both linux and windows, and the binding works fine on osx when I'm compiling instead of using ghci. This problem specific to the osx/ghci combo. In that trace above, I'm using gdb but it crashes regardless of whether I use gdb. I tracked it down to the lines that cause the crash: void _glfwClearWindowHints( void ) { memset( &_glfwLibrary.hints, 0, sizeof( _glfwLibrary.hints ) ); } The trouble maker is that memset line, well actually the problem is that when running inside ghci writing to the hints structure of `_glfwLibrary` is a memory access violation. The hints struct is simply a bunch of ints. It's very flat and simple, and thus I think the problem is an issue either with how I'm linking things or with the way ghci is loading the code. Here are the bits of my makefile that I use to build the dylib and the .a: GCCFLAGS := $(shell ghc --info | ghc -e "fmap read getContents >>= \ putStrLn . unwords . read . Data.Maybe.fromJust . lookup \ \"Gcc Linker flags\"") FRAMEWORK := -framework Cocoa -framework OpenGL GLFW_FLAG := $(GCCFLAGS) -O2 -fno-common -Iglfw/include -Iglfw/lib \ -Iglfw/lib/cocoa $(CFLAGS) all: $(BUILD_DIR)/static/libglfw.a $(BUILD_DIR)/dynamic/libglfw.dylib $(BUILD_DIR)/dynamic/libglfw.dylib: $(OBJS) $(CC) -dynamiclib -Wl,-single_module -compatibility_version 1 \ -current_version 1 \ $(GLFW_FLAG) -o $@ $(OBJS) $(GLFW_SRC) $(FRAMEWORK) $(BUILD_DIR)/static/libglfw.a: $(OBJS) ar -rcs $@ $(OBJS) Most of the flags are taken straight from the GLFW Makefile so I think they should be correct for that library. The first line looks a bit weird but it's the solution I used for this [problem][3]. Platform details: - OSX 10.6.6 - x86_64 - 4 cores - GHC version 7.0.3 installed via Haskell Platform installer - Source repo: https://github.com/dagit/GLFW-b [1]: http://hackage.haskell.org/trac/ghc/ticket/5025 [2]: http://hpaste.org/47457/segfault_in_ghci [3]: http://stackoverflow.com/questions/6217406/how-can-i-detect-if-ghc-is-set-t…

4 10

Darcs 2.8 beta 1 is available
by Florent Becker 23 Jun '11

23 Jun '11

Hi cafe-dwellers, On behalf of the darcs team, I'm pleased to announce the release of darcs 2.8 beta 1. This new darcs is just one cabal install away from you, as you can install it by: -cabal update -cabal install darcs-beta (note this will create an executable named "darcs", not darcs-beta). You can then come back to your usual darcs if you needby -cabal install --reinstall darcs Alternatively, you can get the source tarball from here: http://hackage.haskell.org/packages/archive/darcs-beta/2.7.98.1/darcs-beta-…. More information on the release is available at http://wiki.darcs.net/Releases/2.8 This is a beta, so bug reports are very welcome. Highlights of this new version of darcs are: - a faster and much more readable annotate. - you can use darcs optimize --http to make darcs get from your repository over http much faster - you can now use darcs obliterate -o bundle.dpatch to save some patches into a bundle while unpulling them. This allows to stash these patches away while you do something else. - rollback now has a --no-record option which allows to undo some changes from a past patch without recording that as a new patch straight away. - end of write support of "old-fashionned" repositories. These repositories are the one darcs-1 used to create, and they have been replaced by the new "hashed" repositories since darcs-2. You can still use get, pull and so on from these repositories, but darcs 2.8 will refuse to write to them. Enjoy your new darcs, The darcs team

1 0

Quickcheck: Help on non trivial test
by jean-christophe mincke 23 Jun '11

23 Jun '11

Hello Café, I am using quicheck for some kind of non trivial tests. The general form of these tesst is summarized by the following code. -- Function to be tested. Given a list of splitting functions, split the given list process :: [a] -> [[a] -> [a]] -> [a] process l splitFuns = List.foldl proc l splitFuns where proc l splitFun = splitFun l -- a split function generator splitFunGen :: Gen ([a] -> [a]) splitFunGen = return $ proc where proc l = let splitPos = List.length l `div` 2 -- Problem I would like splitPos = some random value between [0 and length of l] in fst $ List.splitAt splitPos l splitFunsGen :: Gen [[a] -> [a]] splitFunsGen = vectorOf 20 splitFunGen instance Show a => Show ([a] -> [a]) where show _ = " a split fun " r = quickCheck $ forAll splitFunsGen prop where prop splitFuns = let l = process [1..100] splitFuns in List.length l >= 0 -- dummy test here for the sake of example The process to be tested takes a list that is randomly perturbated (spitFuns). The result of each perturbation is fed into the next perturbation. Ideally I would like the perturbating function to depend on the previous perturbated list (see computation of splitPos in splitFunGen). I am not sure how I could use quickcheck in this case. Has anyone a better idea? Thanks J-C

2 1