- Glasgow-haskell-users

How to synchronously shutdown the event manager loop
by Bas van Dijk 31 Aug '11

31 Aug '11

Hello, In my (still unreleased) usb-1.0 (https://github.com/basvandijk/usb) library I use the GHC event manager for managing events from the underlying `libusb` C library. To work with the library a user has to initialize it using: newCtx ∷ IO Ctx The `Ctx` then allows the user to see the USB devices attached to the system using: getDevices ∷ Ctx → IO [Device] >From thereon a user can open devices using: openDevice ∷ Device → IO DeviceHandle and perform IO with them: readBulk ∷ DeviceHandle → EndpointAddress → Size → Timeout → IO (B.ByteString, Status) Internally the synchronous `readBulk` is implemented using asynchronous IO: first a request to read bytes from a USB device is asynchronously submitted. Then it waits on a lock (`takeMVar lock`). When the bytes are read an event is fired on a certain file descriptor. The GHC event manager then wakes up and calls a callback which will release the lock (`putMVar lock ()`). I choose to have one event loop per session (`Ctx`) with the usb library. This means that I make a new `EventManager` in `newCtx`: evtMgr ← EventManager.new register the `libusb` file descriptors with the event manager and fork a thread which will `loop` the `EventManager`: tid <- forkIOWithUnmask $ \unmask → unmask $ EventManager.loop evtMgr The underlying `libusb` C library needs to be deinitialized when you're done with it. However for safety and ease of use a user of my Haskell library doesn't need to explicitly deinitialize the library. Instead this happens automatically when the garbage collector collects an unreferenced `Ctx` by registering (using `Foreign.Concurrent.newForeignPtr`) the following finalizer with the `Ctx` foreign pointer: -- Stop the event handling loop by killing its thread: killThread tid ... -- Finally deinitialize libusb: c'libusb_exit ctxPtr As you see I also kill the thread which is running the event manager loop. However I think this is not the right way to do it because when I use the library I see the following message being continually printed after the `Ctx` is finalized: ghc: ioManagerWakeup: write: Bad file descriptor Note that when I remove the `killThread tid` the messages disappear. It seems that killing the event manager loop is not allowed. Is there another way of shutting down the event manager loop? I know I can shut it down by using the `shutdown` function. However this function is asynchronous. This won't work because I need to make sure the event loop is shutdown before I exit the `libusb` library (because that will close the file descriptors that the event loop is working with). I see the event manager provides the `finished` function which will poll the manager to see if it stopped looping. This function is not exported however. Is it possible to export this function? It would be even nicer to have a synchronous shutdown function so that I don't need to setup a loop that waits till the manager has been shut down. Regards, Bas

2 4

ghc fails to build due to trust issues
by Bas van Dijk 30 Aug '11

30 Aug '11

Hello, I'm trying to build recent ghc-HEAD using ghc-7.2.1 but get the following error: libraries/filepath/System/FilePath/Internal.hs:81:1: base:Data.List can't be safely imported! The package (base) the module resides in isn't trusted. I guess a "-trust base" flag has to be passed to ghc somewhere. Regards, Bas

2 2

ghc-pkg unregister disobedience
by Albert Y. C. Lai 30 Aug '11

30 Aug '11

For package P, "ghc-pkg unregister --user P" drops the global package if P is not in user. "ghc-pkg unregister --global P" drops the user package if P is in user. "ghc-pkg unregister --package-conf=<dir> P" drops the user or the global package, preferring the user one, if P is not found in <dir>. Same with "ghc-pkg --user unregister P", "ghc-pkg unregister P --user", etc. GHC versions 6.10, 6.12, 7.0, 7.2.

1 1

Bug in GC's ordering of ForeignPtr finalization?
by Ben Gamari 29 Aug '11

29 Aug '11

On Tue, 16 Aug 2011 12:32:13 -0400, Ben Gamari <bgamari.foss(a)gmail.com> wrote: > It seems that the notmuch-haskell bindings (version 0.2.2 built against > notmuch from git master; passes notmuch-test) aren't dealing with memory > management properly. In particular, the attached test code[1] causes > talloc to abort. Unfortunately, while the issue is consistently > reproducible, it only occurs with some queries (see source[1]). I have > been unable to establish the exact criterion for failure. > > It seems that the crash is caused by an invalid access to a freed Query > object while freeing a Messages object (see Valgrind trace[3]). I've > taken a brief look at the bindings themselves but, being only minimally > familiar with the FFI, there's nothing obviously wrong (the finalizers > passed to newForeignPtr look sane). I was under the impression that > talloc was reference counted, so the Query object shouldn't have been > freed unless if there was still a Messages object holding a > reference. Any idea what might have gone wrong here? Thanks! > After looking into this issue in a bit more depth, I'm even more confused. In fact, I would not be surprised if I have stumbled into a bug in the GC. It seems that the notmuch-haskell bindings follow the example of the python bindings in that child objects keep references to their parents to prevent the garbage collector from releasing the parent, which would in turn cause talloc to free the child objects, resulting in odd behavior when the child objects were next accessed. For instance, the Query and Messages objects are defined as follows, type MessagesPtr = ForeignPtr S__notmuch_messages type MessagePtr = ForeignPtr S__notmuch_message newtype Query = Query (ForeignPtr S__notmuch_query) data MessagesRef = QueryMessages { qmpp :: Query, msp :: MessagesPtr } | ThreadMessages { tmpp :: Thread, msp :: MessagesPtr } | MessageMessages { mmspp :: Message, msp :: MessagesPtr } data Message = MessagesMessage { msmpp :: MessagesRef, mp :: MessagePtr } | Message { mp :: MessagePtr } type Messages = [Message] As seen in the Valgrind dump given in my previous message, it seems that the Query object is being freed before the Messages object. Since the Messages object is a child of the Query object, this fails. In my case, I'm calling queryMessages which begins by issuing a given notmuch Query, resulting in a MessagesPtr. This is then packaged into a QueryMessages object which is then passed off to unpackMessages. unpackMessages iterates over this collection, creating MessagesMessage objects which themselves refer to the QueryMessages object. Finally, these MessagesMessage objects are packed into a list, resulting in a Messages object. Thus we have the following chain of references, MessagesMessage | | msmpp \/ QueryMessages | | qmpp \/ Query As we can see, each MessagesMessage object in the Messages list resulting from queryMessages holds a reference to the Query object from which it originated. For this reason, I fail to see how it is possible that the RTS would attempt to free the Query before freeing the MessagesPtr. Did I miss something in my analysis? Are there tools for debugging issues such as this? Perhaps this is a bug in the GC? Any help at all would be greatly appreciated. Cheers, - Ben

3 4

Prototype of 'docase' notation
by Tomas Petricek 26 Aug '11

26 Aug '11

Hello,! at the Cambridge Hackathon, I started implementing an extension for GHC that adds the 'docase' notation. The notation is a syntactic sugar that can makes it easier to write code that works with monads that have three additional operations (parallel composition, choice and aliasing). Such monads include various moands for parallel and concurrent programming (Par monad, CPH and Orc) as well as other monads such as parsers. The proposal is in details discussed in our Haskell Symposium paper: http://www.cl.cam.ac.uk/~tp322/papers/docase.html (the page also contains link to earlier pre-processor based prototype with a couple of more examples). I created a GHC Trac ticket to track the discussion about the feature: http://hackage.haskell.org/trac/ghc/ticket/5429. A prototype version of the GHC extension is available here: http://github.com/tpetricek/Haskell.Extensions/commits/. I'd love to hear some feedback about the feature and the extension: * Do you think this is a useful extension and do you know about any monads that could benefit from it? * What is the best syntax for the feature? (The notation intentionally resembles 'case' pattern matching, but there are some differences) * The feature adds some aspects of recently added monad comprehensions using do-style syntax, but both 'docase' and monad comprehensions have features that cannot be written using the other. Is that a good desing, or should there be a closer correspondence? * Can anybody review the prototype implementation? (I believe it is a reasonably modular change, so it should not affect the rest of the language, but is there anything missing/incorrect?) Thanks! Tomas

2 2

With every new GHC release, also released any new versions of libraries
by Johan Tibell 26 Aug '11

26 Aug '11

Hi! As part of GHC releases users also get a bunch of libraries. These libraries (e.g. array-0.3.0.3 in the last release) aren't uploaded on Hackage. This has some problems, including users on older versions of GHC not being able to upgrade to the new versions and broken documentation on Hackage. I suggest that with each GHC release the new library releases should be uploaded to Hackage. Also, for libraries not maintained by GHC but shipped with the compiler (e.g. containers) the new releases should be done by the maintainers of those libraries to make sure no half-finished code gets shipped (i.e. the repo might not be in a good state to release at all times). P.S. Could someone please remind me why containers ships with GHC? Cheers, Johan

6 8

building GHC on a cluster
by Karel Gardas 25 Aug '11

25 Aug '11

Hello, recent "GHC build times on newer MacBook Pros?" thread makes me wondering if someone already attempted to build GHC on a cluster of machines using for example PVM GNU make[1] for this or any other tool ( or even custom generated bin/ghc-stageX scripts using rsh?) which is suitable for the task. I'm asking since build of GHC HEAD on my ARM/Linux (OMAP4 1GHz) using both cores takes around 7 hours and yet buying another 3 machines might be well in the finance envelope of hobby project and speed the development which usually happens only during weekend's evenings... Thanks! Karel [1]: http://pvmgmake.sourceforge.net/

2 1

Cheap and cheerful partial evaluation
by Edward Z. Yang 24 Aug '11

24 Aug '11

Hello all, It occurred to me that it might not be too difficult to use GHC's optimization passes as a cheap and cheerful partial evaluator. Consider some function we would like to partially evaluate: f = g h Partial evaluation proceeds as follows: calculate the type of f, inline and specialize g, inline and specialize h, and then optimize. Effectively, laser-guided inlining. With this (very) heavy hammer, we can, for example, solve the problem posed in http://hackage.haskell.org/trac/ghc/ticket/1349, simply by ensuring all of our "strict functions" are partially evaluated on the continuation handler appropriately. (This is not ideal, since we ought to be able to share the strict worker/wrapper between instances, but might be a reasonable stop-gap for some use cases.) So, am I completely insane, or does this seem plausible and easy enough to implement to make it into GHC? Cheers, Edward

3 7

alternative to loadWithLogge
by Kazu Yamamoto 24 Aug '11

24 Aug '11

Hello, ghc-mod, IDE-like back-end for Emacs, uses warning related APIs including loadWithLogger and getWarnings in GHC 7.0.3 API. I found that they disappeared in GHC 7.2.1 API. What should I use to handle warnings in GHC 7.2.1 API? --Kazu

2 2

What are the preconditions of newArray#
by Johan Tibell 23 Aug '11

23 Aug '11

Hi, I'm seeing a segfault which I suspect is due to allocating Array#s of size 0, using newArray#. Are zero length arrays allowed? What are the preconditions of newArray#? It'd be great if they were documented. -- Johan

4 8