- Glasgow-haskell-users

Re: Foreign.StablePtr: nullPtr & double-free questions.
by Edward Z. Yang 13 Mar '13

13 Mar '13

Excerpts from Remi Turk's message of Fri Mar 08 18:28:56 -0800 2013: > Good night everyone, > > I have two questions with regards to some details of the > Foreign.StablePtr module. [1] > > 1) The documentation suggests, but does not explicitly state, that > castStablePtrToPtr `liftM` newStablePtr x > will never yield a nullPtr. Is this guaranteed to be the case or not? > It would conveniently allow me to store a Maybe "for free", using > nullPtr for Nothing, but I am hesitant about relying on something that > isn't actually guaranteed by the documentation. No, you cannot assume that. In fact, stable pointer zero is base_GHCziTopHandler_runIO_info: ezyang@javelin:~/Dev/haskell$ cat sptr.hs import Foreign.StablePtr import Foreign.Ptr main = do let x = castPtrToStablePtr nullPtr freeStablePtr x ezyang@javelin:~/Dev/haskell$ ~/Dev/ghc-build-tick/inplace/bin/ghc-stage2 --make sptr.hs -debug [1 of 1] Compiling Main ( sptr.hs, sptr.o ) Linking sptr ... ezyang@javelin:~/Dev/haskell$ gdb ./sptr GNU gdb (GDB) 7.5-ubuntu Copyright (C) 2012 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /srv/code/haskell/sptr...done. (gdb) b freeStablePtrUnsafe Breakpoint 1 at 0x73f8a7: file rts/Stable.c, line 263. (gdb) r Starting program: /srv/code/haskell/sptr [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Breakpoint 1, freeStablePtrUnsafe (sp=0x0) at rts/Stable.c:263 263 ASSERT((StgWord)sp < SPT_size); (gdb) list 258 } 259 260 void 261 freeStablePtrUnsafe(StgStablePtr sp) 262 { 263 ASSERT((StgWord)sp < SPT_size); 264 freeSpEntry(&stable_ptr_table[(StgWord)sp]); 265 } 266 267 void (gdb) p stable_ptr_table[(StgWord)sp] $1 = {addr = 0x9d38e0} (gdb) p *(StgClosure*)stable_ptr_table[(StgWord)sp] $2 = {header = {info = 0x4e89c8 <base_GHCziTopHandler_runIO_info>}, payload = 0x9d38e8} Regardless, you don't want to do that anyway, because stable pointers have a bit of overhead. > 2) If I read the documentation correctly, when using StablePtr it is > actually quite difficult to avoid undefined behaviour, at least in > GHC(i). In particular, a double-free on a StablePtr yields undefined > behaviour. However, when called twice on the same value, newStablePtr > yields the same StablePtr in GHC(i). > E.g.: > > module Main where > > import Foreign > > foo x y = do > p1 <- newStablePtr x > p2 <- newStablePtr y > print $ castStablePtrToPtr p1 == castStablePtrToPtr p2 > freeStablePtr p1 > freeStablePtr p2 -- potential double free! > > main = let x = "Hello, world!" in foo x x -- undefined behaviour! > > prints "True" under GHC(i), "False" from Hugs. Considering that foo > and main might be in different packages written by different authors, > this makes correct use rather complicated. Is this behaviour (and the > consequential undefinedness) intentional? I think this bug was inadvertently fixed in the latest version of GHC; see: commit 7e7a4e4d7e9e84b2c57d3d55e372e738b5f8dbf5 Author: Simon Marlow <marlowsd(a)gmail.com> Date: Thu Feb 14 08:46:55 2013 +0000 Separate StablePtr and StableName tables (#7674) To improve performance of StablePtr. Cheers, Edward

2 2

RE: GHC 7.8 release?
by Simon Peyton-Jones 13 Mar '13

13 Mar '13

We seem to be circling ever closer to consensus here! Yay! Indeed! Good :-) However, I’m not getting the bit about API changing vs non-API changing. Firstly I don’t know which APIs are intended. The GHC API is essentially GHC itself, so it changes daily. Maybe you mean the base package? Or what? I suspect you mean that a “non-API-changing” release absolutely guarantees to compile any package that compiled with the previous version. If so, that is a very strong constraint indeed. We do observe it for patch releases for GHC (e g 7.6.2 should compile anything that 7.6.1 compiles). But I think it would be difficult to guarantee for anything beyond a patch release. Every single commit (and the commit rate is many/day) would have to be evaluated against this criterion. And if it failed the criterion, it would have to go on a API-breaking HEAD. In effect we’d have two HEADs. I can’t see us sustaining this. And I don’t yet really see why it’s necessary. If you don’t want an API-breaking change, stick with the patch releases. So, we have a channel for non-API-breaking changes already: the patch releases. So that means we already have all three channels! · Haskell Platform · Patch-level releases · New releases if that’s so, all we need is better signposting. And I’m all for that! Have I got this right? Simon From: Mark Lentczner [mailto:mark.lentczner@gmail.com] Sent: 09 February 2013 17:48 To: Simon Marlow; Manuel M T Chakravarty; Johan Tibell; Simon Peyton-Jones; Mark Lentczner; andreas.voellmy(a)gmail.com; Carter Schonwald; kostirya(a)gmail.com; Edsko de Vries; ghc-devs(a)haskell.org; glasgow-haskell-users Subject: Re: GHC 7.8 release? We seem to be circling ever closer to consensus here! Yay! I think the distinction of non-API breaking and API breaking release is very important. Refining SPJ's trifecta: Haskell Platform comes out twice a year. It is based on very stable version of GHC, and intention is that people can just assume things on Hackage work with it. These are named for the year and sequence of the release: 2013.2, 2013.2.1, 2013.4,... Non-API breaking releases can come out as often as desired. However, the version that is current as of mid-Feb. and mid-Aug. will be the ones considered for HP inclusion. By non-API breaking we mean the whole API surface including all the libraries bundled with GHC, as well as the operation of ghc, cabal, ghc-pkg, etc. Additions of features that must be explicitly enabled are okay. Additions of new APIs into existing modules are discouraged: Much code often imports base modules wholesale, and name clashes could easily result. These should never bump the major revision number: 7.4.1, 7.4.2... API breaking releases happen by being released into a separate channel when ready for library owners to look at them. This channel should probably go through several stages: Ready for core package owners to work with, then HP package owners, then all package owners. I'd imagine this is a several month process. At the end of which, the release can go into the main channel. Such a merge shouldn't happen more than once a year... I think even once every two years is fine (!) To avoid confusion, I'd suggest that while in the separate channel, these release be named with odd number: 7.9, 7.11,..., and when moved to the main channel renamed to even: 7.10, 7.12... This idea of three channels needs to be much more clearly communicated. The warning on the download page is a failure: Googling "ghc" takes you to the home page of GHC which immediately trumpets the "Lastest News" of a release of GHC 7.6.2. Once a user has read that and decided to download, then "STOP!" box is a) going to be skipped as they scan for the download link, and b) if read and followed, causes the "WTF? Why is HP so back rev?" So we need to change the front page so that the three channels are clearly communicated and targeted at the right users. - Mark (BTW: The first few links on the GHC web site are out of date: The second nav link is to a survey that is 7 years old. The License page is 8 years out of date. The FAQ is over a year old.) On Sat, Feb 9, 2013 at 8:24 AM, Ian Lynagh <ian(a)well-typed.com<mailto:ian@well-typed.com>> wrote: On Sat, Feb 09, 2013 at 12:06:12PM +0000, Simon Marlow wrote: > > As a straw man, let's suppose we want to do annual API releases in > September, with intermediate non-API releases in February. That's a non-API release 5 months after the API release. 6.10.2 was 5 months after 6.10.1 (.3 was 1 month later, .4 a further 2) 6.12.2 was 4 months after 6.12.1 (.3 was 2 months later) 7.0.2 was 3.5 months after 7.0.1 (.3 was 1 month later, .4 a further 3) 7.2.2 was 3 months after 7.2.1 7.4.2 was 4 months after 7.4.1 7.6.2 was 4.5 months after 7.6.2 so if we do non-API releases, then perhaps it would make sense to stop doing minor releases (unless a release turns out to just be broken). Thanks Ian

26 103

runghc -fdefer-type-errors
by Kazu Yamamoto 13 Mar '13

13 Mar '13

Hello, Doesn't runghc support the -fdefer-type-errors option? Consider this code: ---- module Main where main :: IO () main = do -- putStrLn は文字列を取る putStrLn "Hello, world!" putStrLn 1 -- 型エラー ---- If I use runghc with -fdefer-type-errors, "Hello, world!" is not printed. Is this a bug? If this behavior is intended, I would like to change it. If GHC can run code like dynamically typed languages, it would be appealing to new Haskell programmers from their community. --Kazu

6 7

Foreign.StablePtr: nullPtr & double-free questions
by Remi Turk 09 Mar '13

09 Mar '13

Good night everyone, I have two questions with regards to some details of the Foreign.StablePtr module. [1] 1) The documentation suggests, but does not explicitly state, that castStablePtrToPtr `liftM` newStablePtr x will never yield a nullPtr. Is this guaranteed to be the case or not? It would conveniently allow me to store a Maybe "for free", using nullPtr for Nothing, but I am hesitant about relying on something that isn't actually guaranteed by the documentation. 2) If I read the documentation correctly, when using StablePtr it is actually quite difficult to avoid undefined behaviour, at least in GHC(i). In particular, a double-free on a StablePtr yields undefined behaviour. However, when called twice on the same value, newStablePtr yields the same StablePtr in GHC(i). E.g.: module Main where import Foreign foo x y = do p1 <- newStablePtr x p2 <- newStablePtr y print $ castStablePtrToPtr p1 == castStablePtrToPtr p2 freeStablePtr p1 freeStablePtr p2 -- potential double free! main = let x = "Hello, world!" in foo x x -- undefined behaviour! prints "True" under GHC(i), "False" from Hugs. Considering that foo and main might be in different packages written by different authors, this makes correct use rather complicated. Is this behaviour (and the consequential undefinedness) intentional? With kind regards, Remi Turk [1] http://www.haskell.org/ghc/docs/latest/html/libraries/base-4.6.0.1/Foreign-…

1 0

Need help interpreting renamer trace output
by Philip K.F. Hölzenspies 06 Mar '13

06 Mar '13

Dear GHC-ers, I'm looking into the inerts of the renamer and I'm trying to get a feel for where things are bound. I thought it would be useful to have a look at how the renamer works on something known, so I compiled the following program (with HEAD): class Foo x where type FooTy x y foo :: x t -> FooTy x t instance Foo [] where type FooTy [] a = a foo (x:xs) = x main = print "Done" In the output of -ddump-rn-trace, I see this around the declaration of the class Foo: bhtv [x, y] [] ([(v1A, x), (v1C, y)], [(afy, x), (afz, y)]) bhtv [t] [] ([(v1A, x), (vjW, t)], [(afy, x), (afA, t)]) rnTycl [(class Foo x where type family FooTy x y foo :: x t -> FooTy x t, [(rfv, FooTy)])] [REC class Foo x where type family FooTy x y foo :: x t -> FooTy x t] I had expected that the 'x' and 'y' would have had the same uniques. The last tuple of bhtv is a LocalRdrEnv, which is (OccEnv Name, NameSet). Can someone explain to me the uses of both of these parts and why the names mentioned in each have different uniques? Regards, Philip

2 2

Re: 'interrupted' when exiting
by Akio Takano 06 Mar '13

06 Mar '13

[Resending to the list] Thank you for your explanation. On Mon, Mar 4, 2013 at 10:32 PM, Simon Marlow <marlowsd(a)gmail.com> wrote: > The issue is this: when the main thread exits, the RTS starts shutting down, > and it kills all the other threads. If any of these threads were the result > of an external in-call from C, then that call returns, with the status > "Interrupted". Now, if this was a foreign export or a foreign import > "wrapper", then there is no provision for returning an error code to the > caller, so the foreign export must either > > a) kill the current OS thread > b) kill the whole process > c) sleep indefinitely > > Right now we do (b) I think. Perhaps (a) would be better; at least that > might fix your problem. Would it make sense to make rts behave differently depending on whether the process is going to be terminated? For example, could it do (c) when shutdownHaskellAndExit() is being called, and do (b) otherwise? - Akio

1 0

'interrupted' when exiting
by Akio Takano 04 Mar '13

04 Mar '13

Hi, If I compile and run the attached program in the following way, it crashes on exit: $ ghc -threaded thr_interrupted.hs $ ./thr_interrupted thr_interrupted: Main_dtl: interrupted This is particularly bad when profiling, because the program terminates before writing to the .prof file. Is this a bug in GHC, or am I supposed to terminate the thread before exiting? If it's a GHC issue, how should it be fixed? It seems that the error was raised in the C stub code GHC generated for the foreign import "wrapper" function. Regards, Takano Akio

2 1

Libffi errors while cross-compiling GHC 7.6.2 (Stage 1)
by Nikita Karetnikov 01 Mar '13

01 Mar '13

I'd like to cross-build GHC 7.6.2. I got stuck when I was trying to build a Stage 1 compiler [1]. Here's a bug report: [2,3]. I'm not sure that it's a bug; maybe I misunderstood the process. Could anyone advise? I have no experience with C. Though, I'll try to help out. [1] http://hackage.haskell.org/trac/ghc/wiki/CrossCompilation [2] http://www.haskell.org/pipermail/ghc-tickets/2013-February/001240.html [3] http://hackage.haskell.org/trac/ghc/ticket/7717

1 0

Race-condition in alternative 'System.Timeout.timeout' implementation
by Herbert Valerio Riedel 28 Feb '13

28 Feb '13

Hello *, I've been experimenting with an alternative implementation of 'System.Timeout.timeout'[1] which avoids the overhead of spawning a new thread for each invocation. Part of my motivation is to see if I can implement a faster version of threadWaitReadTimeout :: Int -> Fd -> IO Bool threadWaitReadTimeout to = liftM (maybe False (const True)) . timeout to . threadWaitRead and thus exploit GHC's event notification system instead of having to reimplement a timeout-manager myself (like popular HTTP server libraries such as Snap or Warp do currently). The following Haskell program shows a proof-of-concept implementation derived directly from 'System.Timeout.timeout' together with a Criterion benchmark comparing the performance between the original and the alternative 'timeout' function wrapping a 'readMVar' call. On a i7-3770 with GHC-7.6.2/Linux/64bit ran with "+RTS -A4m -N4", the benchmark shows a 15x improvement for the new implementation (below 1 uS) compared to the original implementation (~13 uS): ,---- | benchmarking id | mean: 22.60933 ns, lb 22.50331 ns, ub 22.73515 ns, ci 0.950 | std dev: 591.0383 ps, lb 509.6189 ps, ub 663.2670 ps, ci 0.950 | found 17 outliers among 100 samples (17.0%) | 17 (17.0%) high mild | variance introduced by outliers: 19.992% | variance is moderately inflated by outliers | | benchmarking timeout_1ms | mean: 13.79584 us, lb 13.62939 us, ub 13.92814 us, ci 0.950 | std dev: 756.3080 ns, lb 524.7628 ns, ub 1.068547 us, ci 0.950 | found 14 outliers among 100 samples (14.0%) | 4 (4.0%) low severe | 5 (5.0%) high mild | 5 (5.0%) high severe | variance introduced by outliers: 52.484% | variance is severely inflated by outliers | | benchmarking timeout2_1ms | mean: 879.8152 ns, lb 874.5223 ns, ub 885.9759 ns, ci 0.950 | std dev: 29.31963 ns, lb 25.65941 ns, ub 32.98116 ns, ci 0.950 | found 9 outliers among 100 samples (9.0%) | 9 (9.0%) high mild | variance introduced by outliers: 28.734% | variance is moderately inflated by outliers | ... `---- Alas there's a race-condition hidden somewhere I'm struggling with; When the timeout is set low enough, the internal 'Timeout2' exceptions leaks outside the 'timeout2' wrapper: ,---- | ... | benchmarking timeout2_10us | newtimeout: <<timeout2>> `---- I've tried rewriting the code but couldn't figure out a way to keep the exception from escaping 'timeout2'. Does the race-condition actually lie in the 'timeout2' implementation -- and if so, is it possible to rewrite 'timeout2' to solve it? [1]: http://hackage.haskell.org/packages/archive/base/latest/doc/html/System-Tim… cheers, hvr

5 7

Re: Race-condition in alternative 'System.Timeout.timeout' implementation
by Akio Takano 26 Feb '13

26 Feb '13

I accidentally replied to Herbert privately. I'm forwarding the message to the list. - Takano Akio ---------- Forwarded message ---------- From: Akio Takano <tkn.akio(a)gmail.com> Date: Mon, Feb 25, 2013 at 6:15 PM Subject: Re: Race-condition in alternative 'System.Timeout.timeout' implementation To: Herbert Valerio Riedel <hvr(a)gnu.org> Hi, I think the problem is that E.unregisterTimeout doesn't guarantee that no timeout will be delivered after it returns. This allows an execution sequence like: 0. Thread A calls timeout2 1. Thread A registers a callback to the event manager 2. Thread A unregisters the callback 3. Thread A exits from the handleJust 4. The callback is triggered, killing thread A If I understand correctly this can be worked around with an extra mutex in timeout2. I'll attach my implementation. It's called timeout3 and is a bit slower than timeout2, but I haven't seen a leaking exception with it. On Mon, Feb 25, 2013 at 7:31 AM, Herbert Valerio Riedel <hvr(a)gnu.org> wrote: > Hello *, > > I've been experimenting with an alternative implementation of > 'System.Timeout.timeout'[1] which avoids the overhead of spawning a new > thread for each invocation. > > Part of my motivation is to see if I can implement a faster version of > > threadWaitReadTimeout :: Int -> Fd -> IO Bool > threadWaitReadTimeout to = liftM (maybe False (const True)) > . timeout to . threadWaitRead > > and thus exploit GHC's event notification system instead of having to > reimplement a timeout-manager myself (like popular HTTP server libraries > such as Snap or Warp do currently). > > > The following Haskell program shows a proof-of-concept implementation > derived directly from 'System.Timeout.timeout' together with a Criterion > benchmark comparing the performance between the original and the > alternative 'timeout' function wrapping a 'readMVar' call. > > > > On a i7-3770 with GHC-7.6.2/Linux/64bit ran with "+RTS -A4m -N4", the > benchmark shows a 15x improvement for the new implementation (below 1 > uS) compared to the original implementation (~13 uS): > > ,---- > | benchmarking id > | mean: 22.60933 ns, lb 22.50331 ns, ub 22.73515 ns, ci 0.950 > | std dev: 591.0383 ps, lb 509.6189 ps, ub 663.2670 ps, ci 0.950 > | found 17 outliers among 100 samples (17.0%) > | 17 (17.0%) high mild > | variance introduced by outliers: 19.992% > | variance is moderately inflated by outliers > | > | benchmarking timeout_1ms > | mean: 13.79584 us, lb 13.62939 us, ub 13.92814 us, ci 0.950 > | std dev: 756.3080 ns, lb 524.7628 ns, ub 1.068547 us, ci 0.950 > | found 14 outliers among 100 samples (14.0%) > | 4 (4.0%) low severe > | 5 (5.0%) high mild > | 5 (5.0%) high severe > | variance introduced by outliers: 52.484% > | variance is severely inflated by outliers > | > | benchmarking timeout2_1ms > | mean: 879.8152 ns, lb 874.5223 ns, ub 885.9759 ns, ci 0.950 > | std dev: 29.31963 ns, lb 25.65941 ns, ub 32.98116 ns, ci 0.950 > | found 9 outliers among 100 samples (9.0%) > | 9 (9.0%) high mild > | variance introduced by outliers: 28.734% > | variance is moderately inflated by outliers > | ... > `---- > > Alas there's a race-condition hidden somewhere I'm struggling with; When > the timeout is set low enough, the internal 'Timeout2' exceptions leaks > outside the 'timeout2' wrapper: > > ,---- > | ... > | benchmarking timeout2_10us > | newtimeout: <<timeout2>> > `---- > > I've tried rewriting the code but couldn't figure out a way to keep the > exception from escaping 'timeout2'. Does the race-condition actually lie > in the 'timeout2' implementation -- and if so, is it possible to rewrite > 'timeout2' to solve it? > > > [1]: http://hackage.haskell.org/packages/archive/base/latest/doc/html/System-Tim… > > cheers, > hvr > > _______________________________________________ > Glasgow-haskell-users mailing list > Glasgow-haskell-users(a)haskell.org > http://www.haskell.org/mailman/listinfo/glasgow-haskell-users >

1 0