Resource Limits for Haskell

Hey folks, Have you ever wanted to implement this function in Haskell? -- | Forks a thread, but kills it if it has more than 'limit' -- bytes resident on the heap. forkIOWithSpaceLimit :: IO () -> {- limit -} Int -> IO ThreadId Well, now you can! I have a proposal and set of patches here: http://hackage.haskell.org/trac/ghc/wiki/Commentary/ResourceLimits http://hackage.haskell.org/trac/ghc/ticket/7763 There is a lot of subtlety in this space, largely derived from the complexity of interpreting GHC's current profiling information. Your questions, comments and suggestions are greatly appreciated! Cheers, Edward

On Fri, Mar 15, 2013 at 5:17 PM, Edward Z. Yang
There is a lot of subtlety in this space, largely derived from the complexity of interpreting GHC's current profiling information. Your questions, comments and suggestions are greatly appreciated!
How secure is this? One of the reasons for forking a process and then killing it after a timeout in lambdabot/mueval is because a thread can apparently block the GC from running with a tight enough loop and the normal in-GHC method of killing threads doesn't work. Can one simultaneously in a thread allocate ever more memory and suppress kill signals? -- gwern http://www.gwern.net

The particular problem you're referring to is fixed if you compile all your libraries with -falways-yield; see http://hackage.haskell.org/trac/ghc/ticket/367 I believe that it is possible to give a guarantee that the kill signal will hit the thread in a timely fashion. The obvious gap in our coverage at the moment is that there may be some primops that infinite loop, and there are probably other bugs, but I do not believe they are insurmountable. Edward Excerpts from Gwern Branwen's message of Fri Mar 15 14:39:50 -0700 2013:
On Fri, Mar 15, 2013 at 5:17 PM, Edward Z. Yang
wrote: There is a lot of subtlety in this space, largely derived from the complexity of interpreting GHC's current profiling information. Your questions, comments and suggestions are greatly appreciated!
How secure is this? One of the reasons for forking a process and then killing it after a timeout in lambdabot/mueval is because a thread can apparently block the GC from running with a tight enough loop and the normal in-GHC method of killing threads doesn't work. Can one simultaneously in a thread allocate ever more memory and suppress kill signals?

I now have a paper draft describing the system in more detail. It also comes with a brief explanation of how GHC's profiling works, which should also be helpful for people who haven't read the original profiling paper. http://ezyang.com/papers/ezyang13-rlimits.pdf Edward Excerpts from Edward Z. Yang's message of Fri Mar 15 14:17:39 -0700 2013:
Hey folks,
Have you ever wanted to implement this function in Haskell?
-- | Forks a thread, but kills it if it has more than 'limit' -- bytes resident on the heap. forkIOWithSpaceLimit :: IO () -> {- limit -} Int -> IO ThreadId
Well, now you can! I have a proposal and set of patches here:
http://hackage.haskell.org/trac/ghc/wiki/Commentary/ResourceLimits http://hackage.haskell.org/trac/ghc/ticket/7763
There is a lot of subtlety in this space, largely derived from the complexity of interpreting GHC's current profiling information. Your questions, comments and suggestions are greatly appreciated!
Cheers, Edward

On Mon, Apr 1, 2013 at 5:56 PM, Edward Z. Yang
Correct me if I'm wrong, but reading that I don't seem to see any tests against actual adversarial code - just checking that the limits kick in on a bunch of ordinary code. -- gwern http://www.gwern.net
participants (2)
-
Edward Z. Yang
-
Gwern Branwen