#7763: Resource limits for Haskell -----------------------------+---------------------------------------------- Reporter: ezyang | Owner: ezyang Type: feature request | Status: new Priority: normal | Component: Profiling Version: 7.7 | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: None/Unknown | Blockedby: Blocking: | Related: -----------------------------+---------------------------------------------- Description changed by ezyang: Old description:

{{{ commit 1f5eb79cc278fd64ee3f1f4249ea3b7404f62ba1 Author: Edward Z. Yang Date: Thu Mar 7 13:53:31 2013 -0800

Primops and runtime support for space resource limits.

This patch adds support for resource limits in Haskell, especially for monitoring and triggering events when portions of Haskell code exceed memory usage and/or allocation. The system is described in more detail at.

http://hackage.haskell.org/trac/ghc/wiki/Commentary/ResourceLimits

This patch adds the following types:

CostCentre# - non-GCd pointer to CostCentre struct CostCentreStack# - non-GCd pointer to CostCentreStack struct Listener# - GC'd object for listening to pointer events

and the following primops:

ccToAddr addrToCC ccsToAddr addrToCCS

newCC# setCCSOf# withCCS# pushCC# queryCCS# listenCCS# unlistenCCS#

These previously existing primops have had their types changed, mainly for better type safety (replacing Addr# with CostCentreStack#)

getCCSOf# getCurrentCCS#

We add the following new RTS options:

-hl equivalent to -hc but also keeps census information in- memory for access and does not write out a *.hp file

-xf modified cost-centre stack semantics, where function entries never modify the stack (so only CAFs and explicit set CCCS affect cost-centres).

To utilize these new features, compile with -prof and run with +RTS -hl -xf }}}

See also http://hackage.haskell.org/trac/ghc/wiki/Commentary/ResourceLimits

New description: {{{ commit 1f5eb79cc278fd64ee3f1f4249ea3b7404f62ba1 Author: Edward Z. Yang Date: Thu Mar 7 13:53:31 2013 -0800 Primops and runtime support for space resource limits. This patch adds support for resource limits in Haskell, especially for monitoring and triggering events when portions of Haskell code exceed memory usage and/or allocation. The system is described in more detail at. http://hackage.haskell.org/trac/ghc/wiki/Commentary/ResourceLimits This patch adds the following types: CostCentre# - non-GCd pointer to CostCentre struct CostCentreStack# - non-GCd pointer to CostCentreStack struct Listener# - GC'd object for listening to pointer events and the following primops: ccToAddr addrToCC ccsToAddr addrToCCS newCC# setCCSOf# withCCS# pushCC# queryCCS# listenCCS# unlistenCCS# These previously existing primops have had their types changed, mainly for better type safety (replacing Addr# with CostCentreStack#) getCCSOf# getCurrentCCS# We add the following new RTS options: -hl equivalent to -hc but also keeps census information in- memory for access and does not write out a *.hp file -xf modified cost-centre stack semantics, where function entries never modify the stack (so only CAFs and explicit set CCCS affect cost-centres). To utilize these new features, compile with -prof and run with +RTS -hl -xf }}} See also http://hackage.haskell.org/trac/ghc/wiki/Commentary/ResourceLimits Probably the most dodgy parts of the patch is the primop code for {{{withCCS#}}}, and some of the refactoring of the heap profiler. -- -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7763#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#7763: Resource limits for Haskell ---------------------------------+------------------------------------------ Reporter: ezyang | Owner: ezyang Type: feature request | Status: patch Priority: normal | Milestone: Component: Profiling | Version: 7.7 Keywords: | Os: Unknown/Multiple Architecture: Unknown/Multiple | Failure: None/Unknown Difficulty: Unknown | Testcase: Blockedby: | Blocking: Related: | ---------------------------------+------------------------------------------ Changes (by simonpj): * difficulty: => Unknown Comment: Simon M is really the main expert (and gatekeeper) for this kind of feature. It certainly looks interesting, and resource control is an important question. But (tell me if I'm wrong) it's not at all trivial, and the patch runs to 1,200 lines (pretty small actually). I don't know whether it's all very independent of existing stuff, or whether it interacts in subtle ways. I had a look at the Wiki page and I really can't deduce the intended semantics. Would it be possible to add a bit more by way of general introduction to the API? And a function-by-function description saying what each does? I'm missing quite basic understandings like what things I can attach resource limits to (threads? arbitrary expressions?) Plus, how the tricky problems are addressed? For example, when a data structure is shared between two resource-limited things, which does it get charged to, if either? Perhaps it'd be worth emitting to Haskell Cafe the link to the design description, to encourage them to look at it, and take advantage of all the smart people there? Simon -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7763#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#7763: Resource limits for Haskell ---------------------------------+------------------------------------------ Reporter: ezyang | Owner: ezyang Type: feature request | Status: patch Priority: normal | Milestone: Component: Profiling | Version: 7.7 Keywords: | Os: Unknown/Multiple Architecture: Unknown/Multiple | Failure: None/Unknown Difficulty: Unknown | Testcase: Blockedby: | Blocking: Related: | ---------------------------------+------------------------------------------ Comment(by joeyadams): Nice! This addresses #5476. -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7763#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#7763: Resource limits for Haskell ---------------------------------+------------------------------------------ Reporter: ezyang | Owner: simonmar Type: feature request | Status: patch Priority: normal | Milestone: 7.8.1 Component: Profiling | Version: 7.7 Keywords: | Os: Unknown/Multiple Architecture: Unknown/Multiple | Failure: None/Unknown Difficulty: Unknown | Testcase: Blockedby: | Blocking: Related: | ---------------------------------+------------------------------------------ Changes (by igloo): * owner: ezyang => simonmar Comment: Simon, did you see this? Could you let us know if you have any comments please? -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7763#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#7763: Resource limits for Haskell ---------------------------------+------------------------------------------ Reporter: ezyang | Owner: simonmar Type: feature request | Status: patch Priority: normal | Milestone: 7.8.1 Component: Profiling | Version: 7.7 Keywords: | Os: Unknown/Multiple Architecture: Unknown/Multiple | Failure: None/Unknown Difficulty: Unknown | Testcase: Blockedby: | Blocking: Related: | ---------------------------------+------------------------------------------ Comment(by igloo): Generally, this looks OK to me, although I've only skimmed the actual patches. If I understand correctly, this would also allow users to programmatically create cost centres used in heap profiles too? But for that, rather than {{{ newCC :: IO CostCentre }}} we'd presumably want {{{ newCC :: String -> IO CostCentre }}} There are also a couple of scary looking comments: {{{ // XXX Synchronization??? // XXX wrong! }}} -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7763#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler