RE: [Haskell-cafe] Garbage collection and finalizer thread priority

newer
(**) over integers

older
Generic types

Simon Marlow

13 Jun 2005 13 Jun '05

9:15 a.m.

On 13 June 2005 11:30, Gracjan Polak wrote:

...

My space problems continued... :)

I have some ForeignPtr's, with finalizers (finalizerFree only). And I have lazyly produced list of those (yes, there is some unsafePerformIO magic behind, but I think I got this right). The problem is I get out-of-memory condition because elements get produced faster than those consumed are garbage collected.

Example:

list = ... mapM_ writeOut list

writeOut :: Ptr Word8 writeOut dat = do hPutBuh handle dat 1024 -- Control.Concurrent.threadDelay 1000

Uncommenting this line allows gc thread to run finalizers, memory gets freed, everything runs smoothly...

As far as I know finalizers are run in separate thread. How do I increase priority of this thread so it runs faster?

I presume you're running GHC. There's no way to increase the priority of a thread - GHC's scheduler doesn't have a concept of priorities. I would look into whether you can use mallocForeignPtr: this is much faster than using newForeignPtr with finalizerFree, because the memory is garbage collected directly and there's no need for a finalizer. Cheers, Simon

Show replies by date

Gracjan Polak

13 Jun 13 Jun

9:32 a.m.

New subject: Garbage collection and finalizer thread priority

Simon Marlow wrote:

...

I presume you're running GHC. There's no way to increase the priority of a thread - GHC's scheduler doesn't have a concept of priorities.

Yes, I forgot to state it explicitly.

...

I would look into whether you can use mallocForeignPtr: this is much faster than using newForeignPtr with finalizerFree, because the memory is garbage collected directly and there's no need for a finalizer.

I use mallocBytes + reallocBytes + addForeignPtrFinalizer finalizerFree. I wouldn't think this is any different, thanks for your suggestion. May I ask where that difference comes from? -- Gracjan

Sebastian Sylvan

11:55 a.m.

New subject: Garbage collection and finalizer thread priority

On 6/13/05, Simon Marlow wrote:

...

I presume you're running GHC. There's no way to increase the priority of a thread - GHC's scheduler doesn't have a concept of priorities.

Just out of curiousity, what scheme does GHC use for scheduling threads? /S -- Sebastian Sylvan +46(0)736-818655 UIN: 44640862

Gracjan Polak

14 Jun 14 Jun

5:21 a.m.

New subject: Garbage collection and finalizer thread priority

Sebastian Sylvan wrote:

...

On 6/13/05, Simon Marlow wrote:

...
I presume you're running GHC. There's no way to increase the priority of a thread - GHC's scheduler doesn't have a concept of priorities.

Just out of curiousity, what scheme does GHC use for scheduling threads?

As I read somewhere, round robin... -- Gracjan

Gracjan Polak

5:29 a.m.

New subject: Garbage collection and finalizer thread priority

Simon Marlow wrote:

...

On 13 June 2005 11:30, Gracjan Polak wrote:

...
My space problems continued... :)

Follow up :)

...

...
I have some ForeignPtr's, with finalizers (finalizerFree only). And I have lazyly produced list of those (yes, there is some unsafePerformIO magic behind, but I think I got this right). The problem is I get out-of-memory condition because elements get produced faster than those consumed are garbage collected.

Example:

list = ... mapM_ writeOut list

writeOut :: Ptr Word8 writeOut dat = do hPutBuh handle dat 1024 -- Control.Concurrent.threadDelay 1000

This sould be: writeOut :: Ptr Word8 writeOut dat = do hPutBuh handle dat 1024 System.Mem.performGC Control.Concurrent.threadDelay 1000 My small experimets show, that gc is not triggered?!?! What are the conditions to trigger gc? As I read the docs, is some % of heap. Is mallocForeinPtrBytes counted into that %? Anyway I ended up triggering GC by hand and giving it time to run finalizers. Frankly speaking this is nonsolution:( Related question: Documentation says: -Msize [Default: unlimited] Set the maximum heap size to size bytes. The heap normally grows and shrinks according to the memory requirements of the program... GHC 6.4 says: Heap exhausted; Current maximum heap size is 268435456 bytes (256 Mb); use `+RTS -M<size>' to increase it. What am I missing? PS: I might be mistaken in any of above statements. I'll be thankful for any light... :) -- Gracjan

Bulat Ziganshin

8:04 a.m.

New subject: Re[2]: Garbage collection and finalizer thread priority

Hello Gracjan, Tuesday, June 14, 2005, 1:29:09 PM, you wrote: GP> Documentation says: GP> -Msize GP> [Default: unlimited] Set the maximum heap size to size bytes. The GP> heap normally grows and shrinks according to the memory requirements of GP> the program... GP> GHC 6.4 says: GP> Heap exhausted; GP> Current maximum heap size is 268435456 bytes (256 Mb); GP> use `+RTS -M<size>' to increase it. it's an error in GHC 6.4/win32 -- Best regards, Bulat mailto:bulatz@HotPOP.com

Gracjan Polak

8:36 a.m.

New subject: Garbage collection and finalizer thread priority

Bulat Ziganshin wrote:

...

Hello Gracjan,

Tuesday, June 14, 2005, 1:29:09 PM, you wrote: GP> Documentation says: GP> -Msize GP> [Default: unlimited] Set the maximum heap size to size bytes. The GP> heap normally grows and shrinks according to the memory requirements of GP> the program...

GP> GHC 6.4 says: GP> Heap exhausted; GP> Current maximum heap size is 268435456 bytes (256 Mb); GP> use `+RTS -M<size>' to increase it.

it's an error in GHC 6.4/win32

:) How do I take care of that? -- Gracjan

Bulat Ziganshin

10:43 a.m.

New subject: Re[2]: Garbage collection and finalizer thread priority

Hello Gracjan, Tuesday, June 14, 2005, 4:36:26 PM, you wrote:

...

...
it's an error in GHC 6.4/win32

GP> :)

GP> How do I take care of that? use `+RTS -M<size>' to increase it. you can include this optiion in executable, so user don't need to specify it each time. see somewhere in GHC docs -- Best regards, Bulat mailto:bulatz@HotPOP.com

7281

Age (days ago)

7282

Last active (days ago)

List overview

Download

7 comments

4 participants

participants (4)

Bulat Ziganshin
Gracjan Polak
Sebastian Sylvan
Simon Marlow