Is there any interest or movement in developing thread priority or any other realtime support in Haskell? Right now, if I have tasks that need to be responsive in real time, even if the realtime needs are very soft, it seems that the only option is to try to ensure that at least one hardware thread is kept clear of any other activity. To be very useful to me, thread priority would not need to come with very strict guarantees, but there would need to be a way to make sure that `par` sparks and DPH inherit the priority of the sparking thread. In part I ask because I'm working on a small library to support a degree of cooperative task prioritization. Friendly, --Lane
Count me in too I've got a library that endeavours to deliver 'rate-equivalence' - i.e there may be some jitter in when the events should have occurred but their long term rate of progress is stable. Testing has shown that I can get events to occur at the right time within 1ms (99%+ of the time, with 50%+ of the events < 0.5ms) - choose your O/S carefully though. And this is without resorting to marking the threads as real time with the kernel. If it matters you can tune the scheduling to further reduce the timer scheduling latency (at the cost of more CPU) and increase the fidelity of event timing another order of magnitude or more. As for 'garbage collection ruins my real time properties' argument - I've been pragmatic about this, you can configure the scheduler to go and perform a GC if it knows that it is going to sleep for a while (yes, this doesn't resolve the issues of external events arriving in that time but read on) but not perform that too often. Turning off GHCs timers (to stop it garbage collecting after it has actually been idle for a while) and having code that doesn't create vast amounts of garbage means that a GC takes <0.1ms. Now if I could see within Haskell the amount of heap that has been recently mutated and some other GC statistics when it was running I could even decide to schedule that more effectively. We use this to create synthetic network traffic sources and other timing/performance related activities in the wide area context - it serves our needs. Neil On 25 Apr 2009, at 03:48, Christopher Lane Hinson wrote:
Is there any interest or movement in developing thread priority or any other realtime support in Haskell?
Right now, if I have tasks that need to be responsive in real time, even if the realtime needs are very soft, it seems that the only option is to try to ensure that at least one hardware thread is kept clear of any other activity.
To be very useful to me, thread priority would not need to come with very strict guarantees, but there would need to be a way to make sure that `par` sparks and DPH inherit the priority of the sparking thread.
In part I ask because I'm working on a small library to support a degree of cooperative task prioritization.
Friendly, --Lane _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
semanticphilosopher:
Count me in too
I've got a library that endeavours to deliver 'rate-equivalence' - i.e there may be some jitter in when the events should have occurred but their long term rate of progress is stable.
Testing has shown that I can get events to occur at the right time within 1ms (99%+ of the time, with 50%+ of the events < 0.5ms) - choose your O/S carefully though. And this is without resorting to marking the threads as real time with the kernel. If it matters you can tune the scheduling to further reduce the timer scheduling latency (at the cost of more CPU) and increase the fidelity of event timing another order of magnitude or more.
As for 'garbage collection ruins my real time properties' argument - I've been pragmatic about this, you can configure the scheduler to go and perform a GC if it knows that it is going to sleep for a while (yes, this doesn't resolve the issues of external events arriving in that time but read on) but not perform that too often. Turning off GHCs timers (to stop it garbage collecting after it has actually been idle for a while) and having code that doesn't create vast amounts of garbage means that a GC takes <0.1ms.
Now if I could see within Haskell the amount of heap that has been recently mutated and some other GC statistics when it was running I could even decide to schedule that more effectively.
You could actually do this by importing symbols from Stats.c in the runtime (the same variables used for +RTS -sstderr output). I'm not sure anyone's ctually tried this yet though, but the effect would be a program that can monitor its own heap health. -- Don
semanticphilosopher:
Count me in too
I've got a library that endeavours to deliver 'rate-equivalence' - i.e there may be some jitter in when the events should have occurred but their long term rate of progress is stable.
Testing has shown that I can get events to occur at the right time within 1ms (99%+ of the time, with 50%+ of the events < 0.5ms) - choose your O/S carefully though. And this is without resorting to marking the threads as real time with the kernel. If it matters you can tune the scheduling to further reduce the timer scheduling latency (at the cost of more CPU) and increase the fidelity of event timing another order of magnitude or more.
As for 'garbage collection ruins my real time properties' argument - I've been pragmatic about this, you can configure the scheduler to go and perform a GC if it knows that it is going to sleep for a while (yes, this doesn't resolve the issues of external events arriving in that time but read on) but not perform that too often. Turning off GHCs timers (to stop it garbage collecting after it has actually been idle for a while) and having code that doesn't create vast amounts of garbage means that a GC takes <0.1ms.
Now if I could see within Haskell the amount of heap that has been recently mutated and some other GC statistics when it was running I could even decide to schedule that more effectively.
We use this to create synthetic network traffic sources and other timing/performance related activities in the wide area context - it serves our needs.
This is very interesting. Do you have any further documentation / notes on controlling jitteriness? -- Don
Christopher Lane Hinson wrote:
Is there any interest or movement in developing thread priority or any other realtime support in Haskell?
Right now, if I have tasks that need to be responsive in real time, even if the realtime needs are very soft, it seems that the only option is to try to ensure that at least one hardware thread is kept clear of any other activity.
To be very useful to me, thread priority would not need to come with very strict guarantees, but there would need to be a way to make sure that `par` sparks and DPH inherit the priority of the sparking thread.
In part I ask because I'm working on a small library to support a degree of cooperative task prioritization.
Friendly, --Lane
Yea, Thread priorities would be great. ATM you can multiplex IO at different OS-thread levels with IO completion ports etc, but it kinda sucks.
Hi all, Some two year I wrote a library to change thread priorities in Windows, including another library that allows real time processing. With this you can play midi files real time and without interruption from the OS. (Manipulating thread priorities is really easy in Windows, real time uninterrupted processing a little more complicated). Actually, I thought this problem was already resolved (haskore-realtime), but if there is interest I can see if I can dust them off, although I'm sure there will be still lots of things to improve. Right now I'm working in linux only, so it may take a few days to get a suitable xp version. (Actually, I would be really interested in a linux port, but have little time right now. Perhaps this library can help: http://sourceforge.net/projects/high-res-timers ?) Kind regards, Maarten Christopher Lane Hinson wrote:
Is there any interest or movement in developing thread priority or any other realtime support in Haskell?
Right now, if I have tasks that need to be responsive in real time, even if the realtime needs are very soft, it seems that the only option is to try to ensure that at least one hardware thread is kept clear of any other activity.
To be very useful to me, thread priority would not need to come with very strict guarantees, but there would need to be a way to make sure that `par` sparks and DPH inherit the priority of the sparking thread.
In part I ask because I'm working on a small library to support a degree of cooperative task prioritization.
Friendly, --Lane _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
participants (6)
-
Christopher Lane Hinson -
Don Stewart -
Henning Thielemann -
maarten -
Neal Alexander -
Neil Davies