Memory and Threads - MVars or TVars
Hi everyone. I was wondering if someone could just guide me toward some good information, but if anyone wants to help with a personal explanation I welcome it. I'm trying to write a threaded program and I'm not sure how to manage my memory. I read up on MVars and they make a lot of sense. My real question is what is "atomic" and how does it apply to TVars? I don't understand what atomic transactions are and I can't seem to find a concise explanation. I also saw some stuff about TMVars? But I can't find much on them either. Any help would be appreciated. -Eitan
Hi Eitan, I'm right now approaching the subject of concurrency myself for the first time in an application that spiders web pages. The getting the web pages part via http request is the one that is time consuming and thus the one that I wish to "concurrentalize", ie. getting up to 6 six pages concurrently at a time. From what I've learned so far it seems that there are the following approaches to concurrency in haskell: 1. use a primitive approach with the concurrency primitives that haskell / ghc provides, ie. locks, MVars, TVars etc. directly 2. use one abstraction level up, ie. use STM (Software Transactional Memory), there is a chapter in RWH about it.(also available online). 3. use yet another abstraction level up by using the orc library from the galois people, available on hackage. Documentation to this one is a paper available on the galois website. I am currently at the very beginning of familiarizing myself with this approach but it seems the most feasible way to do concurrency. Using this approach is pretty much like not having to worry about garbage collection. Even using STM you still have to do a lot of your own manual forkIO, putVar, kill etc. Best regards Günther Am 29.07.10 02:23, schrieb Eitan Goldshtrom:
Hi everyone. I was wondering if someone could just guide me toward some good information, but if anyone wants to help with a personal explanation I welcome it. I'm trying to write a threaded program and I'm not sure how to manage my memory. I read up on MVars and they make a lot of sense. My real question is what is "atomic" and how does it apply to TVars? I don't understand what atomic transactions are and I can't seem to find a concise explanation. I also saw some stuff about TMVars? But I can't find much on them either. Any help would be appreciated.
-Eitan
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
I can't believe I never knew about Cabal. I'm getting Orc now. It looks very interesting. Thanks for the help. -Eitan On 7/28/2010 9:17 PM, Günther Schmidt wrote:
Hi Eitan,
I'm right now approaching the subject of concurrency myself for the first time in an application that spiders web pages. The getting the web pages part via http request is the one that is time consuming and thus the one that I wish to "concurrentalize", ie. getting up to 6 six pages concurrently at a time.
From what I've learned so far it seems that there are the following approaches to concurrency in haskell:
1. use a primitive approach with the concurrency primitives that haskell / ghc provides, ie. locks, MVars, TVars etc. directly
2. use one abstraction level up, ie. use STM (Software Transactional Memory), there is a chapter in RWH about it.(also available online).
3. use yet another abstraction level up by using the orc library from the galois people, available on hackage. Documentation to this one is a paper available on the galois website. I am currently at the very beginning of familiarizing myself with this approach but it seems the most feasible way to do concurrency. Using this approach is pretty much like not having to worry about garbage collection. Even using STM you still have to do a lot of your own manual forkIO, putVar, kill etc.
Best regards
Günther
Am 29.07.10 02:23, schrieb Eitan Goldshtrom:
Hi everyone. I was wondering if someone could just guide me toward some good information, but if anyone wants to help with a personal explanation I welcome it. I'm trying to write a threaded program and I'm not sure how to manage my memory. I read up on MVars and they make a lot of sense. My real question is what is "atomic" and how does it apply to TVars? I don't understand what atomic transactions are and I can't seem to find a concise explanation. I also saw some stuff about TMVars? But I can't find much on them either. Any help would be appreciated.
-Eitan
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Atomic operations are special operations where you don't have to worry about
some other process messing with things while the operation is taking place.
For a simple example of why atomic operations are important:
(taken from: http://en.wikipedia.org/wiki/Linearizability#Non-atomic)
The naive, non-atomic implementation:
1. reads the value in the memory location;
2. adds one to the value;
3. writes the new value back into the memory location.
Now, imagine two processes are running incrementing a single, shared memory
location:
1. the first process reads the value in memory location;
2. the first process adds one to the value;
but before it can write the new value back to the memory location it is
suspended, and the second process is allowed to run:
1. the second process reads the value in memory location, the *same* value
that the first process read;
2. the second process adds one to the value;
3. the second process writes the new value into the memory location.
The second process is suspended and the first process allowed to run again:
1. the first process writes a now-wrong value into the memory location,
unaware that the other process has already updated the value in the memory
location.
Atomic operations fix this problem by preventing (STM is a little fancier
and doesn't actually _prevent_, but you can pretend that it does) any other
process from writing to the memory in question until the computation is
finished and the result is written back.
For many simple cases something like atomicModifyIORef is all you really
need. However, if you have cases where you need to make sure _multiple_
IORefs/MVars/TVars/etc.. are not written to until you're finished then you
really need something like STMs 'atomically' function. Which runs a block of
STM operations "atomically".
http://en.wikipedia.org/wiki/Linearizability#Non-atomicHope that helps,
- Job
On Wed, Jul 28, 2010 at 8:23 PM, Eitan Goldshtrom
Hi everyone. I was wondering if someone could just guide me toward some good information, but if anyone wants to help with a personal explanation I welcome it. I'm trying to write a threaded program and I'm not sure how to manage my memory. I read up on MVars and they make a lot of sense. My real question is what is "atomic" and how does it apply to TVars? I don't understand what atomic transactions are and I can't seem to find a concise explanation. I also saw some stuff about TMVars? But I can't find much on them either. Any help would be appreciated.
-Eitan
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Ah! That clears that up a lot. I read the wiki page but something just didn't make full sense about it until you used the word "prevent". I understand that the computer doesn't actually prevent other threads from running -- that would defeat the purpose of the concurrency -- but it helped clear it up. Perhaps you guys could help me with Cabal now though? I'm trying to install Orc but it wants base>=4.2 and <=4.3 and I have 4.1 after installing the latest release of GHC. Cabal won't upgrade the base. It complains about a dependency to "integer-simple". Anyone know what that's about? -Eitan
On Thu, Jul 29, 2010 at 3:49 AM, Eitan Goldshtrom
Perhaps you guys could help me with Cabal now though? I'm trying to install Orc but it wants base>=4.2 and <=4.3 and I have 4.1 after installing the latest release of GHC. Cabal won't upgrade the base. It complains about a dependency to "integer-simple". Anyone know what that's about?
The latest version of GHC (6.12) comes with base 4.2. In general, base versions are tied to compiler versions, and you can't upgrade base on its own.
You might try pulling downloading the package ('cabal fetch org' will do
this) and changing the base dependency (to >= 4.1) in the orc.cabal file and
then build it manually (cabal configure && cabal build && cabal install
(while in the same directory as the .cabal file)) and see what happens.
I don't see any obvious reasons why it would need a version greater than
6.10, so it might just be an over restrictive dependency rule, but I might
be missing something.
- Job
On Wed, Jul 28, 2010 at 10:49 PM, Eitan Goldshtrom
Ah! That clears that up a lot. I read the wiki page but something just didn't make full sense about it until you used the word "prevent". I understand that the computer doesn't actually prevent other threads from running -- that would defeat the purpose of the concurrency -- but it helped clear it up. Perhaps you guys could help me with Cabal now though? I'm trying to install Orc but it wants base>=4.2 and <=4.3 and I have 4.1 after installing the latest release of GHC. Cabal won't upgrade the base. It complains about a dependency to "integer-simple". Anyone know what that's about?
-Eitan _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
On Thu, Jul 29, 2010 at 6:53 AM, Job Vranish
You might try pulling downloading the package ('cabal fetch org' will do this) and changing the base dependency (to >= 4.1) in the orc.cabal file
cabal also has an 'unpack' command for the particularly lazy (me). Ex: cabal unpack orc ; sed "s/base\W*>= 4.2/base >= 4.1/" orc*/*.cabal ; cd orc* ; cabal install Should unpack, fix the .cabal file, and install. Cheers, Thomas
This could be useful: Beautiful concurrency by Simon Peyton Jones
http://research.microsoft.com/en-us/um/people/simonpj/papers/stm/beautiful.p...
On 29 July 2010 02:23, Eitan Goldshtrom
Hi everyone. I was wondering if someone could just guide me toward some good information, but if anyone wants to help with a personal explanation I welcome it. I'm trying to write a threaded program and I'm not sure how to manage my memory. I read up on MVars and they make a lot of sense. My real question is what is "atomic" and how does it apply to TVars? I don't understand what atomic transactions are and I can't seem to find a concise explanation. I also saw some stuff about TMVars? But I can't find much on them either. Any help would be appreciated.
-Eitan
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
participants (6)
-
Ben Millwood -
Christopher Done -
Eitan Goldshtrom -
Günther Schmidt -
Job Vranish -
Thomas DuBuisson