Simplest way to move a ... `using` ... line into the IO monad?

David Banas

27 Oct 2023 27 Oct '23

2:36 p.m.

Hi all, What is the simplest way to translate a non-monadic line of Haskell code, such as: res = map foo xs `using` parList rseq For use in the IO monad? This doesn’t work: res <- mapM fooM xs `using` parTraversable rseq complaining: No Traversable instance for IO. Thanks, -db

Attachments:

attachment.html (text/html — 1.4 KB)

Show replies by date

Tom Ellis

27 Oct 27 Oct

2:43 p.m.

New subject: Simplest way to move a ... `using` ... line into the IO monad?

On Fri, Oct 27, 2023 at 11:36:53AM -0700, David Banas wrote:

...

What is the simplest way to translate a non-monadic line of Haskell code, such as:

res = map foo xs `using` parList rseq

For use in the IO monad?

This doesn’t work:

res <- mapM fooM xs `using` parTraversable rseq

The docs for Control.Parallel.Strategies say "The parallel tasks evaluated by a Strategy may have no side effects. For non-deterministic parallel programming, see Control.Concurrent."[1]. So I guess you should look into Control.Concurrent[2]. Tom [1] https://www.stackage.org/haddock/lts-21.17/parallel-3.2.2.0/Control-Parallel... [2] https://www.stackage.org/haddock/lts-21.17/base-4.17.2.0/Control-Concurrent....

Brandon Allbery

2:48 p.m.

New subject: Simplest way to move a ... `using` ... line into the IO monad?

Doesn't `async` have things for this? On Fri, Oct 27, 2023 at 6:43 PM Tom Ellis < tom-lists-haskell-cafe-2023@jaguarpaw.co.uk> wrote:

...

On Fri, Oct 27, 2023 at 11:36:53AM -0700, David Banas wrote:

...
What is the simplest way to translate a non-monadic line of Haskell code, such as:

res = map foo xs `using` parList rseq

For use in the IO monad?

This doesn’t work:

res <- mapM fooM xs `using` parTraversable rseq

The docs for Control.Parallel.Strategies say "The parallel tasks evaluated by a Strategy may have no side effects. For non-deterministic parallel programming, see Control.Concurrent."[1]. So I guess you should look into Control.Concurrent[2].

Tom

[1] https://www.stackage.org/haddock/lts-21.17/parallel-3.2.2.0/Control-Parallel...

[2] https://www.stackage.org/haddock/lts-21.17/base-4.17.2.0/Control-Concurrent.... _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

-- brandon s allbery kf8nh allbery.b@gmail.com

Viktor Dukhovni

2:51 p.m.

New subject: Simplest way to move a ... `using` ... line into the IO monad?

On Fri, Oct 27, 2023 at 07:43:07PM +0100, Tom Ellis wrote:

...

On Fri, Oct 27, 2023 at 11:36:53AM -0700, David Banas wrote:

...
What is the simplest way to translate a non-monadic line of Haskell code, such as:

res = map foo xs `using` parList rseq

For use in the IO monad?

This doesn’t work:

res <- mapM fooM xs `using` parTraversable rseq

The docs for Control.Parallel.Strategies say "The parallel tasks evaluated by a Strategy may have no side effects. For non-deterministic parallel programming, see Control.Concurrent."[1]. So I guess you should look into Control.Concurrent[2].

For a more friendly interface: https://hackage.haskell.org/package/async-2.2.4/docs/Control-Concurrent-Asyn... If the number of concurrent tasks is reasonably modest, a good fit could be `mapConcurrently`. For much larger numbers of tasks, it is best to run the jobs at a bounded concurrency, which may bring a BoundedChan to which the items to process are written, another to which results are written and a set of threads that do the processing, repeatedly reading more data until the channel is drained. *The* book on subject is: https://simonmar.github.io/pages/pcph.html -- Viktor.

David Banas

3:04 p.m.

New subject: Simplest way to move a ... `using` ... line into the IO monad?

Thank you all for the quick replies! `mapConcurrently` was exactly what I needed. :) -db

...

On Oct 27, 2023, at 11:51 AM, Viktor Dukhovni wrote:

On Fri, Oct 27, 2023 at 07:43:07PM +0100, Tom Ellis wrote:

...
On Fri, Oct 27, 2023 at 11:36:53AM -0700, David Banas wrote:

...
What is the simplest way to translate a non-monadic line of Haskell code, such as:

res = map foo xs `using` parList rseq

For use in the IO monad?

This doesn’t work:

res <- mapM fooM xs `using` parTraversable rseq

The docs for Control.Parallel.Strategies say "The parallel tasks evaluated by a Strategy may have no side effects. For non-deterministic parallel programming, see Control.Concurrent."[1]. So I guess you should look into Control.Concurrent[2].

For a more friendly interface:

https://hackage.haskell.org/package/async-2.2.4/docs/Control-Concurrent-Asyn... https://hackage.haskell.org/package/async-2.2.4/docs/Control-Concurrent-Asyn...

If the number of concurrent tasks is reasonably modest, a good fit could be `mapConcurrently`.

For much larger numbers of tasks, it is best to run the jobs at a bounded concurrency, which may bring a BoundedChan to which the items to process are written, another to which results are written and a set of threads that do the processing, repeatedly reading more data until the channel is drained.

*The* book on subject is: https://simonmar.github.io/pages/pcph.html https://simonmar.github.io/pages/pcph.html

-- Viktor. _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

David Banas

3:14 p.m.

New subject: Simplest way to move a ... `using` ... line into the IO monad?

After switching to `mapConcurrently` I’m getting this: SPARKS: 0 (0 converted, 0 overflowed, 0 dud, 0 GC'd, 0 fizzled) Is that to be expected? -db

...

On Oct 27, 2023, at 11:51 AM, Viktor Dukhovni wrote:

On Fri, Oct 27, 2023 at 07:43:07PM +0100, Tom Ellis wrote:

...
On Fri, Oct 27, 2023 at 11:36:53AM -0700, David Banas wrote:

...
What is the simplest way to translate a non-monadic line of Haskell code, such as:

res = map foo xs `using` parList rseq

For use in the IO monad?

This doesn’t work:

res <- mapM fooM xs `using` parTraversable rseq

The docs for Control.Parallel.Strategies say "The parallel tasks evaluated by a Strategy may have no side effects. For non-deterministic parallel programming, see Control.Concurrent."[1]. So I guess you should look into Control.Concurrent[2].

For a more friendly interface:

https://hackage.haskell.org/package/async-2.2.4/docs/Control-Concurrent-Asyn... https://hackage.haskell.org/package/async-2.2.4/docs/Control-Concurrent-Asyn...

If the number of concurrent tasks is reasonably modest, a good fit could be `mapConcurrently`.

For much larger numbers of tasks, it is best to run the jobs at a bounded concurrency, which may bring a BoundedChan to which the items to process are written, another to which results are written and a set of threads that do the processing, repeatedly reading more data until the channel is drained.

*The* book on subject is: https://simonmar.github.io/pages/pcph.html https://simonmar.github.io/pages/pcph.html

-- Viktor. _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

Brandon Allbery

3:23 p.m.

New subject: Simplest way to move a ... `using` ... line into the IO monad?

Yes, because `async` uses actual threads, not sparks like `Control.Parallel.Strategies`. On Fri, Oct 27, 2023 at 7:14 PM David Banas wrote:

...

After switching to `mapConcurrently` I’m getting this:

SPARKS: 0 (0 converted, 0 overflowed, 0 dud, 0 GC'd, 0 fizzled)

Is that to be expected?

-db

On Oct 27, 2023, at 11:51 AM, Viktor Dukhovni wrote:

On Fri, Oct 27, 2023 at 07:43:07PM +0100, Tom Ellis wrote:

On Fri, Oct 27, 2023 at 11:36:53AM -0700, David Banas wrote:

What is the simplest way to translate a non-monadic line of Haskell code, such as:

res = map foo xs `using` parList rseq

For use in the IO monad?

This doesn’t work:

res <- mapM fooM xs `using` parTraversable rseq

The docs for Control.Parallel.Strategies say "The parallel tasks evaluated by a Strategy may have no side effects. For non-deterministic parallel programming, see Control.Concurrent."[1]. So I guess you should look into Control.Concurrent[2].

For a more friendly interface:

https://hackage.haskell.org/package/async-2.2.4/docs/Control-Concurrent-Asyn...

If the number of concurrent tasks is reasonably modest, a good fit could be `mapConcurrently`.

For much larger numbers of tasks, it is best to run the jobs at a bounded concurrency, which may bring a BoundedChan to which the items to process are written, another to which results are written and a set of threads that do the processing, repeatedly reading more data until the channel is drained.

*The* book on subject is: https://simonmar.github.io/pages/pcph.html

-- Viktor. _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

-- brandon s allbery kf8nh allbery.b@gmail.com

571

Age (days ago)

571

Last active (days ago)

List overview

Download

6 comments

4 participants

participants (4)

Brandon Allbery
David Banas
Tom Ellis
Viktor Dukhovni