I'm not the person to ask :) Maybe someone else on the list can chip in here. Another question if you don't mind (I'm learning here): you say below that in atomicModifyIORef2Native, you rely on heap object layout to unsafeCoerce between a pair and any object with an interesting value in the first field. But looking at the source [1], it seems you only do this for base < 4.13, i.e. GHC < 8.8.4. That's been a while. Is it true that for GHCs since 8.8.4 you can just use a primop to do what you want directly, without the magic Generic stuff? If so, perhaps you can even drop the Generic constraints for high enough GHC versions (by extending the scope of the CPP slightly)? 8.6.5 is still popular, but simultaneously a whole bunch of applications don't care about GHCs <9 anymore. - Tom [1]: https://hackage.haskell.org/package/atomic-modify-general-0.1.0.0/docs/src/D... On 10/03/2023 09:45, David Feuer wrote:

Whoops! Thanks for pointing that out. I'll fix it and push new docs. There's not much magic in that part of the code itself; the nasty magic is knowing that the first pointer in any record is in the same position in its heap object as the first component of a pair. Speaking of which, do you have any idea if it'll work for non-record types whose constructors all have the same first field? I'm guessing yes, but I haven't experimented yet.

On Fri, Mar 10, 2023, 3:27 AM Tom Smeding wrote:

Hi David,

Fancy stuffs!

Wondering how much magic was going on in implementing this, I saw that atomicModifyIORef2Native misses the haddock marker '|' in the source; thus your extensive doc comment doesn't show up on hackage.

Cheers, Tom

On 10/03/2023 03:01, David Feuer wrote: > I just put together a new package, atomic-modify-general, for > generalizations of the `atomicModifyIORef` operation. In particular: > > 1. Versions that allow a result of an arbitrary type (not necessarily > a pair), where the caller passes in an extraction function. These work > with `Array` and `SmallArray` from `primitive` as well as `IORef`. > 2. A version that works with record types (not just pairs) whose first > field is the new value to install in the `IORef`. This uses > implementation details of the `atomicModifyMutVar#` primop as well as > GHC's heap object layout to achieve better performance. > > Please try it out and let me know how it goes, and what extras you may want! > > Hackage: https://hackage.haskell.org/package/atomic-modify-general-0.1.0.0 > _______________________________________________ > Haskell-Cafe mailing list > To (un)subscribe, modify options or view archives go to: > http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe > Only members subscribed via the mailman list are allowed to post.

_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

I expect it also can't know if the first field got unpacked. (or will be unpacked? I think that hasn't happened at typechecking time) On Fri, Mar 10, 2023 at 9:06 AM David Feuer wrote:

Unfortunately not. The type the type checker is told for the `atomicModifyMutVar2#` primop is a lie (too permissive). The type documented in its Haddocks is also a lie (too restrictive). Its most precise type, as I understand it, is (translating away the unboxed stuff and the internal I/O representation)

atomicModifyMutVar2 :: IORef a -> (a -> t) -> IO (a, t) -- with the condition that `t` is a record whose first field that is represented by a pointer (what we call BoxedRep) has a type with the same memory representation as `a`.

While it would be possible to express that fairly precisely using the Generics approach, I think it's too hard to think about "the first pointer field", and the "same memory representation" bit mucks with type inference. So I restricted it a little to "the first field, which must be a pointer", and "with type `a`".

There's a sort of "hidden unsafeCoerce" in the primop call, where we know that the "selector thunk" it builds (a thunk applying a record selector, in this case fst) will extract the field we want out of what we give it. The type checker is clueless and just accepts whatever.

On Fri, Mar 10, 2023, 4:09 AM Tom Smeding wrote:

...

I'm not the person to ask :) Maybe someone else on the list can chip in here.

Another question if you don't mind (I'm learning here): you say below that in atomicModifyIORef2Native, you rely on heap object layout to unsafeCoerce between a pair and any object with an interesting value in the first field. But looking at the source [1], it seems you only do this for base < 4.13, i.e. GHC < 8.8.4. That's been a while.

Is it true that for GHCs since 8.8.4 you can just use a primop to do what you want directly, without the magic Generic stuff? If so, perhaps you can even drop the Generic constraints for high enough GHC versions (by extending the scope of the CPP slightly)? 8.6.5 is still popular, but simultaneously a whole bunch of applications don't care about GHCs <9 anymore.

- Tom

[1]: https://hackage.haskell.org/package/atomic-modify-general-0.1.0.0/docs/src/D...

On 10/03/2023 09:45, David Feuer wrote:

Whoops! Thanks for pointing that out. I'll fix it and push new docs. There's not much magic in that part of the code itself; the nasty magic is knowing that the first pointer in any record is in the same position in its heap object as the first component of a pair. Speaking of which, do you have any idea if it'll work for non-record types whose constructors all have the same first field? I'm guessing yes, but I haven't experimented yet.

On Fri, Mar 10, 2023, 3:27 AM Tom Smeding wrote:

...

Hi David,

Fancy stuffs!

Wondering how much magic was going on in implementing this, I saw that atomicModifyIORef2Native misses the haddock marker '|' in the source; thus your extensive doc comment doesn't show up on hackage.

Cheers, Tom

On 10/03/2023 03:01, David Feuer wrote:

...

I just put together a new package, atomic-modify-general, for generalizations of the `atomicModifyIORef` operation. In particular:

1. Versions that allow a result of an arbitrary type (not necessarily a pair), where the caller passes in an extraction function. These work with `Array` and `SmallArray` from `primitive` as well as `IORef`. 2. A version that works with record types (not just pairs) whose first field is the new value to install in the `IORef`. This uses implementation details of the `atomicModifyMutVar#` primop as well as GHC's heap object layout to achieve better performance.

Please try it out and let me know how it goes, and what extras you may want!

Hackage: https://hackage.haskell.org/package/atomic-modify-general-0.1.0.0 _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

-- brandon s allbery kf8nh allbery.b@gmail.com

Ooh, that's decidedly nasty. I suppose I should have read the docs of the primop before drawing conclusions, but then, being lazy is part of the deal here... ;) Thanks for the explanation! - Tom On 10/03/2023 15:05, David Feuer wrote:

Unfortunately not. The type the type checker is told for the `atomicModifyMutVar2#` primop is a lie (too permissive). The type documented in its Haddocks is also a lie (too restrictive). Its most precise type, as I understand it, is (translating away the unboxed stuff and the internal I/O representation)

atomicModifyMutVar2   :: IORef a   -> (a -> t)   -> IO (a, t) -- with the condition that `t` is a record whose first field that is represented by a pointer (what we call BoxedRep) has a type with the same memory representation as `a`.

While it would be possible to express that fairly precisely using the Generics approach, I think it's too hard to think about "the first pointer field", and the "same memory representation" bit mucks with type inference. So I restricted it a little to "the first field, which must be a pointer", and "with type `a`".

There's a sort of "hidden unsafeCoerce" in the primop call, where we know that the "selector thunk" it builds (a thunk applying a record selector, in this case fst) will extract the field we want out of what we give it. The type checker is clueless and just accepts whatever.

On Fri, Mar 10, 2023, 4:09 AM Tom Smeding wrote:

I'm not the person to ask :) Maybe someone else on the list can chip in here.

Another question if you don't mind (I'm learning here): you say below that in atomicModifyIORef2Native, you rely on heap object layout to unsafeCoerce between a pair and any object with an interesting value in the first field. But looking at the source [1], it seems you only do this for base < 4.13, i.e. GHC < 8.8.4. That's been a while.

Is it true that for GHCs since 8.8.4 you can just use a primop to do what you want directly, without the magic Generic stuff? If so, perhaps you can even drop the Generic constraints for high enough GHC versions (by extending the scope of the CPP slightly)? 8.6.5 is still popular, but simultaneously a whole bunch of applications don't care about GHCs <9 anymore.

- Tom

[1]: https://hackage.haskell.org/package/atomic-modify-general-0.1.0.0/docs/src/D...

On 10/03/2023 09:45, David Feuer wrote:

...

Whoops! Thanks for pointing that out. I'll fix it and push new docs. There's not much magic in that part of the code itself; the nasty magic is knowing that the first pointer in any record is in the same position in its heap object as the first component of a pair. Speaking of which, do you have any idea if it'll work for non-record types whose constructors all have the same first field? I'm guessing yes, but I haven't experimented yet.

On Fri, Mar 10, 2023, 3:27 AM Tom Smeding wrote:

Hi David,

Fancy stuffs!

Wondering how much magic was going on in implementing this, I saw that atomicModifyIORef2Native misses the haddock marker '|' in the source; thus your extensive doc comment doesn't show up on hackage.

Cheers, Tom

On 10/03/2023 03:01, David Feuer wrote: > I just put together a new package, atomic-modify-general, for > generalizations of the `atomicModifyIORef` operation. In particular: > > 1. Versions that allow a result of an arbitrary type (not necessarily > a pair), where the caller passes in an extraction function. These work > with `Array` and `SmallArray` from `primitive` as well as `IORef`. > 2. A version that works with record types (not just pairs) whose first > field is the new value to install in the `IORef`. This uses > implementation details of the `atomicModifyMutVar#` primop as well as > GHC's heap object layout to achieve better performance. > > Please try it out and let me know how it goes, and what extras you may want! > > Hackage: https://hackage.haskell.org/package/atomic-modify-general-0.1.0.0 > _______________________________________________ > Haskell-Cafe mailing list > To (un)subscribe, modify options or view archives go to: > http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe > Only members subscribed via the mailman list are allowed to post.

_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.