Indeed. The monad transformer instances for primmonad need to live in primmonad OR transformers to avoid orphans. Either way, unless transformers moves into base (unlikely), no way anything using prim monad will. On Thu, Oct 25, 2018 at 3:34 PM Andrew Martin wrote:

I like the idea of moving the type Addr into base. But we cannot move the entire module since it has functions that talk about PrimMonad, and we definitely don't want to move that into base.

On Thu, Oct 25, 2018 at 11:25 AM Daniel Cartwright wrote:

...

Motivation: There are a lot of places in base where 'Ptr a' is used in place of 'Addr', because in base there is no 'Addr', only 'Addr#'. The problem lies in the fact that many of these uses of 'Ptr a' are lying; the 'a' value is meaningless. Authors of functions therein have used things like 'Ptr ()', 'Ptr Word8', 'Ptr a', but these types do not mean what they say they mean - they're just Addr. There are probably other motivations for this that I can't think of off the top of my head right now. _______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

-- -Andrew Thaddeus Martin _______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

hrmm, what are the pieces of base that are using Ptr when they really should be using Addr? This would help me understand what would be made better in base :) On Thu, Oct 25, 2018 at 6:19 PM David Feuer wrote:

We shouldn't really need to move anything into base except Addr and its base instances.

On Oct 25, 2018 5:59 PM, "Carter Schonwald" wrote:

Indeed. The monad transformer instances for primmonad need to live in primmonad OR transformers to avoid orphans.

Either way, unless transformers moves into base (unlikely), no way anything using prim monad will.

On Thu, Oct 25, 2018 at 3:34 PM Andrew Martin wrote:

...

I like the idea of moving the type Addr into base. But we cannot move the entire module since it has functions that talk about PrimMonad, and we definitely don't want to move that into base.

On Thu, Oct 25, 2018 at 11:25 AM Daniel Cartwright wrote:

...

Motivation: There are a lot of places in base where 'Ptr a' is used in place of 'Addr', because in base there is no 'Addr', only 'Addr#'. The problem lies in the fact that many of these uses of 'Ptr a' are lying; the 'a' value is meaningless. Authors of functions therein have used things like 'Ptr ()', 'Ptr Word8', 'Ptr a', but these types do not mean what they say they mean - they're just Addr. There are probably other motivations for this that I can't think of off the top of my head right now. _______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

-- -Andrew Thaddeus Martin _______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

Now, one could argue that `Ptr ()` isn't a lie, it sort of reads like C's void pointer. But surely something like `Ptr Word8` is a lie, when it is not actually a Ptr to Word8 values. On Thu, Oct 25, 2018 at 8:11 PM Daniel Cartwright wrote:

yes, only the type and its instances should be moved as far as i'm aware.

Also, it's more than just base.

this Ptr is a lie: https://hackage.haskell.org/package/ghc-compact-0.1.0.0/docs/GHC-Compact-Ser... these Ptrs are lies: https://hackage.haskell.org/package/base-4.12.0.0/docs/GHC-IO-Handle.html in GHC.Stats, the foreign import "getRTSStats" has `Ptr () -> IO ()`, this Ptr () is also a lie

These are just off the top of my head, there are more

On Thu, Oct 25, 2018 at 6:46 PM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

hrmm, what are the pieces of base that are using Ptr when they really should be using Addr? This would help me understand what would be made better in base :)

On Thu, Oct 25, 2018 at 6:19 PM David Feuer wrote:

...

We shouldn't really need to move anything into base except Addr and its base instances.

On Oct 25, 2018 5:59 PM, "Carter Schonwald" wrote:

Indeed. The monad transformer instances for primmonad need to live in primmonad OR transformers to avoid orphans.

Either way, unless transformers moves into base (unlikely), no way anything using prim monad will.

On Thu, Oct 25, 2018 at 3:34 PM Andrew Martin wrote:

...

I like the idea of moving the type Addr into base. But we cannot move the entire module since it has functions that talk about PrimMonad, and we definitely don't want to move that into base.

On Thu, Oct 25, 2018 at 11:25 AM Daniel Cartwright < chessai1996@gmail.com> wrote:

...

Motivation: There are a lot of places in base where 'Ptr a' is used in place of 'Addr', because in base there is no 'Addr', only 'Addr#'. The problem lies in the fact that many of these uses of 'Ptr a' are lying; the 'a' value is meaningless. Authors of functions therein have used things like 'Ptr ()', 'Ptr Word8', 'Ptr a', but these types do not mean what they say they mean - they're just Addr. There are probably other motivations for this that I can't think of off the top of my head right now. _______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

-- -Andrew Thaddeus Martin _______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

Pretending we’re talking about this with storable as our example semantics : Ptr Void Corresponds to a memory address you don’t want to read or write to. Ptr () corresponds to a memory location that’s pretty boring to read / write to It’s definitely also true that for nontrivial c data structures , storable isn’t the right model for how to interact with the associated pointers. On Thu, Oct 25, 2018 at 11:22 PM Carter Schonwald < carter.schonwald@gmail.com> wrote:

... when is a valid pointer not going to point at byte addressable memory on memory architectures ghc can support or target?

On Thu, Oct 25, 2018 at 8:27 PM Daniel Cartwright wrote:

...

Now, one could argue that `Ptr ()` isn't a lie, it sort of reads like C's void pointer. But surely something like `Ptr Word8` is a lie, when it is not actually a Ptr to Word8 values.

On Thu, Oct 25, 2018 at 8:11 PM Daniel Cartwright wrote:

...

yes, only the type and its instances should be moved as far as i'm aware.

Also, it's more than just base.

this Ptr is a lie: https://hackage.haskell.org/package/ghc-compact-0.1.0.0/docs/GHC-Compact-Ser... these Ptrs are lies: https://hackage.haskell.org/package/base-4.12.0.0/docs/GHC-IO-Handle.html in GHC.Stats, the foreign import "getRTSStats" has `Ptr () -> IO ()`, this Ptr () is also a lie

These are just off the top of my head, there are more

On Thu, Oct 25, 2018 at 6:46 PM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

hrmm, what are the pieces of base that are using Ptr when they really should be using Addr? This would help me understand what would be made better in base :)

On Thu, Oct 25, 2018 at 6:19 PM David Feuer wrote:

...

We shouldn't really need to move anything into base except Addr and its base instances.

On Oct 25, 2018 5:59 PM, "Carter Schonwald" < carter.schonwald@gmail.com> wrote:

Indeed. The monad transformer instances for primmonad need to live in primmonad OR transformers to avoid orphans.

Either way, unless transformers moves into base (unlikely), no way anything using prim monad will.

On Thu, Oct 25, 2018 at 3:34 PM Andrew Martin < andrew.thaddeus@gmail.com> wrote:

...

I like the idea of moving the type Addr into base. But we cannot move the entire module since it has functions that talk about PrimMonad, and we definitely don't want to move that into base.

On Thu, Oct 25, 2018 at 11:25 AM Daniel Cartwright < chessai1996@gmail.com> wrote:

> Motivation: There are a lot of places in base where 'Ptr a' is used > in place of 'Addr', because in base there is no 'Addr', only 'Addr#'. The > problem lies in the fact that many of these uses of 'Ptr a' are lying; the > 'a' value is meaningless. Authors of functions therein have used things > like 'Ptr ()', 'Ptr Word8', 'Ptr a', but these types do not mean what they > say they mean - they're just Addr. There are probably other motivations for > this that I can't think of off the top of my head right now. > _______________________________________________ > Libraries mailing list > Libraries@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries >

-- -Andrew Thaddeus Martin _______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

Cool. What apis in base should move to use Addr from fake pointer ? Cause just adding it in isolation seems lame! I guess I’m just trying to say “what would the impact on base, if every fake Ptr was moved to be Addr?” Because it’s not something which should be done by halves. 1) what apis would be changed? 2) what internal things would be changed albeit not Api visible 3). Address arithmetic for Addr is very different from addres arithmetic on larger than word8 size storable values, is there anything which would move representations where the change to byte indexed would change the calculations ? If we can layout what the impact / scope of changes needed for these and other considerations, I guess I’d be all for it. Assuming there’s the corresponding interest in executing changes. (I just have this worry in my head that it might be one of those icebergs.. like the split base effort ) On Thu, Oct 25, 2018 at 11:33 PM David Feuer wrote:

On Thu, Oct 25, 2018, 11:28 PM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

Pretending we’re talking about this with storable as our example semantics :

Ptr Void Corresponds to a memory address you don’t want to read or write to.

This is not so easy to think about (there's not much utility in such a type). Perhaps Ptr Void is a reasonable type for pointers known to be null or outside the address space, but I'm not sure.

...

Ptr () corresponds to a memory location that’s pretty boring to read / write to

Yes! In other words, it's *not* what people should be using for a pointer to an arbitrary non-Haskell address. That's precisely what Addr is for.

...

On Thu, Oct 25, 2018 at 11:22 PM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

... when is a valid pointer not going to point at byte addressable memory on memory architectures ghc can support or target?

On Thu, Oct 25, 2018 at 8:27 PM Daniel Cartwright wrote:

...

Now, one could argue that `Ptr ()` isn't a lie, it sort of reads like C's void pointer. But surely something like `Ptr Word8` is a lie, when it is not actually a Ptr to Word8 values.

On Thu, Oct 25, 2018 at 8:11 PM Daniel Cartwright < chessai1996@gmail.com> wrote:

...

yes, only the type and its instances should be moved as far as i'm aware.

Also, it's more than just base.

this Ptr is a lie: https://hackage.haskell.org/package/ghc-compact-0.1.0.0/docs/GHC-Compact-Ser... these Ptrs are lies: https://hackage.haskell.org/package/base-4.12.0.0/docs/GHC-IO-Handle.html in GHC.Stats, the foreign import "getRTSStats" has `Ptr () -> IO ()`, this Ptr () is also a lie

These are just off the top of my head, there are more

On Thu, Oct 25, 2018 at 6:46 PM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

hrmm, what are the pieces of base that are using Ptr when they really should be using Addr? This would help me understand what would be made better in base :)

On Thu, Oct 25, 2018 at 6:19 PM David Feuer wrote:

> We shouldn't really need to move anything into base except Addr and > its base instances. > > On Oct 25, 2018 5:59 PM, "Carter Schonwald" < > carter.schonwald@gmail.com> wrote: > > Indeed. The monad transformer instances for primmonad need to live > in primmonad OR transformers to avoid orphans. > > Either way, unless transformers moves into base (unlikely), no way > anything using prim monad will. > > On Thu, Oct 25, 2018 at 3:34 PM Andrew Martin < > andrew.thaddeus@gmail.com> wrote: > >> I like the idea of moving the type Addr into base. But we cannot >> move the entire module since it has functions that talk about PrimMonad, >> and we definitely don't want to move that into base. >> >> On Thu, Oct 25, 2018 at 11:25 AM Daniel Cartwright < >> chessai1996@gmail.com> wrote: >> >>> Motivation: There are a lot of places in base where 'Ptr a' is >>> used in place of 'Addr', because in base there is no 'Addr', only 'Addr#'. >>> The problem lies in the fact that many of these uses of 'Ptr a' are lying; >>> the 'a' value is meaningless. Authors of functions therein have used things >>> like 'Ptr ()', 'Ptr Word8', 'Ptr a', but these types do not mean what they >>> say they mean - they're just Addr. There are probably other motivations for >>> this that I can't think of off the top of my head right now. >>> _______________________________________________ >>> Libraries mailing list >>> Libraries@haskell.org >>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries >>> >> >> >> -- >> -Andrew Thaddeus Martin >> _______________________________________________ >> Libraries mailing list >> Libraries@haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries >> > _______________________________________________ > Libraries mailing list > Libraries@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries > > > _______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

I would, yeah. Should the next version of `primitive` just re-export `Addr` from base in `Data.Primitive.Addr`? On Fri, Oct 26, 2018 at 10:14 AM Carter Schonwald < carter.schonwald@gmail.com> wrote:

Daniel or David, would either of you be interested/willing to do the legwork on this patch wise (subject to sanity checking whats the impact surface area?)

On Fri, Oct 26, 2018 at 12:04 AM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

Cool. What apis in base should move to use Addr from fake pointer ? Cause just adding it in isolation seems lame!

I guess I’m just trying to say “what would the impact on base, if every fake Ptr was moved to be Addr?” Because it’s not something which should be done by halves.

1) what apis would be changed?

2) what internal things would be changed albeit not Api visible

3). Address arithmetic for Addr is very different from addres arithmetic on larger than word8 size storable values, is there anything which would move representations where the change to byte indexed would change the calculations ?

If we can layout what the impact / scope of changes needed for these and other considerations, I guess I’d be all for it. Assuming there’s the corresponding interest in executing changes. (I just have this worry in my head that it might be one of those icebergs.. like the split base effort )

On Thu, Oct 25, 2018 at 11:33 PM David Feuer wrote:

...

On Thu, Oct 25, 2018, 11:28 PM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

Pretending we’re talking about this with storable as our example semantics :

Ptr Void Corresponds to a memory address you don’t want to read or write to.

This is not so easy to think about (there's not much utility in such a type). Perhaps Ptr Void is a reasonable type for pointers known to be null or outside the address space, but I'm not sure.

...

Ptr () corresponds to a memory location that’s pretty boring to read / write to

Yes! In other words, it's *not* what people should be using for a pointer to an arbitrary non-Haskell address. That's precisely what Addr is for.

...

On Thu, Oct 25, 2018 at 11:22 PM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

... when is a valid pointer not going to point at byte addressable memory on memory architectures ghc can support or target?

On Thu, Oct 25, 2018 at 8:27 PM Daniel Cartwright < chessai1996@gmail.com> wrote:

...

Now, one could argue that `Ptr ()` isn't a lie, it sort of reads like C's void pointer. But surely something like `Ptr Word8` is a lie, when it is not actually a Ptr to Word8 values.

On Thu, Oct 25, 2018 at 8:11 PM Daniel Cartwright < chessai1996@gmail.com> wrote:

> yes, only the type and its instances should be moved as far as i'm > aware. > > Also, it's more than just base. > > this Ptr is a lie: > https://hackage.haskell.org/package/ghc-compact-0.1.0.0/docs/GHC-Compact-Ser... > these Ptrs are lies: > https://hackage.haskell.org/package/base-4.12.0.0/docs/GHC-IO-Handle.html > in GHC.Stats, the foreign import "getRTSStats" has `Ptr () -> IO > ()`, this Ptr () is also a lie > > These are just off the top of my head, there are more > > > On Thu, Oct 25, 2018 at 6:46 PM Carter Schonwald < > carter.schonwald@gmail.com> wrote: > >> hrmm, what are the pieces of base that are using Ptr when they >> really should be using Addr? This would help me understand what would be >> made better in base :) >> >> On Thu, Oct 25, 2018 at 6:19 PM David Feuer >> wrote: >> >>> We shouldn't really need to move anything into base except Addr >>> and its base instances. >>> >>> On Oct 25, 2018 5:59 PM, "Carter Schonwald" < >>> carter.schonwald@gmail.com> wrote: >>> >>> Indeed. The monad transformer instances for primmonad need to >>> live in primmonad OR transformers to avoid orphans. >>> >>> Either way, unless transformers moves into base (unlikely), no way >>> anything using prim monad will. >>> >>> On Thu, Oct 25, 2018 at 3:34 PM Andrew Martin < >>> andrew.thaddeus@gmail.com> wrote: >>> >>>> I like the idea of moving the type Addr into base. But we cannot >>>> move the entire module since it has functions that talk about PrimMonad, >>>> and we definitely don't want to move that into base. >>>> >>>> On Thu, Oct 25, 2018 at 11:25 AM Daniel Cartwright < >>>> chessai1996@gmail.com> wrote: >>>> >>>>> Motivation: There are a lot of places in base where 'Ptr a' is >>>>> used in place of 'Addr', because in base there is no 'Addr', only 'Addr#'. >>>>> The problem lies in the fact that many of these uses of 'Ptr a' are lying; >>>>> the 'a' value is meaningless. Authors of functions therein have used things >>>>> like 'Ptr ()', 'Ptr Word8', 'Ptr a', but these types do not mean what they >>>>> say they mean - they're just Addr. There are probably other motivations for >>>>> this that I can't think of off the top of my head right now. >>>>> _______________________________________________ >>>>> Libraries mailing list >>>>> Libraries@haskell.org >>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries >>>>> >>>> >>>> >>>> -- >>>> -Andrew Thaddeus Martin >>>> _______________________________________________ >>>> Libraries mailing list >>>> Libraries@haskell.org >>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries >>>> >>> _______________________________________________ >>> Libraries mailing list >>> Libraries@haskell.org >>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries >>> >>> >>> _______________________________________________ >> Libraries mailing list >> Libraries@haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries >> >

Before we bike shed these details, let’s evaluate / document the pieces of base that should also be moved to using Addr. Though that module where 1) What exported types / data definitions / functions will have changes ? 2) what internal apis need to change? as a lower bound estimate of work surface area, i grepped for "Ptr ()" and "Addr#" in a fresh recursive clone of ghc at commit 503514b94f8dc7bd9eab5392206649aee45f140b, limiting my search to .hs and .hsc files THERES A LOT of uses of Ptr (), and its siblings StablePtr () and RemotePtr () , The latter two dont have a simple Addr# underneath, heres the list of files that match that search rg 'Ptr \(\)' --type haskell -c ghc/GHCi/Leak.hs:1 utils/hsc2hs/Main.hs:1 utils/runghc/Main.hs:1 compiler/ghci/GHCi.hs:4 compiler/ghci/ByteCodeLink.hs:3 compiler/ghci/ByteCodeGen.hs:4 compiler/ghci/ByteCodeTypes.hs:1 compiler/ghci/ByteCodeAsm.hs:2 compiler/main/HscTypes.hs:1 libraries/ghci/GHCi/ObjLink.hs:3 libraries/ghci/GHCi/StaticPtrTable.hs:1 libraries/ghci/GHCi/ResolvedBCO.hs:1 libraries/ghci/GHCi/RemoteTypes.hs:2 libraries/ghci/GHCi/Run.hs:2 libraries/ghci/GHCi/Message.hs:5 libraries/base/GHC/Stable.hs:2 libraries/base/GHC/StaticPtr.hs:1 libraries/base/GHC/Generics.hs:2 libraries/base/GHC/TopHandler.hs:1 libraries/base/GHC/Windows.hs:1 libraries/base/tests/dynamic002.hs:2 utils/haddock/haddock-api/src/Haddock.hs:1 libraries/array/Data/Array/IO.hs:2 libraries/base/Foreign/Marshal/Pool.hs:1 libraries/bytestring/Data/ByteString/Builder/Extra.hs:1 libraries/base/GHC/IO/Handle/Text.hs:1 testsuite/tests/rts/T7160.hs:1 testsuite/tests/rts/overflow3.hs:1 testsuite/tests/rts/overflow2.hs:1 testsuite/tests/rts/T5250.hs:1 testsuite/tests/rts/T10296b.hs:2 testsuite/tests/th/T3319.hs:1 testsuite/tests/th/TH_foreignInterruptible.hs:1 testsuite/tests/th/TH_foreignCallingConventions.hs:1 libraries/Cabal/cabal-install/Distribution/Client/Win32SelfUpgrade.hs:1 libraries/Cabal/Cabal/Distribution/Simple/InstallDirs.hs:2 testsuite/tests/llvm/should_compile/T5486.hs:2 testsuite/tests/codeGen/should_run/cgrun070.hs:5 testsuite/tests/codeGen/should_run/cgrun026.hs:3 testsuite/tests/primops/should_run/T4442.hs:1 testsuite/tests/ffi/should_fail/T7243.hs:1 testsuite/tests/ffi/should_fail/T3066.hs:1 testsuite/tests/ffi/should_compile/cc009.hs:1 testsuite/tests/ffi/should_run/ffi021.hs:1 testsuite/tests/typecheck/should_compile/T7408.hs:1 libraries/Cabal/cabal-install/Distribution/Client/Compat/ExecutablePath.hs:1 libraries/Cabal/Cabal/Distribution/Simple/Build/PathsModule.hs:1 libraries/time/lib/Data/Time/Clock/Internal/CTimeval.hs:1 testsuite/tests/rename/should_compile/T3103/Foreign/Ptr.hs:1 likewise the locations where Addr# comes up are as follows (some of these perhaps should be an Addr? ) $ rg 'Addr#' --type haskell -c utils/genprimopcode/Main.hs:2 ghc/GHCi/Leak.hs:2 compiler/cmm/CmmMachOp.hs:1 compiler/cmm/CmmUtils.hs:1 compiler/specialise/Rules.hs:2 compiler/basicTypes/MkId.hs:2 compiler/basicTypes/Literal.hs:6 compiler/basicTypes/Unique.hs:2 compiler/utils/StringBuffer.hs:1 compiler/utils/FastString.hs:4 compiler/utils/Encoding.hs:9 compiler/codeGen/StgCmmPrim.hs:3 compiler/coreSyn/MkCore.hs:5 compiler/coreSyn/CoreLint.hs:2 compiler/coreSyn/CoreSyn.hs:4 compiler/coreSyn/CorePrep.hs:1 compiler/stranal/WwLib.hs:1 compiler/simplCore/CSE.hs:1 compiler/simplCore/Simplify.hs:2 compiler/deSugar/DsExpr.hs:1 compiler/typecheck/TcGenDeriv.hs:5 compiler/prelude/ForeignCall.hs:1 compiler/prelude/PrelNames.hs:1 compiler/prelude/TysPrim.hs:1 compiler/ghci/ByteCodeGen.hs:8 libraries/bytestring/tests/revcomp.hs:1 libraries/bytestring/Data/ByteString.hs:5 libraries/ghc-heap/tests/heap_all.hs:1 libraries/ghc-prim/GHC/CString.hs:17 libraries/ghc-prim/GHC/Classes.hs:1 libraries/ghc-prim/GHC/Types.hs:3 libraries/ghci/GHCi/BinaryArray.hs:1 libraries/ghci/GHCi/CreateBCO.hs:5 libraries/base/codepages/MakeTable.hs:1 libraries/base/GHC/Ptr.hs:8 libraries/base/GHC/Generics.hs:5 libraries/base/GHC/Storable.hs:32 libraries/base/GHC/ForeignPtr.hs:6 libraries/base/Foreign/Ptr.hs:2 libraries/bytestring/Data/ByteString/Unsafe.hs:6 libraries/bytestring/Data/ByteString/Internal.hs:7 libraries/ghc-heap/GHC/Exts/Heap.hs:1 libraries/text/Data/Text/Show.hs:2 libraries/text/Data/Text/Lazy.hs:2 libraries/containers/Data/IntSet/Internal.hs:4 libraries/ghc-compact/GHC/Compact/Serialized.hs:11 libraries/base/GHC/IO/Exception.hs:1 libraries/base/Data/Typeable/Internal.hs:3 libraries/base/Control/Exception/Base.hs:1 utils/haddock/html-test/src/Ticket112.hs:1 testsuite/tests/rts/T8242.hs:19 testsuite/tests/ghci.debugger/HappyTest.hs:3 testsuite/tests/arityanal/Main.hs:3 testsuite/tests/th/TH_StringPrimL.hs:2 testsuite/tests/th/T1830_3a.hs:2 libraries/template-haskell/Language/Haskell/TH/Syntax.hs:1 libraries/bytestring/Data/ByteString/Short/Internal.hs:16 libraries/integer-gmp/src/GHC/Integer/Type.hs:11 libraries/base/GHC/IO/Encoding/CodePage.hs:3 libraries/Cabal/Cabal/Distribution/Parsec/Lexer.hs:9 testsuite/tests/generics/GEq/GEq1A.hs:1 testsuite/tests/generics/GEq/GEq1.hs:2 testsuite/tests/codeGen/should_compile/cg008.hs:1 testsuite/tests/codeGen/should_run/cgrun070.hs:4 testsuite/tests/codeGen/should_run/T9577.hs:1 testsuite/tests/codeGen/should_run/cgrun066.hs:1 testsuite/tests/codeGen/should_run/cgrun026.hs:2 testsuite/tests/codeGen/should_run/T2080.hs:2 testsuite/tests/codeGen/should_run/T9577_A.hs:1 testsuite/tests/primops/should_run/T4442.hs:4 testsuite/tests/simplCore/should_compile/T13367.hs:1 testsuite/tests/simplCore/should_compile/T13155.hs:2 testsuite/tests/simplCore/should_compile/T14779a.hs:1 testsuite/tests/simplCore/should_compile/T14779b.hs:2 testsuite/tests/simplCore/should_compile/spec001.hs:1 testsuite/tests/lib/integer/integerGmpInternals.hs:9 testsuite/tests/perf/compiler/T14683.hs:3 testsuite/tests/ghci/should_run/T2589.hs:1 libraries/integer-gmp/src/GHC/Integer/GMP/Internals.hs:7 libraries/text/Data/Text/Internal/Fusion/Common.hs:3 libraries/base/GHC/IO/Encoding/CodePage/Table.hs:1 On Fri, Oct 26, 2018 at 11:01 AM Daniel Cartwright wrote:

Also, where in base should Addr go?

On Fri, Oct 26, 2018 at 10:15 AM Daniel Cartwright wrote:

...

I would, yeah. Should the next version of `primitive` just re-export `Addr` from base in `Data.Primitive.Addr`?

On Fri, Oct 26, 2018 at 10:14 AM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

Daniel or David, would either of you be interested/willing to do the legwork on this patch wise (subject to sanity checking whats the impact surface area?)

On Fri, Oct 26, 2018 at 12:04 AM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

Cool. What apis in base should move to use Addr from fake pointer ? Cause just adding it in isolation seems lame!

I guess I’m just trying to say “what would the impact on base, if every fake Ptr was moved to be Addr?” Because it’s not something which should be done by halves.

1) what apis would be changed?

2) what internal things would be changed albeit not Api visible

3). Address arithmetic for Addr is very different from addres arithmetic on larger than word8 size storable values, is there anything which would move representations where the change to byte indexed would change the calculations ?

If we can layout what the impact / scope of changes needed for these and other considerations, I guess I’d be all for it. Assuming there’s the corresponding interest in executing changes. (I just have this worry in my head that it might be one of those icebergs.. like the split base effort )

On Thu, Oct 25, 2018 at 11:33 PM David Feuer wrote:

...

On Thu, Oct 25, 2018, 11:28 PM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

Pretending we’re talking about this with storable as our example semantics :

Ptr Void Corresponds to a memory address you don’t want to read or write to.

This is not so easy to think about (there's not much utility in such a type). Perhaps Ptr Void is a reasonable type for pointers known to be null or outside the address space, but I'm not sure.

...

Ptr () corresponds to a memory location that’s pretty boring to read / write to

Yes! In other words, it's *not* what people should be using for a pointer to an arbitrary non-Haskell address. That's precisely what Addr is for.

...

On Thu, Oct 25, 2018 at 11:22 PM Carter Schonwald < carter.schonwald@gmail.com> wrote:

> ... when is a valid pointer not going to point at byte addressable > memory on memory architectures ghc can support or target? > > On Thu, Oct 25, 2018 at 8:27 PM Daniel Cartwright < > chessai1996@gmail.com> wrote: > >> Now, one could argue that `Ptr ()` isn't a lie, it sort of reads >> like C's void pointer. But surely something like `Ptr Word8` is a lie, when >> it is not actually a Ptr to Word8 values. >> >> On Thu, Oct 25, 2018 at 8:11 PM Daniel Cartwright < >> chessai1996@gmail.com> wrote: >> >>> yes, only the type and its instances should be moved as far as i'm >>> aware. >>> >>> Also, it's more than just base. >>> >>> this Ptr is a lie: >>> https://hackage.haskell.org/package/ghc-compact-0.1.0.0/docs/GHC-Compact-Ser... >>> these Ptrs are lies: >>> https://hackage.haskell.org/package/base-4.12.0.0/docs/GHC-IO-Handle.html >>> in GHC.Stats, the foreign import "getRTSStats" has `Ptr () -> IO >>> ()`, this Ptr () is also a lie >>> >>> These are just off the top of my head, there are more >>> >>> >>> On Thu, Oct 25, 2018 at 6:46 PM Carter Schonwald < >>> carter.schonwald@gmail.com> wrote: >>> >>>> hrmm, what are the pieces of base that are using Ptr when they >>>> really should be using Addr? This would help me understand what would be >>>> made better in base :) >>>> >>>> On Thu, Oct 25, 2018 at 6:19 PM David Feuer < >>>> david.feuer@gmail.com> wrote: >>>> >>>>> We shouldn't really need to move anything into base except Addr >>>>> and its base instances. >>>>> >>>>> On Oct 25, 2018 5:59 PM, "Carter Schonwald" < >>>>> carter.schonwald@gmail.com> wrote: >>>>> >>>>> Indeed. The monad transformer instances for primmonad need to >>>>> live in primmonad OR transformers to avoid orphans. >>>>> >>>>> Either way, unless transformers moves into base (unlikely), no >>>>> way anything using prim monad will. >>>>> >>>>> On Thu, Oct 25, 2018 at 3:34 PM Andrew Martin < >>>>> andrew.thaddeus@gmail.com> wrote: >>>>> >>>>>> I like the idea of moving the type Addr into base. But we >>>>>> cannot move the entire module since it has functions that talk about >>>>>> PrimMonad, and we definitely don't want to move that into base. >>>>>> >>>>>> On Thu, Oct 25, 2018 at 11:25 AM Daniel Cartwright < >>>>>> chessai1996@gmail.com> wrote: >>>>>> >>>>>>> Motivation: There are a lot of places in base where 'Ptr a' is >>>>>>> used in place of 'Addr', because in base there is no 'Addr', only 'Addr#'. >>>>>>> The problem lies in the fact that many of these uses of 'Ptr a' are lying; >>>>>>> the 'a' value is meaningless. Authors of functions therein have used things >>>>>>> like 'Ptr ()', 'Ptr Word8', 'Ptr a', but these types do not mean what they >>>>>>> say they mean - they're just Addr. There are probably other motivations for >>>>>>> this that I can't think of off the top of my head right now. >>>>>>> _______________________________________________ >>>>>>> Libraries mailing list >>>>>>> Libraries@haskell.org >>>>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> -Andrew Thaddeus Martin >>>>>> _______________________________________________ >>>>>> Libraries mailing list >>>>>> Libraries@haskell.org >>>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries >>>>>> >>>>> _______________________________________________ >>>>> Libraries mailing list >>>>> Libraries@haskell.org >>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries >>>>> >>>>> >>>>> _______________________________________________ >>>> Libraries mailing list >>>> Libraries@haskell.org >>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries >>>> >>>

On the flip side we could just add data Addr ... to foreign .ptr or a child / sibling module like System.Mem.Address and defer the associated cleanup I kinda like the idea of System.Mem.Address, i put a strawman change set on phab https://phabricator.haskell.org/D5268 On Fri, Oct 26, 2018 at 12:23 PM Carter Schonwald < carter.schonwald@gmail.com> wrote:

granted most of the Addr# sites dont matter

a related issue is what operations WOULD be added for Addr?

maybe just memcopy friends plus address arithmetic, plus a mapper from/to Ptr a?

does it simply become a byte size step sibling to Ptr a that put in Foreign.Ptr?

On Fri, Oct 26, 2018 at 12:18 PM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

Before we bike shed these details, let’s evaluate / document the pieces of base that should also be moved to using Addr. Though that module where

1) What exported types / data definitions / functions will have changes ?

2) what internal apis need to change?

as a lower bound estimate of work surface area, i grepped for "Ptr ()" and "Addr#" in a fresh recursive clone of ghc at commit 503514b94f8dc7bd9eab5392206649aee45f140b, limiting my search to .hs and .hsc files

THERES A LOT of uses of Ptr (), and its siblings StablePtr () and RemotePtr () , The latter two dont have a simple Addr# underneath,

heres the list of files that match that search rg 'Ptr \(\)' --type haskell -c

ghc/GHCi/Leak.hs:1 utils/hsc2hs/Main.hs:1 utils/runghc/Main.hs:1 compiler/ghci/GHCi.hs:4 compiler/ghci/ByteCodeLink.hs:3 compiler/ghci/ByteCodeGen.hs:4 compiler/ghci/ByteCodeTypes.hs:1 compiler/ghci/ByteCodeAsm.hs:2 compiler/main/HscTypes.hs:1 libraries/ghci/GHCi/ObjLink.hs:3 libraries/ghci/GHCi/StaticPtrTable.hs:1 libraries/ghci/GHCi/ResolvedBCO.hs:1 libraries/ghci/GHCi/RemoteTypes.hs:2 libraries/ghci/GHCi/Run.hs:2 libraries/ghci/GHCi/Message.hs:5 libraries/base/GHC/Stable.hs:2 libraries/base/GHC/StaticPtr.hs:1 libraries/base/GHC/Generics.hs:2 libraries/base/GHC/TopHandler.hs:1 libraries/base/GHC/Windows.hs:1 libraries/base/tests/dynamic002.hs:2 utils/haddock/haddock-api/src/Haddock.hs:1 libraries/array/Data/Array/IO.hs:2 libraries/base/Foreign/Marshal/Pool.hs:1 libraries/bytestring/Data/ByteString/Builder/Extra.hs:1 libraries/base/GHC/IO/Handle/Text.hs:1 testsuite/tests/rts/T7160.hs:1 testsuite/tests/rts/overflow3.hs:1 testsuite/tests/rts/overflow2.hs:1 testsuite/tests/rts/T5250.hs:1 testsuite/tests/rts/T10296b.hs:2 testsuite/tests/th/T3319.hs:1 testsuite/tests/th/TH_foreignInterruptible.hs:1 testsuite/tests/th/TH_foreignCallingConventions.hs:1 libraries/Cabal/cabal-install/Distribution/Client/Win32SelfUpgrade.hs:1 libraries/Cabal/Cabal/Distribution/Simple/InstallDirs.hs:2 testsuite/tests/llvm/should_compile/T5486.hs:2 testsuite/tests/codeGen/should_run/cgrun070.hs:5 testsuite/tests/codeGen/should_run/cgrun026.hs:3 testsuite/tests/primops/should_run/T4442.hs:1 testsuite/tests/ffi/should_fail/T7243.hs:1 testsuite/tests/ffi/should_fail/T3066.hs:1 testsuite/tests/ffi/should_compile/cc009.hs:1 testsuite/tests/ffi/should_run/ffi021.hs:1 testsuite/tests/typecheck/should_compile/T7408.hs:1

libraries/Cabal/cabal-install/Distribution/Client/Compat/ExecutablePath.hs:1 libraries/Cabal/Cabal/Distribution/Simple/Build/PathsModule.hs:1 libraries/time/lib/Data/Time/Clock/Internal/CTimeval.hs:1 testsuite/tests/rename/should_compile/T3103/Foreign/Ptr.hs:1

likewise the locations where Addr# comes up are as follows (some of these perhaps should be an Addr? )

$ rg 'Addr#' --type haskell -c utils/genprimopcode/Main.hs:2 ghc/GHCi/Leak.hs:2 compiler/cmm/CmmMachOp.hs:1 compiler/cmm/CmmUtils.hs:1 compiler/specialise/Rules.hs:2 compiler/basicTypes/MkId.hs:2 compiler/basicTypes/Literal.hs:6 compiler/basicTypes/Unique.hs:2 compiler/utils/StringBuffer.hs:1 compiler/utils/FastString.hs:4 compiler/utils/Encoding.hs:9 compiler/codeGen/StgCmmPrim.hs:3 compiler/coreSyn/MkCore.hs:5 compiler/coreSyn/CoreLint.hs:2 compiler/coreSyn/CoreSyn.hs:4 compiler/coreSyn/CorePrep.hs:1 compiler/stranal/WwLib.hs:1 compiler/simplCore/CSE.hs:1 compiler/simplCore/Simplify.hs:2 compiler/deSugar/DsExpr.hs:1 compiler/typecheck/TcGenDeriv.hs:5 compiler/prelude/ForeignCall.hs:1 compiler/prelude/PrelNames.hs:1 compiler/prelude/TysPrim.hs:1 compiler/ghci/ByteCodeGen.hs:8 libraries/bytestring/tests/revcomp.hs:1 libraries/bytestring/Data/ByteString.hs:5 libraries/ghc-heap/tests/heap_all.hs:1 libraries/ghc-prim/GHC/CString.hs:17 libraries/ghc-prim/GHC/Classes.hs:1 libraries/ghc-prim/GHC/Types.hs:3 libraries/ghci/GHCi/BinaryArray.hs:1 libraries/ghci/GHCi/CreateBCO.hs:5 libraries/base/codepages/MakeTable.hs:1 libraries/base/GHC/Ptr.hs:8 libraries/base/GHC/Generics.hs:5 libraries/base/GHC/Storable.hs:32 libraries/base/GHC/ForeignPtr.hs:6 libraries/base/Foreign/Ptr.hs:2 libraries/bytestring/Data/ByteString/Unsafe.hs:6 libraries/bytestring/Data/ByteString/Internal.hs:7 libraries/ghc-heap/GHC/Exts/Heap.hs:1 libraries/text/Data/Text/Show.hs:2 libraries/text/Data/Text/Lazy.hs:2 libraries/containers/Data/IntSet/Internal.hs:4 libraries/ghc-compact/GHC/Compact/Serialized.hs:11 libraries/base/GHC/IO/Exception.hs:1 libraries/base/Data/Typeable/Internal.hs:3 libraries/base/Control/Exception/Base.hs:1 utils/haddock/html-test/src/Ticket112.hs:1 testsuite/tests/rts/T8242.hs:19 testsuite/tests/ghci.debugger/HappyTest.hs:3 testsuite/tests/arityanal/Main.hs:3 testsuite/tests/th/TH_StringPrimL.hs:2 testsuite/tests/th/T1830_3a.hs:2 libraries/template-haskell/Language/Haskell/TH/Syntax.hs:1 libraries/bytestring/Data/ByteString/Short/Internal.hs:16 libraries/integer-gmp/src/GHC/Integer/Type.hs:11 libraries/base/GHC/IO/Encoding/CodePage.hs:3 libraries/Cabal/Cabal/Distribution/Parsec/Lexer.hs:9 testsuite/tests/generics/GEq/GEq1A.hs:1 testsuite/tests/generics/GEq/GEq1.hs:2 testsuite/tests/codeGen/should_compile/cg008.hs:1 testsuite/tests/codeGen/should_run/cgrun070.hs:4 testsuite/tests/codeGen/should_run/T9577.hs:1 testsuite/tests/codeGen/should_run/cgrun066.hs:1 testsuite/tests/codeGen/should_run/cgrun026.hs:2 testsuite/tests/codeGen/should_run/T2080.hs:2 testsuite/tests/codeGen/should_run/T9577_A.hs:1 testsuite/tests/primops/should_run/T4442.hs:4 testsuite/tests/simplCore/should_compile/T13367.hs:1 testsuite/tests/simplCore/should_compile/T13155.hs:2 testsuite/tests/simplCore/should_compile/T14779a.hs:1 testsuite/tests/simplCore/should_compile/T14779b.hs:2 testsuite/tests/simplCore/should_compile/spec001.hs:1 testsuite/tests/lib/integer/integerGmpInternals.hs:9 testsuite/tests/perf/compiler/T14683.hs:3 testsuite/tests/ghci/should_run/T2589.hs:1 libraries/integer-gmp/src/GHC/Integer/GMP/Internals.hs:7 libraries/text/Data/Text/Internal/Fusion/Common.hs:3 libraries/base/GHC/IO/Encoding/CodePage/Table.hs:1

On Fri, Oct 26, 2018 at 11:01 AM Daniel Cartwright wrote:

...

Also, where in base should Addr go?

On Fri, Oct 26, 2018 at 10:15 AM Daniel Cartwright < chessai1996@gmail.com> wrote:

...

I would, yeah. Should the next version of `primitive` just re-export `Addr` from base in `Data.Primitive.Addr`?

On Fri, Oct 26, 2018 at 10:14 AM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

Daniel or David, would either of you be interested/willing to do the legwork on this patch wise (subject to sanity checking whats the impact surface area?)

On Fri, Oct 26, 2018 at 12:04 AM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

Cool. What apis in base should move to use Addr from fake pointer ? Cause just adding it in isolation seems lame!

I guess I’m just trying to say “what would the impact on base, if every fake Ptr was moved to be Addr?” Because it’s not something which should be done by halves.

1) what apis would be changed?

2) what internal things would be changed albeit not Api visible

3). Address arithmetic for Addr is very different from addres arithmetic on larger than word8 size storable values, is there anything which would move representations where the change to byte indexed would change the calculations ?

If we can layout what the impact / scope of changes needed for these and other considerations, I guess I’d be all for it. Assuming there’s the corresponding interest in executing changes. (I just have this worry in my head that it might be one of those icebergs.. like the split base effort )

On Thu, Oct 25, 2018 at 11:33 PM David Feuer wrote:

> On Thu, Oct 25, 2018, 11:28 PM Carter Schonwald < > carter.schonwald@gmail.com> wrote: > >> Pretending we’re talking about this with storable as our example >> semantics : >> >> Ptr Void >> Corresponds to a memory address you don’t want to read or write to. >> > > This is not so easy to think about (there's not much utility in such > a type). Perhaps Ptr Void is a reasonable type for pointers known to be > null or outside the address space, but I'm not sure. > > >> Ptr () corresponds to a memory location that’s pretty boring to >> read / write to >> > > Yes! In other words, it's *not* what people should be using for a > pointer to an arbitrary non-Haskell address. That's precisely what Addr is > for. > > >> On Thu, Oct 25, 2018 at 11:22 PM Carter Schonwald < >> carter.schonwald@gmail.com> wrote: >> >>> ... when is a valid pointer not going to point at byte addressable >>> memory on memory architectures ghc can support or target? >>> >>> On Thu, Oct 25, 2018 at 8:27 PM Daniel Cartwright < >>> chessai1996@gmail.com> wrote: >>> >>>> Now, one could argue that `Ptr ()` isn't a lie, it sort of reads >>>> like C's void pointer. But surely something like `Ptr Word8` is a lie, when >>>> it is not actually a Ptr to Word8 values. >>>> >>>> On Thu, Oct 25, 2018 at 8:11 PM Daniel Cartwright < >>>> chessai1996@gmail.com> wrote: >>>> >>>>> yes, only the type and its instances should be moved as far as >>>>> i'm aware. >>>>> >>>>> Also, it's more than just base. >>>>> >>>>> this Ptr is a lie: >>>>> https://hackage.haskell.org/package/ghc-compact-0.1.0.0/docs/GHC-Compact-Ser... >>>>> these Ptrs are lies: >>>>> https://hackage.haskell.org/package/base-4.12.0.0/docs/GHC-IO-Handle.html >>>>> in GHC.Stats, the foreign import "getRTSStats" has `Ptr () -> IO >>>>> ()`, this Ptr () is also a lie >>>>> >>>>> These are just off the top of my head, there are more >>>>> >>>>> >>>>> On Thu, Oct 25, 2018 at 6:46 PM Carter Schonwald < >>>>> carter.schonwald@gmail.com> wrote: >>>>> >>>>>> hrmm, what are the pieces of base that are using Ptr when they >>>>>> really should be using Addr? This would help me understand what would be >>>>>> made better in base :) >>>>>> >>>>>> On Thu, Oct 25, 2018 at 6:19 PM David Feuer < >>>>>> david.feuer@gmail.com> wrote: >>>>>> >>>>>>> We shouldn't really need to move anything into base except >>>>>>> Addr and its base instances. >>>>>>> >>>>>>> On Oct 25, 2018 5:59 PM, "Carter Schonwald" < >>>>>>> carter.schonwald@gmail.com> wrote: >>>>>>> >>>>>>> Indeed. The monad transformer instances for primmonad need to >>>>>>> live in primmonad OR transformers to avoid orphans. >>>>>>> >>>>>>> Either way, unless transformers moves into base (unlikely), no >>>>>>> way anything using prim monad will. >>>>>>> >>>>>>> On Thu, Oct 25, 2018 at 3:34 PM Andrew Martin < >>>>>>> andrew.thaddeus@gmail.com> wrote: >>>>>>> >>>>>>>> I like the idea of moving the type Addr into base. But we >>>>>>>> cannot move the entire module since it has functions that talk about >>>>>>>> PrimMonad, and we definitely don't want to move that into base. >>>>>>>> >>>>>>>> On Thu, Oct 25, 2018 at 11:25 AM Daniel Cartwright < >>>>>>>> chessai1996@gmail.com> wrote: >>>>>>>> >>>>>>>>> Motivation: There are a lot of places in base where 'Ptr a' >>>>>>>>> is used in place of 'Addr', because in base there is no 'Addr', only >>>>>>>>> 'Addr#'. The problem lies in the fact that many of these uses of 'Ptr a' >>>>>>>>> are lying; the 'a' value is meaningless. Authors of functions therein have >>>>>>>>> used things like 'Ptr ()', 'Ptr Word8', 'Ptr a', but these types do not >>>>>>>>> mean what they say they mean - they're just Addr. There are probably other >>>>>>>>> motivations for this that I can't think of off the top of my head right now. >>>>>>>>> _______________________________________________ >>>>>>>>> Libraries mailing list >>>>>>>>> Libraries@haskell.org >>>>>>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> -Andrew Thaddeus Martin >>>>>>>> _______________________________________________ >>>>>>>> Libraries mailing list >>>>>>>> Libraries@haskell.org >>>>>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries >>>>>>>> >>>>>>> _______________________________________________ >>>>>>> Libraries mailing list >>>>>>> Libraries@haskell.org >>>>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>> Libraries mailing list >>>>>> Libraries@haskell.org >>>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries >>>>>> >>>>>

Am Fr., 26. Okt. 2018 um 06:05 Uhr schrieb Carter Schonwald < carter.schonwald@gmail.com>:

[...] I guess I’m just trying to say “what would the impact on base, if every fake Ptr was moved to be Addr?” Because it’s not something which should be done by halves.

1) [...]

2) [...]

3). [...]

The most important question is missing from this list: What are the benefits of touching such a crucial part of our API ecosystem? There should better be extremely good reasons, otherwise lots of annoying work and incompatibility is generated for nothing. Reading through this thread, I haven't seen a good motivation for touching Storable/Ptr and friends. Perhaps I have misunderstood what the actual proposal is, initially I thought it is just replacing various "Ptr foo" with Addr only within GHC itself. That's fine, but the scope seems to have broadened. Just a historical remark: Addr# is a GHCism, and so was Addr. At the time the FFI came up, Ptr was intended to be the portable type. So yes, "Ptr a" and "Ptr ()" are basically just Addr (modulo casts), but intentionally so.

there was never any hint at helping primitive, i'm not sure where you got that from. also, the primary motivation was to avoid using 'Ptr a', where 'a' is a lie. not only that, but base seems a natural home for 'Addr', and people can avoid incurring a dep on primitive when they just need 'Addr'. 'super useful' was never the goal - only correctness and convenience. Yeah, i agree that Storable should not be touched. On Fri, Oct 26, 2018 at 1:36 PM Carter Schonwald wrote:

your perspective here is a good one

1) i absolutely agree, nothing should touch storable, whatesoever,

2) the baseline proposal is to add an Addr to Base (essentially Ptr ()), which would be a crippled sibling of how its exposed in Data.Primitive I guess?

3) as a litmus for "what would this acomplish/support", i was asking "how would this get used/help base be nicer "? If

anyways: i did a strawman of what Addr ripped out of Data.Primitive.Addr and into base would look like, and it doesn't look especially compelling / nicer than Ptr shenanigans. Especially since to be useful in base it would have to have a bunch of IO / ST specific operations that have the option perhaps of using Storable to read /write at locations. At which point I still need to have the same API surface area again in Primitive,

see https://phabricator.haskell.org/D5268 for the bare bones stuff (doesn't type check)

the api i could expose there in base does not reduce the implementation surface area needed for Primitve...

i'm open to being convinced otherwise, but with Sven's perspective weighed in, 1) i dont see it being super useful within base/ghc apis, as they exist today 2) it doesn't reduce implementation surface area burden in the Primitive package.

so at this point im' weakly against the addition. more work, no clear reward for me :)

On Fri, Oct 26, 2018 at 1:26 PM Sven Panne wrote:

...

Am Fr., 26. Okt. 2018 um 06:05 Uhr schrieb Carter Schonwald < carter.schonwald@gmail.com>:

...

[...] I guess I’m just trying to say “what would the impact on base, if every fake Ptr was moved to be Addr?” Because it’s not something which should be done by halves.

1) [...]

2) [...]

3). [...]

The most important question is missing from this list: What are the benefits of touching such a crucial part of our API ecosystem? There should better be extremely good reasons, otherwise lots of annoying work and incompatibility is generated for nothing. Reading through this thread, I haven't seen a good motivation for touching Storable/Ptr and friends. Perhaps I have misunderstood what the actual proposal is, initially I thought it is just replacing various "Ptr foo" with Addr only within GHC itself. That's fine, but the scope seems to have broadened.

Just a historical remark: Addr# is a GHCism, and so was Addr. At the time the FFI came up, Ptr was intended to be the portable type. So yes, "Ptr a" and "Ptr ()" are basically just Addr (modulo casts), but intentionally so.

_______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

anyways, if you want to do a cleaned up sibling of https://phabricator.haskell.org/D5268, i vote yes On Fri, Oct 26, 2018 at 1:55 PM Carter Schonwald wrote:

What api? It’s not obvious, could you spell out what you have in mind. Relative to the phab straw man?

On Fri, Oct 26, 2018 at 1:42 PM Daniel Cartwright wrote:

...

there was never any hint at helping primitive, i'm not sure where you got that from. also, the primary motivation was to avoid using 'Ptr a', where 'a' is a lie. not only that, but base seems a natural home for 'Addr', and people can avoid incurring a dep on primitive when they just need 'Addr'.

'super useful' was never the goal - only correctness and convenience.

Yeah, i agree that Storable should not be touched.

On Fri, Oct 26, 2018 at 1:36 PM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

your perspective here is a good one

1) i absolutely agree, nothing should touch storable, whatesoever,

2) the baseline proposal is to add an Addr to Base (essentially Ptr ()), which would be a crippled sibling of how its exposed in Data.Primitive I guess?

3) as a litmus for "what would this acomplish/support", i was asking "how would this get used/help base be nicer "? If

anyways: i did a strawman of what Addr ripped out of Data.Primitive.Addr and into base would look like, and it doesn't look especially compelling / nicer than Ptr shenanigans. Especially since to be useful in base it would have to have a bunch of IO / ST specific operations that have the option perhaps of using Storable to read /write at locations. At which point I still need to have the same API surface area again in Primitive,

see https://phabricator.haskell.org/D5268 for the bare bones stuff (doesn't type check)

the api i could expose there in base does not reduce the implementation surface area needed for Primitve...

i'm open to being convinced otherwise, but with Sven's perspective weighed in, 1) i dont see it being super useful within base/ghc apis, as they exist today 2) it doesn't reduce implementation surface area burden in the Primitive package.

so at this point im' weakly against the addition. more work, no clear reward for me :)

On Fri, Oct 26, 2018 at 1:26 PM Sven Panne wrote:

...

Am Fr., 26. Okt. 2018 um 06:05 Uhr schrieb Carter Schonwald < carter.schonwald@gmail.com>:

...

[...] I guess I’m just trying to say “what would the impact on base, if every fake Ptr was moved to be Addr?” Because it’s not something which should be done by halves.

1) [...]

2) [...]

3). [...]

The most important question is missing from this list: What are the benefits of touching such a crucial part of our API ecosystem? There should better be extremely good reasons, otherwise lots of annoying work and incompatibility is generated for nothing. Reading through this thread, I haven't seen a good motivation for touching Storable/Ptr and friends. Perhaps I have misunderstood what the actual proposal is, initially I thought it is just replacing various "Ptr foo" with Addr only within GHC itself. That's fine, but the scope seems to have broadened.

Just a historical remark: Addr# is a GHCism, and so was Addr. At the time the FFI came up, Ptr was intended to be the portable type. So yes, "Ptr a" and "Ptr ()" are basically just Addr (modulo casts), but intentionally so.

_______________________________________________

...

Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

Am Fr., 26. Okt. 2018 um 20:00 Uhr schrieb Daniel Cartwright < chessai1996@gmail.com>:

[...] Eventually any non-user-facing functions that use 'Ptr LyingType' should be switched from using 'Ptr' to 'Addr'. [...]

Again my question is: Why? When someone sees a "Ptr a" or "Ptr ()", it's blindingly obvious that this is equivalent to Addr, i.e. you need some additional information what that thing is pointing to or you don't care about what's at that location. This is not "lying", quite the opposite. You can turn the whole proposal 180 degrees around: Let's nuke "Addr" in favor of "Ptr a"/"Ptr ()"! :-) That would even get rid of a whole type, something the current proposal doesn't achieve. Addr is strictly less powerful than Ptr, so why keep the former?

Am Fr., 26. Okt. 2018 um 22:16 Uhr schrieb Daniel Cartwright < chessai1996@gmail.com>:

I am referring to the situations when someone uses 'Ptr a', but the Ptr does not point to anything of type 'a'. If I write 'Ptr Word8', but I am pointing to a Char, then that is not true.

It might not be true, but you might be forced to do such things to use some ugly C library. The Ptr type carries some valuable information, namely: What does my C counterpart expect? It might not be 100% true on the Haskell side, but you are in "unsafe land", anyway, and at some level you *have* to be able to use some white lie when necessary. Note that I'm not saying that this is nice, but I haven't heard of a better alternative yet.

Am Fr., 26. Okt. 2018 um 22:31 Uhr schrieb Andrew Martin < andrew.thaddeus@gmail.com>:

The better alternative suggested in this thread is to use Addr instead. That way, you don't have to lie about the type of the serialized data that the pointer is pointing to.

Even if I'm repeating myself: That is not lying, quite the opposite. If you e.g. use a C library with suboptimal typing (i.e. more often than not), you make the impedance matching explicit. How would you achieve that with Addr?

Daniel Cartwright wrote:

There is no operational difference when using `Ptr a` vs `Addr` in such a case. The problem lies in the type variable `a`. Types are here to guide us - if I use `Ptr Word8`, and this Ptr is not a Ptr to Word8, there is not much enforcing this. In such cases, the type annotation is untruthful.

This is an argument against using a fully polymorphic `Ptr a` value as an abstract pointer, but I don't see anything wrong with using `Ptr ()` values or fully polymorphic `Ptr a` function arguments for that purpose. With any such low-level code, you have to trust the programmer to get things right anyway; replacing `Ptr ()` values by `Addr` values isn't going to make a big difference.

We would like it more often than not to be truthful. Yes, in the underlying C, there exists suboptimal typing, but that is why we are not writing in C, and instead lifting the code to Haskell.

We should also discuss the cost associated with mixing two functionally equivalent types in the same library, which will make the API less predictable, are likely to result in duplication in the APIs for Ptr and Addr, and will lead to more explicit type conversions with little benefit. And of course there is a risk of breaking existing code as existing API is changed to use the new type. (The term "user-facing" has popped up in this thread -- what does that mean in the case of such a low-level API?) Cheers, Bertram

Addr only exists within the primitive package. And it only seems to be used in one spot in all of the vector package. And it’s a very weird spot that I honestly may wanna redesign/see about changing given how strange it is. Especially since it may be one of those things where the current code needs some benchmarks https://github.com/haskell/vector/blob/master/Data/Vector/Storable/Mutable.h... It’s the only instance in all of vector On Sat, Oct 27, 2018 at 6:05 AM Sven Panne wrote:

Am Sa., 27. Okt. 2018 um 11:07 Uhr schrieb Bertram Felgenhauer via Libraries :

...

[...] We should also discuss the cost associated with mixing two functionally equivalent types in the same library, which will make the API less predictable, are likely to result in duplication in the APIs for Ptr and Addr, and will lead to more explicit type conversions with little benefit. [...]

The more I think about it, the more serious I am about my "180 degree" proposal: Nuke Addr completely in favor of "Ptr a"/"Ptr ()". What can Addr do what a "Ptr a"/"Ptr ()" can't? Off the top of my head I would say "nothing". In most other circumstances we like generalizations, why not here? Having Addr just gives us a bigger API surface and is probably used mostly within GHC and its closely tied packages, not really "in the wild".

If "Ptr a" or "Ptr ()" or "Ptr Void" is the right thing to use in a specific place is always a tradeoff between ease of use, being explicit, readability, and perhaps what a corresponding C library uses. Type safety, as much as we like it, is just wishful thinking at that level, there's always castPtr (and of course the FunPtr counterparts). _______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

'Ptr Void' is not a pointer to value of type 'Void'; there are no values of type 'Void'; this type is nonsensical. 'Ptr a' is not a pointer to a value of type a; what is 'a'? 'Ptr ()' could be a pointer to a value of type '()', and if so, then this is a correct usage of 'Ptr'. If not, it is an incorrect usage of 'Ptr'. On Sat, Oct 27, 2018 at 2:43 PM Carter Schonwald wrote:

*only use i'm aware of

and its a pretty gross one.

are there any ACTUAL uses of Addr? (granted, having a lifted / data'd version of unlifted types is general a useful thing)

Ptr Void having the byte address oriented interface for indexing arithmetic sounds good to me

On Sat, Oct 27, 2018 at 9:56 AM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

Addr only exists within the primitive package. And it only seems to be used in one spot in all of the vector package. And it’s a very weird spot that I honestly may wanna redesign/see about changing given how strange it is. Especially since it may be one of those things where the current code needs some benchmarks

https://github.com/haskell/vector/blob/master/Data/Vector/Storable/Mutable.h...

It’s the only instance in all of vector

On Sat, Oct 27, 2018 at 6:05 AM Sven Panne wrote:

...

Am Sa., 27. Okt. 2018 um 11:07 Uhr schrieb Bertram Felgenhauer via Libraries :

...

[...] We should also discuss the cost associated with mixing two functionally equivalent types in the same library, which will make the API less predictable, are likely to result in duplication in the APIs for Ptr and Addr, and will lead to more explicit type conversions with little benefit. [...]

The more I think about it, the more serious I am about my "180 degree" proposal: Nuke Addr completely in favor of "Ptr a"/"Ptr ()". What can Addr do what a "Ptr a"/"Ptr ()" can't? Off the top of my head I would say "nothing". In most other circumstances we like generalizations, why not here? Having Addr just gives us a bigger API surface and is probably used mostly within GHC and its closely tied packages, not really "in the wild".

If "Ptr a" or "Ptr ()" or "Ptr Void" is the right thing to use in a specific place is always a tradeoff between ease of use, being explicit, readability, and perhaps what a corresponding C library uses. Type safety, as much as we like it, is just wishful thinking at that level, there's always castPtr (and of course the FunPtr counterparts). _______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

Am Mo., 29. Okt. 2018 um 14:27 Uhr schrieb Daniel Cartwright < chessai1996@gmail.com>:

'Ptr Void' is not a pointer to a value of type 'Void'; there are no values of type 'Void': this type is nonsensical.

That's the whole point, and it actually makes sense: If you see "Ptr Void", you can't do much with it, apart from passing it around or using castPtr on it. This is exactly what should be achieved by using "Ptr Void" in an API. This is basically the same as "void *" in C/C++.

'Ptr a' is not a pointer to a value of type a; what is 'a'?

As usual, it's the "I don't care about it" type. :) Take e.g. mallocBytes: It returns a "Ptr a" exactly because of the fact that it doesn't care about what will eventually be stored in the new area. Another API design option would have been returning "Ptr Void" or "Ptr ()" instead, but this would have forced API users to insert castPtr, which is only annoying there and wouldn't buy us a single bit of security in those circumstances.

'Ptr ()' could be a pointer to a value of type '()', and if so, then this is a correct usage of 'Ptr'. If not, it is an incorrect usage of 'Ptr'.

You can't store or read "()", so the same holds as for Void (which didn't exist when the FFI was created IIRC).

The point , hahah, of a Ptr void is that you can’t dereference it. But you certainly can cast it and do address arithmetic on it!! On Mon, Oct 29, 2018 at 10:10 AM David Feuer wrote:

On Mon, Oct 29, 2018, 10:05 AM Sven Panne wrote:

...

Am Mo., 29. Okt. 2018 um 14:27 Uhr schrieb Daniel Cartwright < chessai1996@gmail.com>:

...

'Ptr Void' is not a pointer to a value of type 'Void'; there are no values of type 'Void': this type is nonsensical.

That's the whole point, and it actually makes sense: If you see "Ptr Void", you can't do much with it, apart from passing it around or using castPtr on it. This is exactly what should be achieved by using "Ptr Void" in an API. This is basically the same as "void *" in C/C++.

No, it does not make sense. The approximate equivalent of C's void* is Ptr Any. Ptr Void promises to give you anything you want on dereference, which is nonsense.

...

You can't store or read "()", so the same holds as for Void (which didn't exist when the FFI was created IIRC).

Sure you can. Storing () does nothing and reading it gives (). Our () is somewhat similar to C's void return type. _______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

absolutely false, represeentational equality of the type a in `Ptr a` does not mean the memory representation at the corresponding address is the same. (it sometimes is true, but memory packing/alignment details in structs in C for otherwise equivlanet structs should rule this out) aka, `a` being representationally equal to `b` via haskell newtypes does not mean the memory representation at `Ptr a`, and `Ptr b` are the same. a trivial example is when host and network byte order aren't the same (eg big vs little endian memory encodings) On Mon, Oct 29, 2018 at 12:28 PM David Feuer wrote:

What? Of course you can dereference it. You dereference it, getting a value of type `Void`, and apply absurd to get whatever you want in the world. This, of course, is utter nonsense, unless *having* the Ptr Void means that something has already gone wrong. It's pretty hard for me to imagine a situation where this is actually what you want. A Ptr () isn't nonsense. It is not terrible to use Ptr () to represent an Addr, but I wonder if it sends the wrong message. By the way: there's another argument for having Addr in base for now. We would really *like* for Ptr's parameter to have a *representational* role, but we *don't* want to require unsafeCoerce to cast Ptrs. The solution to that in the current role system:

data Addr = Addr Addr#

newtype Ptr a = Ptr_ Addr type role Ptr representational

pattern Ptr :: Addr# -> Ptr a pattern Ptr addr# = Ptr_ (Addr addr#)

-- Allow users to reveal coercibility of pointer types locally ptrCoercion :: Coercion (Ptr a) (Ptr b) ptrCoercion = Coercion

castPtr :: Ptr a -> Ptr b castPtr = coerceWith ptrCoercion -- (or the now-free unwrap-rewrap definition)

So even if we don't *expose* Addr in base, we should almost certainly *define* it there. On Mon, Oct 29, 2018 at 12:11 PM Carter Schonwald wrote:

...

The point , hahah, of a Ptr void is that you can’t dereference it. But

you certainly can cast it and do address arithmetic on it!!

...

On Mon, Oct 29, 2018 at 10:10 AM David Feuer

wrote:

...

...

On Mon, Oct 29, 2018, 10:05 AM Sven Panne wrote:

...

Am Mo., 29. Okt. 2018 um 14:27 Uhr schrieb Daniel Cartwright <

chessai1996@gmail.com>:

...

...

...

'Ptr Void' is not a pointer to a value of type 'Void'; there are no

values of type 'Void': this type is nonsensical.

That's the whole point, and it actually makes sense: If you see "Ptr Void", you can't do much with it, apart from passing it around or using castPtr on it. This is exactly what should be achieved by using "Ptr Void" in an API. This is basically the same as "void *" in C/C++.

No, it does not make sense. The approximate equivalent of C's void* is Ptr Any. Ptr Void promises to give you anything you want on dereference, which is nonsense.

...

You can't store or read "()", so the same holds as for Void (which

didn't exist when the FFI was created IIRC).

Sure you can. Storing () does nothing and reading it gives (). Our () is somewhat similar to C's void return type. _______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

to zoom out: what code is improved? what code is made better/clearer? No one has articulated this clearly. The one example of Addr being used in Vector.Storable.Mutable is not an argument in favor of using Addr. Its an argument against it existing. i'm looking for evidence, in the form of code i can look at then say "yes, this is better code" when comparing the two. Or a mathematical statement of "what is made better" @David Feuer , @Daniel , do you have one? when i'm writing complicated code, MORE polymorphism helps me usually. I can write some code like the following and even though I'm using it with Int at argument, I *Know* that i'm not mixing up arguments/values that i write as different types. I cannot do this with Address! (the type / function below can be found at https://github.com/wellposed/numerical/blob/3a0bbf50bc6ce0b710aee755f5a4bfce... ) {-# SPECIALIZE INLINE computeStarts :: [(Int,Int)]->Int->Int ->[(Int,Int)] #-} computeStarts:: (Enum a, Ord a, Num b )=>[(a,b)]-> a -> a -> [(a,b)] parametricity (even when constrained by type classes) is a powerful and foundational tool for good programming in haskell and similar languages there has been nothing stated here that successfully articulates a good reason to forgo/discourage parametricity as an engineering tool. for thats what Addr is. A datatype thats never safe in isolation, and discourages using parametricity to write correct software. a very strong case is needed to forgo parametricity. On Mon, Oct 29, 2018 at 5:33 PM David Feuer wrote:

Good point! Call it nominal then.

On Mon, Oct 29, 2018, 5:24 PM Carter Schonwald wrote:

...

absolutely false, represeentational equality of the type a in `Ptr a` does not mean the memory representation at the corresponding address is the same. (it sometimes is true, but memory packing/alignment details in structs in C for otherwise equivlanet structs should rule this out)

aka, `a` being representationally equal to `b` via haskell newtypes does not mean the memory representation at `Ptr a`, and `Ptr b` are the same. a trivial example is when host and network byte order aren't the same (eg big vs little endian memory encodings)

On Mon, Oct 29, 2018 at 12:28 PM David Feuer wrote:

...

What? Of course you can dereference it. You dereference it, getting a value of type `Void`, and apply absurd to get whatever you want in the world. This, of course, is utter nonsense, unless *having* the Ptr Void means that something has already gone wrong. It's pretty hard for me to imagine a situation where this is actually what you want. A Ptr () isn't nonsense. It is not terrible to use Ptr () to represent an Addr, but I wonder if it sends the wrong message. By the way: there's another argument for having Addr in base for now. We would really *like* for Ptr's parameter to have a *representational* role, but we *don't* want to require unsafeCoerce to cast Ptrs. The solution to that in the current role system:

data Addr = Addr Addr#

newtype Ptr a = Ptr_ Addr type role Ptr representational

pattern Ptr :: Addr# -> Ptr a pattern Ptr addr# = Ptr_ (Addr addr#)

-- Allow users to reveal coercibility of pointer types locally ptrCoercion :: Coercion (Ptr a) (Ptr b) ptrCoercion = Coercion

castPtr :: Ptr a -> Ptr b castPtr = coerceWith ptrCoercion -- (or the now-free unwrap-rewrap definition)

So even if we don't *expose* Addr in base, we should almost certainly *define* it there. On Mon, Oct 29, 2018 at 12:11 PM Carter Schonwald wrote:

...

The point , hahah, of a Ptr void is that you can’t dereference it.

But you certainly can cast it and do address arithmetic on it!!

...

On Mon, Oct 29, 2018 at 10:10 AM David Feuer

wrote:

...

...

On Mon, Oct 29, 2018, 10:05 AM Sven Panne

wrote:

...

...

Am Mo., 29. Okt. 2018 um 14:27 Uhr schrieb Daniel Cartwright <

chessai1996@gmail.com>:

...

> > 'Ptr Void' is not a pointer to a value of type 'Void'; there are no values of type 'Void': this type is nonsensical.

That's the whole point, and it actually makes sense: If you see "Ptr Void", you can't do much with it, apart from passing it around or using castPtr on it. This is exactly what should be achieved by using "Ptr Void" in an API. This is basically the same as "void *" in C/C++.

No, it does not make sense. The approximate equivalent of C's void* is Ptr Any. Ptr Void promises to give you anything you want on dereference, which is nonsense.

...

You can't store or read "()", so the same holds as for Void (which

didn't exist when the FFI was created IIRC).

Sure you can. Storing () does nothing and reading it gives (). Our () is somewhat similar to C's void return type. _______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

the parametricity isn't for when you know things, its for saying "these are possibly different or possibly the same, dont let me mix them up though, cause I dont know yet" On Mon, Oct 29, 2018 at 7:03 PM Daniel Cartwright wrote:

I'm not sure that argument applies at all, when talking about _incorrect_ usages of Ptr. Sure, Addr probably shouldn't be used when there is meaningful type information/value to recover, but neither should Ptr be used when there is none.

The argument being made is not to make 'better', per se, and there definitely won't be a 'mathematical statement' about this, but it certainly may be made clearer - in my opinion, the usages of 'Ptr' that i've already brought up are inherently unclear because of the bogus phantom type associated with 'Ptr'. The illustration of this begs no code that doesn't already exist in corelibs.

On Mon, Oct 29, 2018 at 6:19 PM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

to zoom out: what code is improved? what code is made better/clearer? No one has articulated this clearly.

The one example of Addr being used in Vector.Storable.Mutable is not an argument in favor of using Addr. Its an argument against it existing.

i'm looking for evidence, in the form of code i can look at then say "yes, this is better code" when comparing the two. Or a mathematical statement of "what is made better"

@David Feuer , @Daniel , do you have one?

when i'm writing complicated code, MORE polymorphism helps me usually.

I can write some code like the following and even though I'm using it with Int at argument, I *Know* that i'm not mixing up arguments/values that i write as different types. I cannot do this with Address! (the type / function below can be found at https://github.com/wellposed/numerical/blob/3a0bbf50bc6ce0b710aee755f5a4bfce... )

{-# SPECIALIZE INLINE computeStarts :: [(Int,Int)]->Int->Int ->[(Int,Int)] #-} computeStarts:: (Enum a, Ord a, Num b )=>[(a,b)]-> a -> a -> [(a,b)]

parametricity (even when constrained by type classes) is a powerful and foundational tool for good programming in haskell and similar languages

there has been nothing stated here that successfully articulates a good reason to forgo/discourage parametricity as an engineering tool. for thats what Addr is. A datatype thats never safe in isolation, and discourages using parametricity to write correct software.

a very strong case is needed to forgo parametricity.

On Mon, Oct 29, 2018 at 5:33 PM David Feuer wrote:

...

Good point! Call it nominal then.

On Mon, Oct 29, 2018, 5:24 PM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

absolutely false, represeentational equality of the type a in `Ptr a` does not mean the memory representation at the corresponding address is the same. (it sometimes is true, but memory packing/alignment details in structs in C for otherwise equivlanet structs should rule this out)

aka, `a` being representationally equal to `b` via haskell newtypes does not mean the memory representation at `Ptr a`, and `Ptr b` are the same. a trivial example is when host and network byte order aren't the same (eg big vs little endian memory encodings)

On Mon, Oct 29, 2018 at 12:28 PM David Feuer wrote:

...

What? Of course you can dereference it. You dereference it, getting a value of type `Void`, and apply absurd to get whatever you want in the world. This, of course, is utter nonsense, unless *having* the Ptr Void means that something has already gone wrong. It's pretty hard for me to imagine a situation where this is actually what you want. A Ptr () isn't nonsense. It is not terrible to use Ptr () to represent an Addr, but I wonder if it sends the wrong message. By the way: there's another argument for having Addr in base for now. We would really *like* for Ptr's parameter to have a *representational* role, but we *don't* want to require unsafeCoerce to cast Ptrs. The solution to that in the current role system:

data Addr = Addr Addr#

newtype Ptr a = Ptr_ Addr type role Ptr representational

pattern Ptr :: Addr# -> Ptr a pattern Ptr addr# = Ptr_ (Addr addr#)

-- Allow users to reveal coercibility of pointer types locally ptrCoercion :: Coercion (Ptr a) (Ptr b) ptrCoercion = Coercion

castPtr :: Ptr a -> Ptr b castPtr = coerceWith ptrCoercion -- (or the now-free unwrap-rewrap definition)

So even if we don't *expose* Addr in base, we should almost certainly *define* it there. On Mon, Oct 29, 2018 at 12:11 PM Carter Schonwald wrote:

...

The point , hahah, of a Ptr void is that you can’t dereference it.

But you certainly can cast it and do address arithmetic on it!!

...

On Mon, Oct 29, 2018 at 10:10 AM David Feuer

wrote:

...

> > On Mon, Oct 29, 2018, 10:05 AM Sven Panne wrote: >> >> Am Mo., 29. Okt. 2018 um 14:27 Uhr schrieb Daniel Cartwright < chessai1996@gmail.com>: >>> >>> 'Ptr Void' is not a pointer to a value of type 'Void'; there are no values of type 'Void': this type is nonsensical. >> >> >> That's the whole point, and it actually makes sense: If you see "Ptr Void", you can't do much with it, apart from passing it around or using castPtr on it. This is exactly what should be achieved by using "Ptr Void" in an API. This is basically the same as "void *" in C/C++. > > > No, it does not make sense. The approximate equivalent of C's void* is Ptr Any. Ptr Void promises to give you anything you want on dereference, which is nonsense. > >> >> You can't store or read "()", so the same holds as for Void (which didn't exist when the FFI was created IIRC). > > > Sure you can. Storing () does nothing and reading it gives (). Our () is somewhat similar to C's void return type. > _______________________________________________ > Libraries mailing list > Libraries@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

@danny... storable is only for fixed size pinned memory values On Mon, Oct 29, 2018 at 8:31 PM Dannyu NDos wrote:

It is Storable that makes Ptr operate as typed pointers. I also think Ptr on its own doesn't have that much about its argument type, so I'm +1 on the proposal.

Btw, shouldn't every type be storable? In current Haskell, Maybes or (->)s aren't Storable, yet in C++, arrays of std::optionals or std::functions are well-defined.

As an exception, for Void, I agree that they must remain not Storable since it has no values.

2018년 10월 30일 (화) 08:31, Carter Schonwald 님이 작성:

...

the parametricity isn't for when you know things, its for saying "these are possibly different or possibly the same, dont let me mix them up though, cause I dont know yet"

On Mon, Oct 29, 2018 at 7:03 PM Daniel Cartwright wrote:

...

I'm not sure that argument applies at all, when talking about _incorrect_ usages of Ptr. Sure, Addr probably shouldn't be used when there is meaningful type information/value to recover, but neither should Ptr be used when there is none.

The argument being made is not to make 'better', per se, and there definitely won't be a 'mathematical statement' about this, but it certainly may be made clearer - in my opinion, the usages of 'Ptr' that i've already brought up are inherently unclear because of the bogus phantom type associated with 'Ptr'. The illustration of this begs no code that doesn't already exist in corelibs.

On Mon, Oct 29, 2018 at 6:19 PM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

to zoom out: what code is improved? what code is made better/clearer? No one has articulated this clearly.

The one example of Addr being used in Vector.Storable.Mutable is not an argument in favor of using Addr. Its an argument against it existing.

i'm looking for evidence, in the form of code i can look at then say "yes, this is better code" when comparing the two. Or a mathematical statement of "what is made better"

@David Feuer , @Daniel , do you have one?

when i'm writing complicated code, MORE polymorphism helps me usually.

I can write some code like the following and even though I'm using it with Int at argument, I *Know* that i'm not mixing up arguments/values that i write as different types. I cannot do this with Address! (the type / function below can be found at https://github.com/wellposed/numerical/blob/3a0bbf50bc6ce0b710aee755f5a4bfce... )

{-# SPECIALIZE INLINE computeStarts :: [(Int,Int)]->Int->Int ->[(Int,Int)] #-} computeStarts:: (Enum a, Ord a, Num b )=>[(a,b)]-> a -> a -> [(a,b)]

parametricity (even when constrained by type classes) is a powerful and foundational tool for good programming in haskell and similar languages

there has been nothing stated here that successfully articulates a good reason to forgo/discourage parametricity as an engineering tool. for thats what Addr is. A datatype thats never safe in isolation, and discourages using parametricity to write correct software.

a very strong case is needed to forgo parametricity.

On Mon, Oct 29, 2018 at 5:33 PM David Feuer wrote:

...

Good point! Call it nominal then.

On Mon, Oct 29, 2018, 5:24 PM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

absolutely false, represeentational equality of the type a in `Ptr a` does not mean the memory representation at the corresponding address is the same. (it sometimes is true, but memory packing/alignment details in structs in C for otherwise equivlanet structs should rule this out)

aka, `a` being representationally equal to `b` via haskell newtypes does not mean the memory representation at `Ptr a`, and `Ptr b` are the same. a trivial example is when host and network byte order aren't the same (eg big vs little endian memory encodings)

On Mon, Oct 29, 2018 at 12:28 PM David Feuer wrote:

> What? Of course you can dereference it. You dereference it, getting a > value of type `Void`, > and apply absurd to get whatever you want in the world. This, of > course, is utter nonsense, > unless *having* the Ptr Void means that something has already gone > wrong. It's pretty > hard for me to imagine a situation where this is actually what you > want. A Ptr () isn't nonsense. > It is not terrible to use Ptr () to represent an Addr, but I wonder > if > it sends the wrong message. > By the way: there's another argument for having Addr in base for now. > We would really > *like* for Ptr's parameter to have a *representational* role, but we > *don't* want to require > unsafeCoerce to cast Ptrs. The solution to that in the current role > system: > > data Addr = Addr Addr# > > newtype Ptr a = Ptr_ Addr > type role Ptr representational > > pattern Ptr :: Addr# -> Ptr a > pattern Ptr addr# = Ptr_ (Addr addr#) > > -- Allow users to reveal coercibility of pointer types locally > ptrCoercion :: Coercion (Ptr a) (Ptr b) > ptrCoercion = Coercion > > castPtr :: Ptr a -> Ptr b > castPtr = coerceWith ptrCoercion -- (or the now-free > unwrap-rewrap > definition) > > > So even if we don't *expose* Addr in base, we should almost > certainly *define* > it there. > On Mon, Oct 29, 2018 at 12:11 PM Carter Schonwald > wrote: > > > > The point , hahah, of a Ptr void is that you can’t dereference > it. But you certainly can cast it and do address arithmetic on it!! > > > > > > > > On Mon, Oct 29, 2018 at 10:10 AM David Feuer < > david.feuer@gmail.com> wrote: > >> > >> On Mon, Oct 29, 2018, 10:05 AM Sven Panne > wrote: > >>> > >>> Am Mo., 29. Okt. 2018 um 14:27 Uhr schrieb Daniel Cartwright < > chessai1996@gmail.com>: > >>>> > >>>> 'Ptr Void' is not a pointer to a value of type 'Void'; there > are no values of type 'Void': this type is nonsensical. > >>> > >>> > >>> That's the whole point, and it actually makes sense: If you see > "Ptr Void", you can't do much with it, apart from passing it around or > using castPtr on it. This is exactly what should be achieved by using "Ptr > Void" in an API. This is basically the same as "void *" in C/C++. > >> > >> > >> No, it does not make sense. The approximate equivalent of C's > void* is Ptr Any. Ptr Void promises to give you anything you want on > dereference, which is nonsense. > >> > >>> > >>> You can't store or read "()", so the same holds as for Void > (which didn't exist when the FFI was created IIRC). > >> > >> > >> Sure you can. Storing () does nothing and reading it gives (). > Our () is somewhat similar to C's void return type. > >> _______________________________________________ > >> Libraries mailing list > >> Libraries@haskell.org > >> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries > _______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

On Mon, Oct 29, 2018 at 8:31 PM Dannyu NDos wrote:

Btw, shouldn't every type be storable? In current Haskell, Maybes or (->)s aren't Storable, yet in C++, arrays of std::optionals or std::functions are well-defined.

Consider [a]. There is no fixed size representation of an entire list of elements. There is no supplementary memory management of auxillary stuff being managed by Storable instances. std::function holds onto its own data off to the side for the contexts of each functor. We have StablePtrs which give you a way to marshal any Haskell data type out to C, including functions if you actually need it. However, it pins the haskell object down on the heap, because we have no insight into what happens to it once it goes out into C. Maybe is just one extreme where you _could_ make up some convention and tagging scheme and store it in-place, same with Either. However, Storable makes no judgment about how to compose storables even for products, because there are many reasonable schemes for packing (or not) the data structures involved, and no real hope of ensuring you comply with whatever the host compiler's ABI might be. -Edward As an exception, for Void, I agree that they must remain not Storable since

it has no values.

2018년 10월 30일 (화) 08:31, Carter Schonwald 님이 작성:

...

the parametricity isn't for when you know things, its for saying "these are possibly different or possibly the same, dont let me mix them up though, cause I dont know yet"

On Mon, Oct 29, 2018 at 7:03 PM Daniel Cartwright wrote:

...

I'm not sure that argument applies at all, when talking about _incorrect_ usages of Ptr. Sure, Addr probably shouldn't be used when there is meaningful type information/value to recover, but neither should Ptr be used when there is none.

The argument being made is not to make 'better', per se, and there definitely won't be a 'mathematical statement' about this, but it certainly may be made clearer - in my opinion, the usages of 'Ptr' that i've already brought up are inherently unclear because of the bogus phantom type associated with 'Ptr'. The illustration of this begs no code that doesn't already exist in corelibs.

On Mon, Oct 29, 2018 at 6:19 PM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

to zoom out: what code is improved? what code is made better/clearer? No one has articulated this clearly.

The one example of Addr being used in Vector.Storable.Mutable is not an argument in favor of using Addr. Its an argument against it existing.

i'm looking for evidence, in the form of code i can look at then say "yes, this is better code" when comparing the two. Or a mathematical statement of "what is made better"

@David Feuer , @Daniel , do you have one?

when i'm writing complicated code, MORE polymorphism helps me usually.

I can write some code like the following and even though I'm using it with Int at argument, I *Know* that i'm not mixing up arguments/values that i write as different types. I cannot do this with Address! (the type / function below can be found at https://github.com/wellposed/numerical/blob/3a0bbf50bc6ce0b710aee755f5a4bfce... )

{-# SPECIALIZE INLINE computeStarts :: [(Int,Int)]->Int->Int ->[(Int,Int)] #-} computeStarts:: (Enum a, Ord a, Num b )=>[(a,b)]-> a -> a -> [(a,b)]

parametricity (even when constrained by type classes) is a powerful and foundational tool for good programming in haskell and similar languages

there has been nothing stated here that successfully articulates a good reason to forgo/discourage parametricity as an engineering tool. for thats what Addr is. A datatype thats never safe in isolation, and discourages using parametricity to write correct software.

a very strong case is needed to forgo parametricity.

On Mon, Oct 29, 2018 at 5:33 PM David Feuer wrote:

...

Good point! Call it nominal then.

On Mon, Oct 29, 2018, 5:24 PM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

absolutely false, represeentational equality of the type a in `Ptr a` does not mean the memory representation at the corresponding address is the same. (it sometimes is true, but memory packing/alignment details in structs in C for otherwise equivlanet structs should rule this out)

aka, `a` being representationally equal to `b` via haskell newtypes does not mean the memory representation at `Ptr a`, and `Ptr b` are the same. a trivial example is when host and network byte order aren't the same (eg big vs little endian memory encodings)

On Mon, Oct 29, 2018 at 12:28 PM David Feuer wrote:

> What? Of course you can dereference it. You dereference it, getting a > value of type `Void`, > and apply absurd to get whatever you want in the world. This, of > course, is utter nonsense, > unless *having* the Ptr Void means that something has already gone > wrong. It's pretty > hard for me to imagine a situation where this is actually what you > want. A Ptr () isn't nonsense. > It is not terrible to use Ptr () to represent an Addr, but I wonder > if > it sends the wrong message. > By the way: there's another argument for having Addr in base for now. > We would really > *like* for Ptr's parameter to have a *representational* role, but we > *don't* want to require > unsafeCoerce to cast Ptrs. The solution to that in the current role > system: > > data Addr = Addr Addr# > > newtype Ptr a = Ptr_ Addr > type role Ptr representational > > pattern Ptr :: Addr# -> Ptr a > pattern Ptr addr# = Ptr_ (Addr addr#) > > -- Allow users to reveal coercibility of pointer types locally > ptrCoercion :: Coercion (Ptr a) (Ptr b) > ptrCoercion = Coercion > > castPtr :: Ptr a -> Ptr b > castPtr = coerceWith ptrCoercion -- (or the now-free > unwrap-rewrap > definition) > > > So even if we don't *expose* Addr in base, we should almost > certainly *define* > it there. > On Mon, Oct 29, 2018 at 12:11 PM Carter Schonwald > wrote: > > > > The point , hahah, of a Ptr void is that you can’t dereference > it. But you certainly can cast it and do address arithmetic on it!! > > > > > > > > On Mon, Oct 29, 2018 at 10:10 AM David Feuer < > david.feuer@gmail.com> wrote: > >> > >> On Mon, Oct 29, 2018, 10:05 AM Sven Panne > wrote: > >>> > >>> Am Mo., 29. Okt. 2018 um 14:27 Uhr schrieb Daniel Cartwright < > chessai1996@gmail.com>: > >>>> > >>>> 'Ptr Void' is not a pointer to a value of type 'Void'; there > are no values of type 'Void': this type is nonsensical. > >>> > >>> > >>> That's the whole point, and it actually makes sense: If you see > "Ptr Void", you can't do much with it, apart from passing it around or > using castPtr on it. This is exactly what should be achieved by using "Ptr > Void" in an API. This is basically the same as "void *" in C/C++. > >> > >> > >> No, it does not make sense. The approximate equivalent of C's > void* is Ptr Any. Ptr Void promises to give you anything you want on > dereference, which is nonsense. > >> > >>> > >>> You can't store or read "()", so the same holds as for Void > (which didn't exist when the FFI was created IIRC). > >> > >> > >> Sure you can. Storing () does nothing and reading it gives (). > Our () is somewhat similar to C's void return type. > >> _______________________________________________ > >> Libraries mailing list > >> Libraries@haskell.org > >> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries > _______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

I am not sure if everybody fully comprehends what Storable is all about: It is meant as the lowest-level building block in an Addr-free world (remember: Addr is a GHCism and is *not* mentioned anywhere in the report) to put a few well-defined simple Haskell types into memory or read them from there. Its explicit non-goals are: * Achieve 100% type safety. In the presence of raw memory access, castPtr, C calls etc. this would be a total illusion. Forcing API users to sprinkle tons of castPtr at every possible place over their code wouldn't improve safety at all, it would only hurt readability. * Handle more complicated sum/product types. How would you do this? Respect your native ABI (i.e. automatically handle padding/alignment)? Tightly packed? Or even handle a foreign ABI? Your own ABI? Some funny encoding like OpenGL's packed data types? Etc. etc. You can build all of those things in a layer above Storable, probably introducing other type classes or some marshaling DSLs. * Portability of the written values. This is more in the realm of serialization libraries. More concretely: Am Di., 30. Okt. 2018 um 14:34 Uhr schrieb Daniel Cartwright < chessai1996@gmail.com>:

[19:26:50] hPutBuf :: Handle -> Ptr a -> Int -> IO () [...]

The signature for this is actually perfect: hPutBuf doesn't care about what stuff has been written into the given buffer, it just cares about its start and its size. Forcing castPtr Kung Fu here wouldn't buy you anything: The buffer will probably contain a wild mix of Haskell values or even no Haskell values at all, but that doesn't matter. Whatever you pass as "a" or whatever you cast from/to is probably a lie from the typing perspective. At this level this is no problem at all.

[19:30:02] peekByteOff :: Ptr b -> Int -> IO a [19:30:09] peekByteOff :: Addr -> Int -> IO a [19:30:26] what is 'b' doing there? it's not used in any meaningful way by peekByteOff [...]

If you have a pointer pointing to something and shift that pointer by some bytes, you are probably pointing to something completely different, so of course "b" and "a" have nothing to do with each other. So peekByteOff intentionally ignores "b".

[19:32:22] <carter> pokeElemOff :: Ptr a -> Int -> a -> IO () --- way better than peak [...]

Yes, because this is intended to be used for *arrays* of values of the same type. Note "Elem" vs. "Byte".

[19:33:12] <carter> hvr: lets add safePeekByteOff :: Ptr a -> Int -> IO a ? [...]

This signature doesn't make sense, see above: Shifting a pointer by an arbitrary amount of bytes will probably change the type of what you're pointing to. If you shift by units of the underlying type, well, that's peekElemOff.

[19:35:31] carter: i am glad we agree on the smell

I don't have the full chat log, but I think I don't even agree on the smell, at least not at the places I've seen... :-)

Also, just to be clear, no one is talking about changing Storable, that seems terrible at this point. Your argument still makes sense to make outside of the context of wanting to change Storable, i just want to make sure it's reiterated that this is not being suggested. On Tue, Oct 30, 2018 at 10:18 AM Daniel Cartwright wrote:

...

hPutBuf doesn't care about what stuff has been written into the given buffer, it just cares about its start and its size. Seems like a good use for Addr?

...

If you have a pointer pointing to something and shift that pointer by some bytes, you are probably pointing to something completely different, so of course "b" and "a" have nothing to do with each other. So peekByteOff intentionally ignores "b". This also seems like a good use for Addr?

On Tue, Oct 30, 2018 at 10:10 AM Sven Panne wrote:

...

I am not sure if everybody fully comprehends what Storable is all about: It is meant as the lowest-level building block in an Addr-free world (remember: Addr is a GHCism and is *not* mentioned anywhere in the report) to put a few well-defined simple Haskell types into memory or read them from there. Its explicit non-goals are:

* Achieve 100% type safety. In the presence of raw memory access, castPtr, C calls etc. this would be a total illusion. Forcing API users to sprinkle tons of castPtr at every possible place over their code wouldn't improve safety at all, it would only hurt readability.

* Handle more complicated sum/product types. How would you do this? Respect your native ABI (i.e. automatically handle padding/alignment)? Tightly packed? Or even handle a foreign ABI? Your own ABI? Some funny encoding like OpenGL's packed data types? Etc. etc. You can build all of those things in a layer above Storable, probably introducing other type classes or some marshaling DSLs.

* Portability of the written values. This is more in the realm of serialization libraries.

More concretely:

Am Di., 30. Okt. 2018 um 14:34 Uhr schrieb Daniel Cartwright < chessai1996@gmail.com>:

...

[19:26:50] hPutBuf :: Handle -> Ptr a -> Int -> IO () [...]

The signature for this is actually perfect: hPutBuf doesn't care about what stuff has been written into the given buffer, it just cares about its start and its size. Forcing castPtr Kung Fu here wouldn't buy you anything: The buffer will probably contain a wild mix of Haskell values or even no Haskell values at all, but that doesn't matter. Whatever you pass as "a" or whatever you cast from/to is probably a lie from the typing perspective. At this level this is no problem at all.

...

[19:30:02] peekByteOff :: Ptr b -> Int -> IO a [19:30:09] peekByteOff :: Addr -> Int -> IO a [19:30:26] what is 'b' doing there? it's not used in any meaningful way by peekByteOff [...]

If you have a pointer pointing to something and shift that pointer by some bytes, you are probably pointing to something completely different, so of course "b" and "a" have nothing to do with each other. So peekByteOff intentionally ignores "b".

...

[19:32:22] <carter> pokeElemOff :: Ptr a -> Int -> a -> IO () --- way better than peak [...]

Yes, because this is intended to be used for *arrays* of values of the same type. Note "Elem" vs. "Byte".

...

[19:33:12] <carter> hvr: lets add safePeekByteOff :: Ptr a -> Int -> IO a ? [...]

This signature doesn't make sense, see above: Shifting a pointer by an arbitrary amount of bytes will probably change the type of what you're pointing to. If you shift by units of the underlying type, well, that's peekElemOff.

...

[19:35:31] carter: i am glad we agree on the smell

I don't have the full chat log, but I think I don't even agree on the smell, at least not at the places I've seen... :-)

Daniel : at this point you’re not show casing examples of why Address leads to better code than Pointer a A good library proposal makes code *better* in an unambiguous way. As I mentioned yesterday , that’s been lacking here. Or any improvements are ones sven , myself and others don’t see. :) On Tue, Oct 30, 2018 at 10:41 AM Sven Panne wrote:

Am Di., 30. Okt. 2018 um 15:18 Uhr schrieb Daniel Cartwright < chessai1996@gmail.com>:

...

...

hPutBuf doesn't care about what stuff has been written into the given buffer, it just cares about its start and its size. Seems like a good use for Addr?

Nope, not at all: Using a free type variable like "a" in such a situation is *the* common idiom to specify that the function doesn't care about the type, just like "length" doesn't care about the type of the elements. If you don't like that idiom, fine, but it has been officially enshrined in the standard and people are using it for decades (well, almost) without any problems, at least I haven't heard of them.

What would using Addr buy us? Addr is non-standard, and you would have to use some cast from a Ptr somehow (the world outside GHC's innards is using Ptr, not Addr). This would buy you exactly zero safety or clarity, and it would only introduce API friction.

...

...

If you have a pointer pointing to something and shift that pointer by some bytes, you are probably pointing to something completely different, so of course "b" and "a" have nothing to do with each other. So peekByteOff intentionally ignores "b". This also seems like a good use for Addr?

Nope again, see above.

I still fail to see what problem this whole proposal is actually trying to solve...

This example seems more a discussion about how the ghc-compact lib could benefit from better docs. On Tue, Oct 30, 2018 at 11:48 AM Andrew Martin wrote:

Look at the definition of SerializedCompact (in ghc-compact):

data SerializedCompact a = SerializedCompact { serializedCompactBlockList :: [(Ptr a, Word)] , serializedCompactRoot :: Ptr a }

The type variables on those Ptrs are a lie. The SerializedCompact needs that phantom type variable to keep track of what type of object in a compact region is represented by these blocks. But, the types of its two fields are extremely misleading. When I first saw this type, I was led to believe that I could call `peek` on the `serializedCompactRoot` field and get a value of type `a`. You cannot actually do that since it isn't actually a pointer to a value of that type. More correct would be:

data SerializedCompact a = SerializedCompact { serializedCompactBlockList :: [(Addr, Word)] , serializedCompactRoot :: Addr }

This better documents what's actually going on here. We don't have a bunch of pointers to `a` values in a list. We value memory addresses that refer to arbitrary fragments of an object on the heap.

On Tue, Oct 30, 2018 at 11:32 AM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

Daniel : at this point you’re not show casing examples of why Address leads to better code than Pointer a

A good library proposal makes code *better* in an unambiguous way. As I mentioned yesterday , that’s been lacking here. Or any improvements are ones sven , myself and others don’t see. :)

On Tue, Oct 30, 2018 at 10:41 AM Sven Panne wrote:

...

Am Di., 30. Okt. 2018 um 15:18 Uhr schrieb Daniel Cartwright < chessai1996@gmail.com>:

...

...

hPutBuf doesn't care about what stuff has been written into the given buffer, it just cares about its start and its size. Seems like a good use for Addr?

Nope, not at all: Using a free type variable like "a" in such a situation is *the* common idiom to specify that the function doesn't care about the type, just like "length" doesn't care about the type of the elements. If you don't like that idiom, fine, but it has been officially enshrined in the standard and people are using it for decades (well, almost) without any problems, at least I haven't heard of them.

What would using Addr buy us? Addr is non-standard, and you would have to use some cast from a Ptr somehow (the world outside GHC's innards is using Ptr, not Addr). This would buy you exactly zero safety or clarity, and it would only introduce API friction.

...

...

If you have a pointer pointing to something and shift that pointer by some bytes, you are probably pointing to something completely different, so of course "b" and "a" have nothing to do with each other. So peekByteOff intentionally ignores "b". This also seems like a good use for Addr?

Nope again, see above.

I still fail to see what problem this whole proposal is actually trying to solve...

_______________________________________________

...

Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

-- -Andrew Thaddeus Martin

Anyways. This proposal is dead. Changing this internal stuff from Ptr to Addr doesn’t make an argument that the code is better for it. That has not be made. Saying ptr a is a lie is about as informative as saying virtual memory is a lie. Or the cpu caches are a lie. Evidence works. Saying something is a lie doesn’t. On Tue, Oct 30, 2018 at 12:11 PM Carter Schonwald < carter.schonwald@gmail.com> wrote:

There’s no class constraint here, so I would instead assume I need to read the paper / docs and source code for how heap objects are encoded here. Changing the type to Addr vs Ptr a or Ptr void is a red herring / irrelevant

On Tue, Oct 30, 2018 at 12:09 PM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

This example seems more a discussion about how the ghc-compact lib could benefit from better docs.

On Tue, Oct 30, 2018 at 11:48 AM Andrew Martin wrote:

...

Look at the definition of SerializedCompact (in ghc-compact):

data SerializedCompact a = SerializedCompact { serializedCompactBlockList :: [(Ptr a, Word)] , serializedCompactRoot :: Ptr a }

The type variables on those Ptrs are a lie. The SerializedCompact needs that phantom type variable to keep track of what type of object in a compact region is represented by these blocks. But, the types of its two fields are extremely misleading. When I first saw this type, I was led to believe that I could call `peek` on the `serializedCompactRoot` field and get a value of type `a`. You cannot actually do that since it isn't actually a pointer to a value of that type. More correct would be:

data SerializedCompact a = SerializedCompact { serializedCompactBlockList :: [(Addr, Word)] , serializedCompactRoot :: Addr }

This better documents what's actually going on here. We don't have a bunch of pointers to `a` values in a list. We value memory addresses that refer to arbitrary fragments of an object on the heap.

On Tue, Oct 30, 2018 at 11:32 AM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

Daniel : at this point you’re not show casing examples of why Address leads to better code than Pointer a

A good library proposal makes code *better* in an unambiguous way. As I mentioned yesterday , that’s been lacking here. Or any improvements are ones sven , myself and others don’t see. :)

On Tue, Oct 30, 2018 at 10:41 AM Sven Panne wrote:

...

Am Di., 30. Okt. 2018 um 15:18 Uhr schrieb Daniel Cartwright < chessai1996@gmail.com>:

...

> hPutBuf doesn't care about what stuff has been written into the given buffer, it just cares about its start and its size. Seems like a good use for Addr?

Nope, not at all: Using a free type variable like "a" in such a situation is *the* common idiom to specify that the function doesn't care about the type, just like "length" doesn't care about the type of the elements. If you don't like that idiom, fine, but it has been officially enshrined in the standard and people are using it for decades (well, almost) without any problems, at least I haven't heard of them.

What would using Addr buy us? Addr is non-standard, and you would have to use some cast from a Ptr somehow (the world outside GHC's innards is using Ptr, not Addr). This would buy you exactly zero safety or clarity, and it would only introduce API friction.

...

> If you have a pointer pointing to something and shift that pointer by some bytes, you are probably pointing to something completely different, so of course "b" and "a" have nothing to do with each other. So peekByteOff intentionally ignores "b". This also seems like a good use for Addr?

Nope again, see above.

I still fail to see what problem this whole proposal is actually trying to solve...

_______________________________________________

...

Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

-- -Andrew Thaddeus Martin

Repeating it’s a lie isn’t an argument. Writing code and comparing it is a way to make a case. I even put some up on phabticator. Then decided it’s actually not a useful piece of code. I’ve spoke with a number of folks privately and I’m arriculating the lack of suport and clear case. I’m open to being wrong and that this proposal is great. But changing a type signature isn’t show casing what code is improved. It’s just a proof that the change can be made. I need evidence. As does everyone :) On Tue, Oct 30, 2018 at 12:25 PM David Feuer wrote:

Thankfully, Carter, you're not the sole arbiter of whether a library proposal is dead. The discussion can surely continue without you.

On Tue, Oct 30, 2018, 12:22 PM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

Anyways. This proposal is dead.

Changing this internal stuff from Ptr to Addr doesn’t make an argument that the code is better for it. That has not be made.

Saying ptr a is a lie is about as informative as saying virtual memory is a lie. Or the cpu caches are a lie.

Evidence works. Saying something is a lie doesn’t.

On Tue, Oct 30, 2018 at 12:11 PM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

There’s no class constraint here, so I would instead assume I need to read the paper / docs and source code for how heap objects are encoded here. Changing the type to Addr vs Ptr a or Ptr void is a red herring / irrelevant

On Tue, Oct 30, 2018 at 12:09 PM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

This example seems more a discussion about how the ghc-compact lib could benefit from better docs.

On Tue, Oct 30, 2018 at 11:48 AM Andrew Martin < andrew.thaddeus@gmail.com> wrote:

...

Look at the definition of SerializedCompact (in ghc-compact):

data SerializedCompact a = SerializedCompact { serializedCompactBlockList :: [(Ptr a, Word)] , serializedCompactRoot :: Ptr a }

The type variables on those Ptrs are a lie. The SerializedCompact needs that phantom type variable to keep track of what type of object in a compact region is represented by these blocks. But, the types of its two fields are extremely misleading. When I first saw this type, I was led to believe that I could call `peek` on the `serializedCompactRoot` field and get a value of type `a`. You cannot actually do that since it isn't actually a pointer to a value of that type. More correct would be:

data SerializedCompact a = SerializedCompact { serializedCompactBlockList :: [(Addr, Word)] , serializedCompactRoot :: Addr }

This better documents what's actually going on here. We don't have a bunch of pointers to `a` values in a list. We value memory addresses that refer to arbitrary fragments of an object on the heap.

On Tue, Oct 30, 2018 at 11:32 AM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

Daniel : at this point you’re not show casing examples of why Address leads to better code than Pointer a

A good library proposal makes code *better* in an unambiguous way. As I mentioned yesterday , that’s been lacking here. Or any improvements are ones sven , myself and others don’t see. :)

On Tue, Oct 30, 2018 at 10:41 AM Sven Panne wrote:

> Am Di., 30. Okt. 2018 um 15:18 Uhr schrieb Daniel Cartwright < > chessai1996@gmail.com>: > >> > hPutBuf doesn't care about what stuff has been written into the >> given buffer, it just cares about its start and its size. >> Seems like a good use for Addr? >> > > Nope, not at all: Using a free type variable like "a" in such a > situation is *the* common idiom to specify that the function doesn't care > about the type, just like "length" doesn't care about the type of the > elements. If you don't like that idiom, fine, but it has been officially > enshrined in the standard and people are using it for decades (well, > almost) without any problems, at least I haven't heard of them. > > What would using Addr buy us? Addr is non-standard, and you would > have to use some cast from a Ptr somehow (the world outside GHC's innards > is using Ptr, not Addr). This would buy you exactly zero safety or clarity, > and it would only introduce API friction. > > >> > If you have a pointer pointing to something and shift that >> pointer by some bytes, you are probably pointing to something completely >> different, so of course "b" and "a" have nothing to do with each other. So >> peekByteOff intentionally ignores "b". >> This also seems like a good use for Addr? >> > Nope again, see above. > > I still fail to see what problem this whole proposal is actually > trying to solve... > _______________________________________________

...

Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

-- -Andrew Thaddeus Martin

_______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

David: Please don’t make this personal. Either way you’re definitely correct that this is a discussion list to inform clc decisions and or applicable library maintainers. I’m speaking as a maintainer.of vector and primitive On Tue, Oct 30, 2018 at 12:38 PM David Feuer wrote:

Private off-list discussions are not, were never, and will never be a part of the proposal process, and for good reasons. Mentioning them only makes you look like you're trying to use your connections to bypass the process. Please don't.

On Tue, Oct 30, 2018, 12:36 PM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

Repeating it’s a lie isn’t an argument.

Writing code and comparing it is a way to make a case. I even put some up on phabticator. Then decided it’s actually not a useful piece of code.

I’ve spoke with a number of folks privately and I’m arriculating the lack of suport and clear case.

I’m open to being wrong and that this proposal is great. But changing a type signature isn’t show casing what code is improved. It’s just a proof that the change can be made.

I need evidence. As does everyone :)

On Tue, Oct 30, 2018 at 12:25 PM David Feuer wrote:

...

Thankfully, Carter, you're not the sole arbiter of whether a library proposal is dead. The discussion can surely continue without you.

On Tue, Oct 30, 2018, 12:22 PM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

Anyways. This proposal is dead.

Changing this internal stuff from Ptr to Addr doesn’t make an argument that the code is better for it. That has not be made.

Saying ptr a is a lie is about as informative as saying virtual memory is a lie. Or the cpu caches are a lie.

Evidence works. Saying something is a lie doesn’t.

On Tue, Oct 30, 2018 at 12:11 PM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

There’s no class constraint here, so I would instead assume I need to read the paper / docs and source code for how heap objects are encoded here. Changing the type to Addr vs Ptr a or Ptr void is a red herring / irrelevant

On Tue, Oct 30, 2018 at 12:09 PM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

This example seems more a discussion about how the ghc-compact lib could benefit from better docs.

On Tue, Oct 30, 2018 at 11:48 AM Andrew Martin < andrew.thaddeus@gmail.com> wrote:

> Look at the definition of SerializedCompact (in ghc-compact): > > data SerializedCompact a = SerializedCompact > { serializedCompactBlockList :: [(Ptr a, Word)] > , serializedCompactRoot :: Ptr a > } > > The type variables on those Ptrs are a lie. The SerializedCompact > needs that phantom type variable to keep track of what type of object in a > compact region is represented by these blocks. But, the types of its two > fields are extremely misleading. When I first saw this type, I was led to > believe that I could call `peek` on the `serializedCompactRoot` field and > get a value of type `a`. You cannot actually do that since it isn't > actually a pointer to a value of that type. More correct would be: > > data SerializedCompact a = SerializedCompact > { serializedCompactBlockList :: [(Addr, Word)] > , serializedCompactRoot :: Addr > } > > This better documents what's actually going on here. We don't have a > bunch of pointers to `a` values in a list. We value memory addresses that > refer to arbitrary fragments of an object on the heap. > > > On Tue, Oct 30, 2018 at 11:32 AM Carter Schonwald < > carter.schonwald@gmail.com> wrote: > >> Daniel : at this point you’re not show casing examples of why >> Address leads to better code than Pointer a >> >> A good library proposal makes code *better* in an unambiguous way. >> As I mentioned yesterday , that’s been lacking here. Or any improvements >> are ones sven , myself and others don’t see. :) >> >> On Tue, Oct 30, 2018 at 10:41 AM Sven Panne >> wrote: >> >>> Am Di., 30. Okt. 2018 um 15:18 Uhr schrieb Daniel Cartwright < >>> chessai1996@gmail.com>: >>> >>>> > hPutBuf doesn't care about what stuff has been written into the >>>> given buffer, it just cares about its start and its size. >>>> Seems like a good use for Addr? >>>> >>> >>> Nope, not at all: Using a free type variable like "a" in such a >>> situation is *the* common idiom to specify that the function doesn't care >>> about the type, just like "length" doesn't care about the type of the >>> elements. If you don't like that idiom, fine, but it has been officially >>> enshrined in the standard and people are using it for decades (well, >>> almost) without any problems, at least I haven't heard of them. >>> >>> What would using Addr buy us? Addr is non-standard, and you would >>> have to use some cast from a Ptr somehow (the world outside GHC's innards >>> is using Ptr, not Addr). This would buy you exactly zero safety or clarity, >>> and it would only introduce API friction. >>> >>> >>>> > If you have a pointer pointing to something and shift that >>>> pointer by some bytes, you are probably pointing to something completely >>>> different, so of course "b" and "a" have nothing to do with each other. So >>>> peekByteOff intentionally ignores "b". >>>> This also seems like a good use for Addr? >>>> >>> Nope again, see above. >>> >>> I still fail to see what problem this whole proposal is actually >>> trying to solve... >>> >> _______________________________________________ > > >> Libraries mailing list >> Libraries@haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries >> > > > -- > -Andrew Thaddeus Martin > _______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

One messy sketch of how this could proceed would be to make additional Addr-based analogues of the report specified type-unrelated "Ptr a" functions and install them side by side with the existing functions. This would behave much like how we have both readsPrec and readPrec for standard vs. GHC specific Read internals. With MINIMAL pragmas it should basically become transparent if they are mutually defined. Since generally Storable dictionaries aren't built out of compositions of other Storable dictionaries, growing the class shouldn't do any measurable harm to performance. We haven't been shy about adding new members to report-specified classes like Bits. This doesn't strike me as much different. At some point in the future we can work out if it is worth it to get the report fixed up by incorporating the Addr-based API as the default and make the _other_ the legacy. -Edward On Tue, Oct 30, 2018 at 10:11 AM Sven Panne wrote:

I am not sure if everybody fully comprehends what Storable is all about: It is meant as the lowest-level building block in an Addr-free world (remember: Addr is a GHCism and is *not* mentioned anywhere in the report) to put a few well-defined simple Haskell types into memory or read them from there. Its explicit non-goals are:

* Achieve 100% type safety. In the presence of raw memory access, castPtr, C calls etc. this would be a total illusion. Forcing API users to sprinkle tons of castPtr at every possible place over their code wouldn't improve safety at all, it would only hurt readability.

* Handle more complicated sum/product types. How would you do this? Respect your native ABI (i.e. automatically handle padding/alignment)? Tightly packed? Or even handle a foreign ABI? Your own ABI? Some funny encoding like OpenGL's packed data types? Etc. etc. You can build all of those things in a layer above Storable, probably introducing other type classes or some marshaling DSLs.

* Portability of the written values. This is more in the realm of serialization libraries.

More concretely:

Am Di., 30. Okt. 2018 um 14:34 Uhr schrieb Daniel Cartwright < chessai1996@gmail.com>:

...

[19:26:50] hPutBuf :: Handle -> Ptr a -> Int -> IO () [...]

The signature for this is actually perfect: hPutBuf doesn't care about what stuff has been written into the given buffer, it just cares about its start and its size. Forcing castPtr Kung Fu here wouldn't buy you anything: The buffer will probably contain a wild mix of Haskell values or even no Haskell values at all, but that doesn't matter. Whatever you pass as "a" or whatever you cast from/to is probably a lie from the typing perspective. At this level this is no problem at all.

...

[19:30:02] peekByteOff :: Ptr b -> Int -> IO a [19:30:09] peekByteOff :: Addr -> Int -> IO a [19:30:26] what is 'b' doing there? it's not used in any meaningful way by peekByteOff [...]

If you have a pointer pointing to something and shift that pointer by some bytes, you are probably pointing to something completely different, so of course "b" and "a" have nothing to do with each other. So peekByteOff intentionally ignores "b".

...

[19:32:22] <carter> pokeElemOff :: Ptr a -> Int -> a -> IO () --- way better than peak [...]

Yes, because this is intended to be used for *arrays* of values of the same type. Note "Elem" vs. "Byte".

...

[19:33:12] <carter> hvr: lets add safePeekByteOff :: Ptr a -> Int -> IO a ? [...]

This signature doesn't make sense, see above: Shifting a pointer by an arbitrary amount of bytes will probably change the type of what you're pointing to. If you shift by units of the underlying type, well, that's peekElemOff.

...

[19:35:31] carter: i am glad we agree on the smell

I don't have the full chat log, but I think I don't even agree on the smell, at least not at the places I've seen... :-)

_______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

On Wed, Oct 31, 2018 at 9:08 PM Anthony Cowley wrote:

Edward Kmett writes:

...

One messy sketch of how this could proceed would be to make additional Addr-based analogues of the report specified type-unrelated "Ptr a" functions and install them side by side with the existing functions. This would behave much like how we have both readsPrec and readPrec for standard vs. GHC specific Read internals. With MINIMAL pragmas it should basically become transparent if they are mutually defined. Since generally Storable dictionaries aren't built out of compositions of other Storable dictionaries, growing the class shouldn't do any measurable harm to performance.

This doesn't sound right to me. I'm probably an outlier, but I rely heavily on composite Storable instances in vinyl and Frames and general FFI situations. I don't know if a more common usage like an ad hoc record relying on the Storable instances of its fields would be greatly impacted, but this needs some investigation.

Fair point. It does feel at least worth benchmarking if we do decide to go down this path.

More generally, I'm in agreement with the points Sven has been making in this discussion, and wouldn't like to see such an unresolved debate be resolved by imposing a performance penalty on everyone as a compromise.

That said coercing copies of ~2 definitions into a class that already has 8 members seems unlikely to move the needle on this to me. If, say, 1-3% is a cost that could never be borne then we'd never be able to get around to fixing things like, say, sizeOf or alignment taking actively harmful arguments, rather than just type arguments or proxies, so we might need to consider what is an acceptable trade off in terms of leaning into the current local optima vs. allowing for growth. Fixing those warts some day in an eventually standardizable way would incur exactly the same amount of overhead. If on the other hand the cost winds up being appreciably higher than that, then by all means its worth considering other approaches like leaving the methods in the class alone and adding the Addr versions as top level methods, if we decide we do want to move Addr into base. I like that option somewhat less as it means that there is no real sane roadmap for how to go from there to a simpler story in the future, even if we don't choose to start down the road to standardizing that behavior prematurely today.

We haven't been shy about adding new members to

...

report-specified classes like Bits. This doesn't strike me as much different.

It is similar, but I think composite Storable instances are much more common than composite Bits instances.

Fair, I wound up using a similar scheme for OpenGL Uniform and SSBO serialization. It is kind of a mess though as you need to track not just the max alignment, you need to run through a pass computing current term alignments in order to correctly pack and compute sizeOf. Otherwise if you stuck a pair of Word16s after a Word32 you're stuck with Word32 alignment all the way through, and it doesn't even manage to resemble even the worst struct packing schemes of the C world. It strikes me that relative to _that_ setup or evaluation overhead, the overhead of gluing an extra couple of methods for {peek|poke}ByteOffAddr onto a class that already has 8 fields in it is unlikely to be appreciable. I could well be wrong. My comment was not an attempt to shut down debate but to try to find some way under which the proposal could plausibly proceed. -Edward Anthony

...

At some point in the future we can work out if it is worth it to get the report fixed up by incorporating the Addr-based API as the default and

...

the _other_ the legacy.

-Edward

On Tue, Oct 30, 2018 at 10:11 AM Sven Panne wrote:

...

I am not sure if everybody fully comprehends what Storable is all about: It is meant as the lowest-level building block in an Addr-free world (remember: Addr is a GHCism and is *not* mentioned anywhere in the report) to put a few well-defined simple Haskell types into memory or read them from there. Its explicit non-goals are:

* Achieve 100% type safety. In the presence of raw memory access, castPtr, C calls etc. this would be a total illusion. Forcing API users to sprinkle tons of castPtr at every possible place over their code wouldn't improve safety at all, it would only hurt readability.

* Handle more complicated sum/product types. How would you do this? Respect your native ABI (i.e. automatically handle padding/alignment)? Tightly packed? Or even handle a foreign ABI? Your own ABI? Some funny encoding like OpenGL's packed data types? Etc. etc. You can build all of those things in a layer above Storable, probably introducing other type classes or some marshaling DSLs.

* Portability of the written values. This is more in the realm of serialization libraries.

More concretely:

Am Di., 30. Okt. 2018 um 14:34 Uhr schrieb Daniel Cartwright < chessai1996@gmail.com>:

...

[19:26:50] hPutBuf :: Handle -> Ptr a -> Int -> IO () [...]

The signature for this is actually perfect: hPutBuf doesn't care about what stuff has been written into the given buffer, it just cares about its start and its size. Forcing castPtr Kung Fu here wouldn't buy you anything: The buffer will probably contain a wild mix of Haskell values or even no Haskell values at all, but that doesn't matter. Whatever you pass as "a" or whatever you cast from/to is probably a lie from the typing

make perspective. At

...

...

this level this is no problem at all.

...

[19:30:02] peekByteOff :: Ptr b -> Int -> IO a [19:30:09] peekByteOff :: Addr -> Int -> IO a [19:30:26] what is 'b' doing there? it's not used in any meaningful way by peekByteOff [...]

If you have a pointer pointing to something and shift that pointer by some bytes, you are probably pointing to something completely different, so of course "b" and "a" have nothing to do with each other. So peekByteOff intentionally ignores "b".

...

[19:32:22] <carter> pokeElemOff :: Ptr a -> Int -> a -> IO () --- way better than peak [...]

Yes, because this is intended to be used for *arrays* of values of the same type. Note "Elem" vs. "Byte".

...

[19:33:12] <carter> hvr: lets add safePeekByteOff :: Ptr a -> Int -> IO a ? [...]

This signature doesn't make sense, see above: Shifting a pointer by an arbitrary amount of bytes will probably change the type of what you're pointing to. If you shift by units of the underlying type, well, that's peekElemOff.

...

[19:35:31] carter: i am glad we agree on the smell

I don't have the full chat log, but I think I don't even agree on the smell, at least not at the places I've seen... :-)

_______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

---

Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

Sven raises a good point : Why can’t this all be workshopped out in user space ? Plus it’s pretty easy to ground proposals that might break stuff by ... trying to build stuff! On Thu, Nov 1, 2018 at 8:08 AM Sven Panne wrote:

Am Do., 1. Nov. 2018 um 01:22 Uhr schrieb Edward Kmett :

...

[...] At some point in the future we can work out if it is worth it to get the report fixed up by incorporating the Addr-based API as the default and make the _other_ the legacy.

Even though this thread is slowly approaching 100 mails, I still haven't heard a single convincing argument *why* anything should changed or *what* exactly is broken and *how* it effects API users. I very much disagree that there is anything to be "fixed" in that area. If you dislike the part of the Haskell Report, you are free to build your own APIs built on Addr, nothing is stopping you from doing that. Of course you will have some friction to the rest of the world which is using Ptr, but this is the price to pay when you deviate from standards.

In a nutshell: Please show me e.g. some Wiki page with facts and real problems, not aesthetic opinions. It should be clear by now that we will never reach a consensus on that level, so in consequence the current status quo *must* win.

Regarding the more fancy suggestions to sizeOf/alignment (proxies, some type Kung Fu): Remember that the FFI is part of the Haskell Report, so unless these things are in the report, too, there is no point discussing about such options. I'm not opposed to include such things in the report, far from it, but these things must go in first. _______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

Yeah, that's a good point, David. Also, no one is talking about removing the existence of Ptr - just providing Addr and using it where appropriate.

The arithmetic parts are pretty obvious (indeed, it might be nice to add some more arithmetic primops to treat Addr# as a uintptr_t). The rest are much less obvious; it may make more sense to convert an Addr to a Ptr before performing any dereferencing operations. On Fri, Oct 26, 2018, 1:55 PM Carter Schonwald wrote:

What api? It’s not obvious, could you spell out what you have in mind. Relative to the phab straw man?

On Fri, Oct 26, 2018 at 1:42 PM Daniel Cartwright wrote:

...

there was never any hint at helping primitive, i'm not sure where you got that from. also, the primary motivation was to avoid using 'Ptr a', where 'a' is a lie. not only that, but base seems a natural home for 'Addr', and people can avoid incurring a dep on primitive when they just need 'Addr'.

'super useful' was never the goal - only correctness and convenience.

Yeah, i agree that Storable should not be touched.

On Fri, Oct 26, 2018 at 1:36 PM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

your perspective here is a good one

1) i absolutely agree, nothing should touch storable, whatesoever,

2) the baseline proposal is to add an Addr to Base (essentially Ptr ()), which would be a crippled sibling of how its exposed in Data.Primitive I guess?

3) as a litmus for "what would this acomplish/support", i was asking "how would this get used/help base be nicer "? If

anyways: i did a strawman of what Addr ripped out of Data.Primitive.Addr and into base would look like, and it doesn't look especially compelling / nicer than Ptr shenanigans. Especially since to be useful in base it would have to have a bunch of IO / ST specific operations that have the option perhaps of using Storable to read /write at locations. At which point I still need to have the same API surface area again in Primitive,

see https://phabricator.haskell.org/D5268 for the bare bones stuff (doesn't type check)

the api i could expose there in base does not reduce the implementation surface area needed for Primitve...

i'm open to being convinced otherwise, but with Sven's perspective weighed in, 1) i dont see it being super useful within base/ghc apis, as they exist today 2) it doesn't reduce implementation surface area burden in the Primitive package.

so at this point im' weakly against the addition. more work, no clear reward for me :)

On Fri, Oct 26, 2018 at 1:26 PM Sven Panne wrote:

...

Am Fr., 26. Okt. 2018 um 06:05 Uhr schrieb Carter Schonwald < carter.schonwald@gmail.com>:

...

[...] I guess I’m just trying to say “what would the impact on base, if every fake Ptr was moved to be Addr?” Because it’s not something which should be done by halves.

1) [...]

2) [...]

3). [...]

The most important question is missing from this list: What are the benefits of touching such a crucial part of our API ecosystem? There should better be extremely good reasons, otherwise lots of annoying work and incompatibility is generated for nothing. Reading through this thread, I haven't seen a good motivation for touching Storable/Ptr and friends. Perhaps I have misunderstood what the actual proposal is, initially I thought it is just replacing various "Ptr foo" with Addr only within GHC itself. That's fine, but the scope seems to have broadened.

Just a historical remark: Addr# is a GHCism, and so was Addr. At the time the FFI came up, Ptr was intended to be the portable type. So yes, "Ptr a" and "Ptr ()" are basically just Addr (modulo casts), but intentionally so.

_______________________________________________

...

Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

hrmm, it looks like those aren't exposed directly to the HS layer because of the necessary compiler time static alignment requirements for that in code gen, On Fri, Oct 26, 2018 at 2:13 PM Carter Schonwald wrote:

yeah, lets do that, so roughly very little impact but easy opt in.

Is the design code strawman in https://phabricator.haskell.org/D5268 acceptable?

@David Feuer could you help me dig out where in ghc / base / ghc-prims we expose the memmove/memcopy/memset intrinsics? I can only seem to find the c ffi call ones, but I know ghc itself can compile that more smartly? i started but then i got lost, seems like everywhere in base its just a bespoked c ffi binding introduced locally ..

On Fri, Oct 26, 2018 at 2:02 PM David Feuer wrote:

...

The arithmetic parts are pretty obvious (indeed, it might be nice to add some more arithmetic primops to treat Addr# as a uintptr_t). The rest are much less obvious; it may make more sense to convert an Addr to a Ptr before performing any dereferencing operations.

On Fri, Oct 26, 2018, 1:55 PM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

What api? It’s not obvious, could you spell out what you have in mind. Relative to the phab straw man?

On Fri, Oct 26, 2018 at 1:42 PM Daniel Cartwright wrote:

...

there was never any hint at helping primitive, i'm not sure where you got that from. also, the primary motivation was to avoid using 'Ptr a', where 'a' is a lie. not only that, but base seems a natural home for 'Addr', and people can avoid incurring a dep on primitive when they just need 'Addr'.

'super useful' was never the goal - only correctness and convenience.

Yeah, i agree that Storable should not be touched.

On Fri, Oct 26, 2018 at 1:36 PM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

your perspective here is a good one

1) i absolutely agree, nothing should touch storable, whatesoever,

2) the baseline proposal is to add an Addr to Base (essentially Ptr ()), which would be a crippled sibling of how its exposed in Data.Primitive I guess?

3) as a litmus for "what would this acomplish/support", i was asking "how would this get used/help base be nicer "? If

anyways: i did a strawman of what Addr ripped out of Data.Primitive.Addr and into base would look like, and it doesn't look especially compelling / nicer than Ptr shenanigans. Especially since to be useful in base it would have to have a bunch of IO / ST specific operations that have the option perhaps of using Storable to read /write at locations. At which point I still need to have the same API surface area again in Primitive,

see https://phabricator.haskell.org/D5268 for the bare bones stuff (doesn't type check)

the api i could expose there in base does not reduce the implementation surface area needed for Primitve...

i'm open to being convinced otherwise, but with Sven's perspective weighed in, 1) i dont see it being super useful within base/ghc apis, as they exist today 2) it doesn't reduce implementation surface area burden in the Primitive package.

so at this point im' weakly against the addition. more work, no clear reward for me :)

On Fri, Oct 26, 2018 at 1:26 PM Sven Panne wrote:

...

Am Fr., 26. Okt. 2018 um 06:05 Uhr schrieb Carter Schonwald < carter.schonwald@gmail.com>:

> [...] I guess I’m just trying to say “what would the impact on base, > if every fake Ptr was moved to be Addr?” Because it’s not something which > should be done by halves. > > 1) [...] > > 2) [...] > > 3). [...] >

The most important question is missing from this list: What are the benefits of touching such a crucial part of our API ecosystem? There should better be extremely good reasons, otherwise lots of annoying work and incompatibility is generated for nothing. Reading through this thread, I haven't seen a good motivation for touching Storable/Ptr and friends. Perhaps I have misunderstood what the actual proposal is, initially I thought it is just replacing various "Ptr foo" with Addr only within GHC itself. That's fine, but the scope seems to have broadened.

Just a historical remark: Addr# is a GHCism, and so was Addr. At the time the FFI came up, Ptr was intended to be the portable type. So yes, "Ptr a" and "Ptr ()" are basically just Addr (modulo casts), but intentionally so.

_______________________________________________

...

Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

Ptr Word8 is not (just) a pointer to byte-addressable memory. It's specifically a pointer to something representing an 8-bit unsigned integer. If you use a Ptr Word8 for the address of a floating point value, then you are *wrong* (or doing something very weird, anyway). To quote the documentation, A value of type Ptr a represents a pointer to an object, or an array of objects, which may be marshalled to or from Haskell values of type a. On Thu, Oct 25, 2018, 11:23 PM Carter Schonwald wrote:

... when is a valid pointer not going to point at byte addressable memory on memory architectures ghc can support or target?

On Thu, Oct 25, 2018 at 8:27 PM Daniel Cartwright wrote:

...

Now, one could argue that `Ptr ()` isn't a lie, it sort of reads like C's void pointer. But surely something like `Ptr Word8` is a lie, when it is not actually a Ptr to Word8 values.

On Thu, Oct 25, 2018 at 8:11 PM Daniel Cartwright wrote:

...

yes, only the type and its instances should be moved as far as i'm aware.

Also, it's more than just base.

this Ptr is a lie: https://hackage.haskell.org/package/ghc-compact-0.1.0.0/docs/GHC-Compact-Ser... these Ptrs are lies: https://hackage.haskell.org/package/base-4.12.0.0/docs/GHC-IO-Handle.html in GHC.Stats, the foreign import "getRTSStats" has `Ptr () -> IO ()`, this Ptr () is also a lie

These are just off the top of my head, there are more

On Thu, Oct 25, 2018 at 6:46 PM Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

hrmm, what are the pieces of base that are using Ptr when they really should be using Addr? This would help me understand what would be made better in base :)

On Thu, Oct 25, 2018 at 6:19 PM David Feuer wrote:

...

We shouldn't really need to move anything into base except Addr and its base instances.

On Oct 25, 2018 5:59 PM, "Carter Schonwald" < carter.schonwald@gmail.com> wrote:

Indeed. The monad transformer instances for primmonad need to live in primmonad OR transformers to avoid orphans.

Either way, unless transformers moves into base (unlikely), no way anything using prim monad will.

On Thu, Oct 25, 2018 at 3:34 PM Andrew Martin < andrew.thaddeus@gmail.com> wrote:

...

I like the idea of moving the type Addr into base. But we cannot move the entire module since it has functions that talk about PrimMonad, and we definitely don't want to move that into base.

On Thu, Oct 25, 2018 at 11:25 AM Daniel Cartwright < chessai1996@gmail.com> wrote:

> Motivation: There are a lot of places in base where 'Ptr a' is used > in place of 'Addr', because in base there is no 'Addr', only 'Addr#'. The > problem lies in the fact that many of these uses of 'Ptr a' are lying; the > 'a' value is meaningless. Authors of functions therein have used things > like 'Ptr ()', 'Ptr Word8', 'Ptr a', but these types do not mean what they > say they mean - they're just Addr. There are probably other motivations for > this that I can't think of off the top of my head right now. > _______________________________________________ > Libraries mailing list > Libraries@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries >

-- -Andrew Thaddeus Martin _______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

While we're in that realm, it would be nice to find a mechanism that will explain what we need for both Storable (stick something at an Addr address) and Primitive (stick something in a MutableByteArray). The current situation is rather annoying. Can the compiler help? On Fri, Oct 26, 2018, 5:15 AM Henning Thielemann < lemming@henning-thielemann.de> wrote:

On Thu, 25 Oct 2018, David Feuer wrote:

...

Ptr Word8 is not (just) a pointer to byte-addressable memory. It's specifically a pointer to something representing an 8-bit unsigned integer. If you use a Ptr Word8 for the address of a floating point value, then you are *wrong* (or doing something very weird, anyway). To quote the documentation,

A value of type Ptr a represents a pointer to an object, or an array of objects, which may be marshalled to or from Haskell values of type a.

Interesting observations.

There are the Storable methods

peekByteOff :: Ptr b -> Int -> IO a pokeByteOff :: Ptr b -> Int -> a -> IO ()

They ignore the target types of their pointers. Should be Addr, then, too?

And if we are touching Storable class, how about passing Proxy's to sizeOf and alignment instead of undefined values? Maybe we should add new methods to Storable class with default implementations that redirect to the old methods.

Am Fr., 26. Okt. 2018 um 11:15 Uhr schrieb Henning Thielemann < lemming@henning-thielemann.de>:

[...] There are the Storable methods

peekByteOff :: Ptr b -> Int -> IO a pokeByteOff :: Ptr b -> Int -> a -> IO ()

They ignore the target types of their pointers.

... and for a good reason: This was intentionally specified this way to avoid the castPtr-Kung-Fu which would arise if the pointer arguments were e.g. "Ptr ()". Remember: You are on a totally unsafe level here, anyway, so adding tons of casts would not really help here much.

[...] And if we are touching Storable class, [...]

I really hope that Storable remains *totally* untouched, at least in the sense that it is only changed in a 100% backwards-compatible away. Otherwise basically the whole Haskell ecosystem with its tons of native bindings will explode... There is nothing wrong with a new class, but this one is effectively "untouchable".

That’s not a bad design pattern. I think eds struct Lib is sortah similar to that. On Sat, Nov 3, 2018 at 2:56 AM Henning Thielemann < lemming@henning-thielemann.de> wrote:

On Fri, 26 Oct 2018, Henning Thielemann wrote:

...

There are the Storable methods

peekByteOff :: Ptr b -> Int -> IO a pokeByteOff :: Ptr b -> Int -> a -> IO ()

They ignore the target types of their pointers. Should be Addr, then, too?

Thinking about it ... these functions are intended for access of record fields. Then the following type signatures would have been better:

peekField :: Ptr b -> Offset b a -> IO a pokeField :: Ptr b -> Offset b a -> a -> IO ()

newtype Offset b a = Offset Int

This way we had more type safety and no need for Addr or lying Ptr's.

http://hackage.haskell.org/package/structs On Sat, Nov 3, 2018 at 1:04 PM Henning Thielemann wrote:

On Sat, 3 Nov 2018, Carter Schonwald wrote:

...

That’s not a bad design pattern. I think eds struct Lib is sortah similar to that.

What is "eds struct Lib" ?_______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries