- ghc-devs - Haskell.org

Re: Support conversion from (UArray i Word8) to ShortByteString?
by Matthew Craven 14 Nov '23

14 Nov '23

Viktor, Your proposed `arrayToByteArray` seems plausible. Your proposed `byteArrayToShort` is just the newtype-constructor `ShortByteString` which is exposed from Data.ByteString.Short since bytestring-0.12.0.0. > An alternative is to add a tailored version of the UArray and STUArray > APIs to 'MutableByteArray' by extending the rather limited API of > 'Data.Array.Byte': How does this compare to the interface provided by primitive:Data.Primitive.ByteArray? Their `runByteArray` is your `runMutableByteArray`. > The 'ShortByteString' type in the "bytestring" package has seen some > singnificant improvement recently, and yet its API is still noticeably > limited in comparison to its pinned, I/O friendly 'ByteString' elder > sibling. > > One of the limitations is that that there are fewer ways to construct a > 'ShortByteString' object. One often has to restort to constructing a > pinned ByteString, and then copy. (There no "ST" Builders that write > to resizable MutableByteArrays instead of raw memory pointers). > > Meanwhile, under-the covers, both the "UArray i Word8" type and > 'ShortByteString' hold an immutable 'ByteArray', and the STUArray API > provides a flexible "UArray" construction interface. > > Would it be reasonable to "bridge" the two APIs: > > Data.Array.Unboxed: (re-export from Data.Array.Base) > import Data.Array.Byte > > arrayToByteArray :: UArray i Word8 -> ByteArray > arrayToByteArray (UArray _ _ _ ba#) = ByteArray ba# > {-# INLINE arrayToByArrray #-} > > Data.ByteString.Short: (re-export from Data.ByteString.Short.Internal) > byteArrayToShort :: ByteArray -> ShortByteString > byteArrayToShort = coerce > {-# INLINE byteArrayToShort #-} > > It would then be possible to write: > > short = byteArrayToShort $ arrayToByteArray $ runSTUArray m > where > m = do > a <- newArray (0, last) 0 -- zero fill > sequence_ [ writeArray a ix e | (ix, e) <- generator ] > > and generate the bytes of a 'ShortByteString' from an arbitrary > computation, possibly merging multiple inputs into some bytes by using > the recently introduced "modifyArray" (or explicit read/modify/write). > > Any thoughts about the wisdom or lack thereof of this proposal? > > An alternative is to add a tailored version of the UArray and STUArray > APIs to 'MutableByteArray' by extending the rather limited API of > 'Data.Array.Byte': > > runMutableByteArray :: (forall s. ST s (MutableByteArray s)) > -> ByteArray > runMutableByteArray m = runST $ m >>= freezeMutableByteArray > > freezeMutableByteArray (MutableByteArray mba#) = > ST $ \s -> case unsafeFreezeByteArray# mba# s of > (# s', ba# #) -> (# s', ByteArray ba# #) > > Since "Data.Array.Byte" is an "array" (rather than string) interface, it > could have a richer set of indexed read/write/modify primitives along > the lines of those found in "Data.Array.STUArray", but specialised to > 'Word8' elements and implicit zero-based integer indexing. > > The flexible construction I seek would then be via "Data.Array.Bytes", > rather than the somewhat too general index and value types from UArray. > > short = byteArrayToShort $ runMutableByteArray m > where > m = do > a <- newByteArray size 0 -- 0 fill > sequence_ [ writeByteArray a ix e | (ix, e) <- generator ] > > In this scenario, the indexed-mutation of ShortByteStrings under > construction, or indexed-mutation of copies for various transformations, > could live in Data.Array.Byte, with ShortByteString and various > applications leveraging the random-access mutation (and resizing, ...) > to implement higher level operations. > > -- > Viktor.

2 1

Support conversion from (UArray i Word8) to ShortByteString?
by Viktor Dukhovni 11 Nov '23

11 Nov '23

The 'ShortByteString' type in the "bytestring" package has seen some singnificant improvement recently, and yet its API is still noticeably limited in comparison to its pinned, I/O friendly 'ByteString' elder sibling. One of the limitations is that that there are fewer ways to construct a 'ShortByteString' object. One often has to restort to constructing a pinned ByteString, and then copy. (There no "ST" Builders that write to resizable MutableByteArrays instead of raw memory pointers). Meanwhile, under-the covers, both the "UArray i Word8" type and 'ShortByteString' hold an immutable 'ByteArray', and the STUArray API provides a flexible "UArray" construction interface. Would it be reasonable to "bridge" the two APIs: Data.Array.Unboxed: (re-export from Data.Array.Base) import Data.Array.Byte arrayToByteArray :: UArray i Word8 -> ByteArray arrayToByteArray (UArray _ _ _ ba#) = ByteArray ba# {-# INLINE arrayToByArrray #-} Data.ByteString.Short: (re-export from Data.ByteString.Short.Internal) byteArrayToShort :: ByteArray -> ShortByteString byteArrayToShort = coerce {-# INLINE byteArrayToShort #-} It would then be possible to write: short = byteArrayToShort $ arrayToByteArray $ runSTUArray m where m = do a <- newArray (0, last) 0 -- zero fill sequence_ [ writeArray a ix e | (ix, e) <- generator ] and generate the bytes of a 'ShortByteString' from an arbitrary computation, possibly merging multiple inputs into some bytes by using the recently introduced "modifyArray" (or explicit read/modify/write). Any thoughts about the wisdom or lack thereof of this proposal? An alternative is to add a tailored version of the UArray and STUArray APIs to 'MutableByteArray' by extending the rather limited API of 'Data.Array.Byte': runMutableByteArray :: (forall s. ST s (MutableByteArray s)) -> ByteArray runMutableByteArray m = runST $ m >>= freezeMutableByteArray freezeMutableByteArray (MutableByteArray mba#) = ST $ \s -> case unsafeFreezeByteArray# mba# s of (# s', ba# #) -> (# s', ByteArray ba# #) Since "Data.Array.Byte" is an "array" (rather than string) interface, it could have a richer set of indexed read/write/modify primitives along the lines of those found in "Data.Array.STUArray", but specialised to 'Word8' elements and implicit zero-based integer indexing. The flexible construction I seek would then be via "Data.Array.Bytes", rather than the somewhat too general index and value types from UArray. short = byteArrayToShort $ runMutableByteArray m where m = do a <- newByteArray size 0 -- 0 fill sequence_ [ writeByteArray a ix e | (ix, e) <- generator ] In this scenario, the indexed-mutation of ShortByteStrings under construction, or indexed-mutation of copies for various transformations, could live in Data.Array.Byte, with ShortByteString and various applications leveraging the random-access mutation (and resizing, ...) to implement higher level operations. -- Viktor.

1 0

Nofib
by Simon Peyton Jones 11 Nov '23

11 Nov '23

Dear devs Building 'nofib' with HEAD fails with real/scs/Parse.hs:40:19: error: [GHC-87543] Ambiguous occurrence ‘List’. It could refer to either ‘Data.List.List’, imported from ‘Data.List’ at real/scs/Parse.hs:4:1-16 (and originally defined in ‘GHC.Types’), or ‘Types.List’, imported from ‘Types’ at real/scs/Parse.hs:8:1-12. | 40 | list :: Parser List | ^^^^ This is because of a change in the export list of Data.List. The fix is easy (hide the import of List). But I don't know how to update nofib and the subdmodule magic to make HEAD pick the new version. Could anyone possible execute on this? Thanks Simon

3 3

[ANNOUNCE] GHC 9.4.8 is now available
by Zubin Duggal 10 Nov '23

10 Nov '23

The GHC developers are happy to announce the availability of GHC 9.4.8. Binary distributions, source distributions, and documentation are available on the [release page](/download_ghc_9_4_8.html). This release is primarily a bugfix release addressing a few issues found in the 9.4 series. These include: * A fix for a recompilation checking bug where GHC may miss changes in transitive dependencies when deciding to relink a program (#23724). * A fix for a code generator bug on AArch64 platforms resulting in invalid conditional jumps (#23746). * Support for `-split-sections` on Windows. * Enabling `-split-sections` for various Linux and Windows binary distributions, enabling GHC to produce smaller binaries on these platforms. * And a few other fixes A full accounting of changes can be found in the [release notes]. As some of the fixed issues do affect correctness users are encouraged to upgrade promptly. We would like to thank Microsoft Azure, GitHub, IOG, the Zw3rk stake pool, Well-Typed, Tweag I/O, Serokell, Equinix, SimSpace, Haskell Foundation, and other anonymous contributors whose on-going financial and in-kind support has facilitated GHC maintenance and release management over the years. Finally, this release would not have been possible without the hundreds of open-source contributors whose work comprise this release. As always, do give this release a try and open a [ticket][] if you see anything amiss. Happy compiling! -Zubin [ticket]: https://gitlab.haskell.org/ghc/ghc/-/issues/new [release notes]: https://downloads.haskell.org/~ghc/9.4.8/docs/users_guide/9.4.8-notes.html

2 2

Report on my little ticket triage exercise
by Bryan Richter 03 Nov '23

03 Nov '23

Hello, Over the past few months, I have looked at tickets labeled "Priority: highest" to get a feel for things. I looked at the three stalest once a fortnight using this link. https://gitlab.haskell.org/ghc/ghc/-/issues/?sort=updated_asc&state=opened&… I managed to find some that could be closed, some that could be deprioritized, and some that had fallen through the cracks. Overall it felt really nice. Now I've started seeing the same issues, meaning I've worked my way through the entire list. So I think all of the remainders are current and valid. Having done that, I decided I'd try looking at "Priority: high" tickets. Unfortunately the same triage strategy won't work. While "highest" says to me, "Hey, we need to fix this!", it's not clear what "high" means. I think a mature project like GHC may very well have many tickets that are important, but not important enough to halt other work. That means these tickets may just be stale and that's that. But then how might I review them? My strategy of simply looking at the 3 stalest once a fortnight won't work anymore, because it'll just be the same three each time. :) At least with "Priority: Highest" tickets, I felt justified commenting, "Hi, is this still valid?" if I had nothing else meaningful to add. That would push the ticket to the bottom of the queue. Would people mind if I do the same thing with "Priority: High" tickets? I do worry it might cause useless churn. Nonetheless, I think a slow review like this is valuable, since undoubtedly there are tickets that can be closed or reprioritized. I could also do something dull like toggle some metadata, e.g. assign and then unassign myself, which would also "refresh" the ticket and push it to the bottom of the list. Does anybody have any better ideas?

1 0

Type-level sized Word literals???
by Viktor Dukhovni 31 Oct '23

31 Oct '23

I am working on some code where it is useful to have types indexed by a 16-bit unsigned value. Presently, I am using type-level naturals and having to now and then provide witnesses that a 'Nat' value I am working with is at most 65535. Also, perhaps there's some inefficiency here, as Naturals have two constructors, and so a more complex representation with (AFAIK) no unboxed forms. I was wondering what it would take to have type-level fixed-size Words (Word8, Word16, Word32, Word64) to go along with the Nats? It looks like some of the machinery (KnownWord16, SomeWord16, wordVal16, etc.) can be copied straight out of GHC.TypeNats with minor changes, and that much works, but the three major things that are't easily done seem to be: - There are it seems no TypeReps for types of Kind Word16, so one can't have (Typeable (Foo w)) with (w :: Word16). - There are no literals of a promoted Word16 Kind. type Foo :: Word16 -> Type data Foo w = MkFoo Int -- 1 has Kind 'Natural' (a.k.a. Nat) x = MkFoo 13 :: Foo 1 -- Rejected, -- The new ExtendedLiterals syntax does not help -- x = MkFoo 13 :: Foo (W16# 1#Word16) -- Syntax error! - There are unsurprisingly also no built-in 'KnownWord16' instances for any hypothetical type-level literals of Kind Word16. Likely the use case for type-level fixed-size words is too specialised to rush to shoehorn into GHC, but is there something on the not too distant horizon that would make it easier and reasonable to have fixed-size unsigned integral type literals available? [ I don't see a use-case for unsigned versions, they can trivially be represented by the unsigned value of the same width. ] With some inconvenience, in many cases I can perhaps synthesise Proxies for types of Kind Word16, and just never use literals directly. -- Viktor.

5 6

"Merge-buddy" request
by Alan & Kim Zimmerman 27 Oct '23

27 Oct '23

Hi all I have been landing a series of MRs to simplify the exact print annotations. They are split into pieces so each change is not too big. I still have quite a number of them to land, but am finding the process slow, as I - make the MR - wait for CI to go green - wait for a review, or ping people on chat to do the review - once approved, land it. I don't want to loudly shout for reviewers each time, and become a general irritant. Is there anyone (ideally with an interest in the exact print annotations) that would like to be my "merge buddy" that I can easily ask to do the reviews? I am happy for each one to take a couple of days, so it is not a high pressure thing, I just want to get into a cadence on them. My current one is https://gitlab.haskell.org/ghc/ghc/-/merge_requests/11496. Admittedly only one reviewer requested, and he is always busy, and I do not want to make him even more so. Alan

4 4

PMC: addConCt and newtypes bottom info
by Rodrigo Mesquita 27 Oct '23

27 Oct '23

Dear Sebastian and GHC devs, Regarding this bit from the function addConCt in the GHC.HsToCore.Pmc.Solver module, Nothing -> do let pos' = PACA alt tvs args : pos let nabla_with bot' = nabla{ nabla_tm_st = ts{ts_facts = addToUSDFM env x (vi{vi_pos = pos', vi_bot = bot'})} } -- Do (2) in Note [Coverage checking Newtype matches] case (alt, args) of (PmAltConLike (RealDataCon dc), [y]) | isNewDataCon dc -> case bot of MaybeBot -> pure (nabla_with MaybeBot) IsBot -> addBotCt (nabla_with MaybeBot) y IsNotBot -> addNotBotCt (nabla_with MaybeBot) y _ -> assert (isPmAltConMatchStrict alt ) pure (nabla_with IsNotBot) -- strict match ==> not ⊥ My understanding is that given some x which we know e.g. cannot be bottom, if we learn that x ~ N y, where N is a newtype (NT), we move our knowledge of x not being bottom to the underlying NT Id y, since forcing the newtype in a pattern is equivalent to forcing the underlying NT Id. Additionally, we set x’s BottomInfo to MaybeBot — However, I don’t understand why we must reset x’s BotInfo to MaybeBot — couldn’t we keep it as it is while setting y’s BotInfo to the same info? An example where resetting this info on the newtype-match is important/necessary would be excellent. FWIW, I built and tested the PMC of ghc devel2 with MaybeBot -> pure (nabla_with MaybeBot) IsBot -> addBotCt (nabla_with IsBot) y IsNotBot -> addNotBotCt (nabla_with IsNotBot) y And it worked without warnings or errors… Thanks in advance! Rodrigo

2 1

How do you keep tabs on commits that fix issues?
by Bryan Richter 16 Oct '23

16 Oct '23

I am not sure of the best ways for checking if a certain issue has been fixed on a certain release. My past ways of using git run into certain problems: The commit (or commits!) that fix an issue get rewritten once by Marge as they are rebased onto master, and then potentially a second time as they are cherry-picked onto release branches. So just following the original commits doesn't work. If a commit mentions the issue it fixes, you might get some clues as to where it has ended up from GitLab. But those clues are often drowning in irrelevant mentions: each failed Marge batch, for instance, of which there can be many. The only other thing I can think to do is look at the original merge request, pluck out the commit messages, and use git to search for commits by commit message and check each one for which branches contain it. But then I also need to know the context of the fix to know whether I should also be looking for other, logically related commits, and repeat the dance. (Sometimes fixes are only partially applied to certain releases, exacerbating the need for knowing the context.) This seems like a mechanism that can't rely on trusting the author of the original set of patches (which may be your past self) and instead requires a deep understanding to be brought to bear every time you would want to double check the situation. So it's not very scalable and I wouldn't expect many people to be able to do it. Are there better mechanisms already available? As I've said before, I am used to a different git workflow and I'm still learning how to use the one used by GHC. I'd like to know how others handle it. Thanks! -Bryan

7 7

Proposal for communication with GHC API users
by Facundo Domínguez 13 Oct '23

13 Oct '23

Dear GHC devs, Your feedback is appreciated in a new proposal to streamline communication with the authors of the GHC API as GHC evolves [1]. The proposal aims to set a process to collect detailed information on how users depend on the GHC API, and how GHC developers think these dependencies should be adjusted when there are breaking changes. This proposal is related to the endeavor to stabilize the GHC API [2]. Although, rather than trying to define a stable API, it sets a process to deal with the current state of affairs where users depend on many unstable parts. Looking forward to your thoughts! Facundo [1] https://gitlab.haskell.org/ghc/ghc/-/issues/24096 [2] https://discourse.haskell.org/t/charting-a-course-toward-a-stable-api-for-g… -- All views and opinions expressed in this email message are the personal opinions of the author and do not represent those of the organization or its customers.

1 1