- ghc-devs - Haskell.org

Unmerged Patch: 3358
by Matthew Pickering 21 Jul '20

21 Jul '20

Hi, My patch 3358 needs to get merged before the 8.12 fork. When I finished it (in May), it passed CI and after this point I lacked time to work on it further. Now I have asked 4 times for this patch to get merged and it is still open. The GHC proposal for this patch already took an extortionate amount of time to get accepted. Please can we close this chapter by merging the patch. Cheers, Matt

5 4

New Windows I/O manager in GHC 8.12
by Phyx 20 Jul '20

20 Jul '20

Hi All, In case you've missed it, about 150 or so commits were committed to master yesterday. These commits add WinIO (Windows I/O) to GHC. This is a new I/O manager that is designed for the native Windows I/O subsystem instead of relying on the broken posix-ish compatibility layer that MIO used. This is one of 3 big patches I have been working on for years now.. So before I continue on why WinIO was made I'll add a TL;DR; WinIO adds an internal API break compared to previous GHC releases. That is the internal code was modified to support a completely asynchronous I/O system. What this means is that we have to keep track of the file pointer offset which previously was done by the C runtime. This is because in async I/O you cannot assume the offset to be at any given location. What does this mean for you? Very little. If you did not use internal GHC I/O code. In particular if you haven't used Buffer, BufferIO and RawIO. If you have you will to explicitly add support for GHC 8.12+. Because FDs are a Unix concept and don't behave as you would expect on Windows, the new I/O manager also uses HANDLE instead of FD. This means that any library that has used the internal GHC Fd type won't work with WinIO. Luckily the number of libraries that have seems quite low. If you can please stick to the external Handle interface for I/O functions. The boot libraries have been updated, and in particular process *requires* the version that is shipped with GHC. Please respect the version bounds here! I will be writing a migration guide for those that need to migrate code. The amount of work is usually trivial as Base provides shims to do most of the common things you would have used Fd for. Also if I may make a plea to GHC developers.. Do not add non-trivial implementations in the external exposed modules (e.g. System.xxx, Data.xxx) but rather add them to internal modules (GHC.xxx) and re-export them from the external modules. This allows us to avoid import cycles inside the internal modules :) -- So why WinIO? Over the years a number of hard to fix issues popped up on Windows, including proper Unicode console I/O, cooked inputs, ability to cancel I/O requests. This also allows libraries like Brick to work on Windows without re-inventing the wheel or have to hide their I/O from the I/O manager. In order to attempt to do some of these with MIO layer upon layers of hacks were added. This means that things sometimes worked.., but when it didn't was rather unpredictable. Some of the issues were simply unfixable with MIO. I will be making some posts about how WinIO works (and also archiving them on the wiki don't worry :)) but for now some highlights: WinIO is 3 years of work, First started by Joey Hess, then picked up by Mikhail Glushenkov before landing at my feet. While the majority has been rewritten their work did provide a great jumping off point so thanks! Also thanks to Ben and AndreasK for helping me get it over the line.. As you can imagine I was exhausted by this point :). Some stats: ~8000 new lines and ~1100 removed ones spread over 130+ commits (sorry this was the smallest we could get it while not losing some historical context) and with over 153 files changed not counting the changes to boot libraries. It Fixes #18307, #17035, #16917, #15366, #14530, #13516, #13396, #13359, #12873, #12869, #11394, #10542, #10484, #10477, #9940, #7593, #7353, #5797, #5305, #4471, #3937, #3081, #12117, #2408, #10956, #2189 (but only on native windows consoles, so no msys shells) and #806 which is 14 years old! WinIO is a dynamic choice, so you can switch between I/O managers using the RTS flag --io-manager=[native|posix]. On non-Windows native is the same as posix. The chosen Async interface for this implementation is using Completion Ports. The I/O manager uses a new interface added in Windows Vista called GetQueuedCompletionStatusEx which allows us to service multiple request interrupts in one go. Some highlights: * Drops Windows Vista support Vista is out of extended support as of 2017. The new minimum is Windows 7. This allows us to use much more efficient OS provided abstractions. * Replace Events and Monitor locks with much faster and efficient Conditional Variables and SlimReaderWriterLocks. * Change GHC's Buffer and I/O structs to support asynchronous operation by not relying on the OS managing File Offset. * Implement a new command line flag +RTS --io-manager=[native|posix] to control which I/O manager is used. * Implement a new Console I/O interface supporting much faster reads/writes and unicode output correctly. Also supports things like cooked input etc. * In new I/O manager if the user still has their code-page set to OEM, then we use UTF-8 by default. This allows Unicode to work correctly out of the box. * Add Atomic Exchange PrimOp and implement Atomic Ptr exchanges. * Flush event logs eagerly as to not rely on finalizers running. * A lot of refactoring and more use of hsc2hs to share constants * Control aborts Ctrl+C should be a bit more reliable. * Add a new IOPort primitive that should be only used for these I/O operations. Essentially an IOPort is based on an MVar with the following major differences: - Does not allow multiple pending writes. If the port is full a second write is just discarded. - There is no deadlock avoidance guarantee. If you block on an IOPort and your Haskell application does not have any work left to do the whole application is stalled. In the threaded RTS we just continue idling, in the non-threaded rts the scheduler is blocked. * Support various optimizations in the Windows I/O manager such as skipping I/O Completion if the request finished synchronously etc. * The I/O manager is now agnostic to the handle type. i.e. There is no socket specific code in the manager. This is now all pushed to the network library. Completely de-coupling these. * Unified threaded and non-threaded I/O code. The only major difference is where event loop is driven from and that the non-threaded rts will always use a single OS thread to service requests. We cannot use more as there are no rts locks to make concurrent modifications safe. Cheers, Tamar

4 7

HEAD doesn't build. Totally stalled.
by Simon Peyton Jones 20 Jul '20

20 Jul '20

I'm getting this failure in a clean HEAD build. Any ideas? I'm totally stalled because I can't build GHC any more. I'm using Windows Subsystem for Linux (WSL). Help help! Thanks Simon /home/simonpj/code/HEAD-9/rts/dist/build/libHSrts_thr_p.a(RtsSymbols.thr_p_o): RtsSymbols.c:rtsSyms: error: undefined reference to '__stack_chk_guard' collect2: error: ld returned 1 exit status `cc' failed in phase `Linker'. (Exit code: 1) utils/iserv/ghc.mk:105: recipe for target 'utils/iserv/stage2_p/build/tmp/ghc-iserv-prof' failed make[1]: *** [utils/iserv/stage2_p/build/tmp/ghc-iserv-prof] Error 1 make[1]: *** Waiting for unfinished jobs....

5 12

DreamObjects outage
by Ben Gamari 18 Jul '20

18 Jul '20

Hi everyone, It seems like our block storage provider is currently experiencing a service interruption. Consequently you may find that accessing CI logs and CI artifacts fails. I have contact the provider and hopefully the issue will be resolved shortly. Cheers, - Ben

1 0

Error messages
by Erik de Castro Lopo 17 Jul '20

17 Jul '20

Hi all, Although it was many years ago I did spend soem time working on GHC and I do know what a thankless task it is. I made a compliant about GHC error messages on an internal Slack channel and Mortiz encouraged me to repeat it here. I am incredibly happy about the quality of error messges for older more standard parts of Haskell, probably most things in Haskell98 and Haskell2010. By way of contrast, error messages for some newer parts of Haskell are incredibly poor. In fact, for the new parts, the error rmessages are often wrong, just defaulting to error messages for older parts of Haskell. As an example (open source code in a public GH repo): src/Cardano/DbSync/StateQuery.hs:87:44: error: • Data constructor not in scope: QueryAnytime :: QueryHardFork xs0 (Interpreter xs0) -> Query (CardanoBlock TPraosStandardCrypto) (Interpreter (CardanoEras TPraosStandardCrypto)) • Perhaps you want to add ‘QueryAnytime’ to the import list in the import of ‘Ouroboros.Consensus.HardFork.Combinator.Ledger.Query’ (src/Cardano/DbSync/StateQuery.hs:49:1-116). | 87 | queryHistoryInterpreter connInfo (point, QueryAnytime GetInterpreter) The suggestion is that I need to import `QueryAnytime` but just 20 line above I have: import Ouroboros.....Query (QueryAnytime (..), QueryHardFork (GetInterpreter)) The problem is that `QueryAnytime` is defined as a pattern synonym. I have only the tinest amount of experience using pattern synonyms and that error message is not really useful. I would like to suggest that a prerequesite for merging of new features would be that it provides good error messages and more importantly does not provide wrong or misleading error messages like the one above. Erik -- ---------------------------------------------------------------------- Erik de Castro Lopo http://www.mega-nerd.com/

5 4

Re: HEAD doesn't build. Totally stalled.
by Ryan Scott 17 Jul '20

17 Jul '20

> And while Simon is the only > one I'm aware of, for whom this breaks For what it's worth, I am hitting the same issue when building GHC. Ryan S.

1 0

GHC formally discontinuing 32-bit Windows support?
by Ben Gamari 16 Jul '20

16 Jul '20

tl;dr. Unless someone speaks up, GHC will formally discontinue its (currently-broken) support for 32-bit Windows in 8.12. Hi everyone, As some have noticed, recent GHC releases' support for 32-bit Windows support can be generously described as "unreliable". This has been due to a combination of platform limitations, native toolchain bugs, and a general lack of capacity within the GHC community focusing on Windows support. I won't summarise the concrete issues here (see #17961, and #17700 for the current state-of-play) but let it suffice to say that we are currently stuck due to a bug in GNU binutils. However, I was recently informed that Cygwin and msys have recently discontinued their support for 32-bit Windows. While GHC uses a toolchain from the mingw32-w64 project, it seems only a matter of time before 32-bit builds cease there as well (see [1] for a summary of the relationships between these projects). Furthermore, Microsoft itself has said that 32-bit Windows 10 releases will cease later this year. All of this suggests to me that supporting 32-bit Windows in GHC will be, at best, an up-hill battle. Even worse, it is a battle with little to gained: essentially all Intel-based Windows systems today run on 64-bit-capable systems. I know of no compelling reasons why users would opt to use 32-bit Windows in 2020. Consequently, I suggest that we should formally discontinue 32-bit Windows support in GHC 8.12. In my opinion, GHC's limited engineering capacity on Windows is better spent elsewhere. However, if there are compelling reasons why some users still rely on 32-bit Windows support (despite it being largely unusable for the last two years), please do let me know. I have been consistently surprised by the number of users who have noted the absence of 32-bit Windows builds; I would love to know why they seem to be so popular. Cheers, - Ben [1] https://gitlab.haskell.org/ghc/ghc/-/wikis/surviving-windows

1 0

[ANNOUNCE] GHC 8.8.4 is now available
by Ben Gamari 15 Jul '20

15 Jul '20

Hello everyone, The GHC team is proud to announce the release of GHC 8.8.4. The source distribution, binary distributions, and documentation are available at https://downloads.haskell.org/~ghc/8.8.4 Release notes are also available [1]. This release fixes a handful of issues affecting 8.8.3: - Fixes a bug in process creation on Windows (#17926). Due to this fix we strongly encourage all Windows users to upgrade immediately. - Works around a Linux kernel bug in the implementation of timerfd (#18033) - Fixes a few linking issues affecting ARM - Fixes "missing interface file" error triggered by some uses of Data.Ord.Ordering (#18185) - Fixes an integer overflow in the compact-normal-form import implementation (#16992) - `configure` now accepts a `--enable-numa` flag to enable/disable `numactl` support on Linux. - Fixes potentially lost sharing due to the desugaring of left operator sections (#18151). - Fixes a build-system bug resulting in potential miscompilation by unregisteised compilers (#18024) As always, if anything looks amiss do let us know. Happy compiling! Cheers, - Ben [1] https://downloads.haskell.org/ghc/8.8.4/docs/html/users_guide/8.8.4-notes.h…

1 0

Review for !2377 Haddock and GHC
by Simon Peyton Jones 15 Jul '20

15 Jul '20

Fellow GHC devs I'd like to ask your help with reviewing a GHC patch !2377: Accumulate Haddock comments in P (#17544, #17561, #8944)<https://gitlab.haskell.org/ghc/ghc/-/merge_requests/2377/diffs#diff-content…> It's a substantial patch to the parser, changing the way in which Haddock documentation is parsed. It moves us in a good direction; it makes us more robust for the future; Vlad has invested a lot of effort in it; and I'm sure we should land it. But it's had surprisingly little review, especially for a patch that potentially affects quite a lot of people (incl Haddock folk, IDE folk). So, would any of you be willing to review the code itself? Especially folk involved in Haddock or Haskell IDE. Early versions were not easy to understand, but Vlad has invested a lot of effort in both simplifying it and in documenting it. We'll land it regardless in a couple of weeks. Thanks! Simon Summary of the payload (taken from #17544) Currently Haddock comments are, first and foremost, comments. It's very annoying to incorporate them into the grammar. We can take advantage of an important property: adding a Haddock comment does not change the parse tree in any way other than wrapping some nodes in HsDocTy and the like (and if it does, that's a bug). Here's the design I propose instead: 1. Accumulate Haddock comments with their locations in the P monad. This logic will be very easy to add in the lexer. 2. After parsing, do a pass over the AST to associate Haddock comments with AST nodes using location info. 3. Report the leftover comments to the user as a warning or an error. This means * The happy grammar won't even have the notion of a Haddock comment. * Parser.y becomes quite a bit simpler. * Ultimately, it is conceivable we could move the logic of associating Haddock comments with AST nodes from GHC to Haddock itself A detailed overview of the implementation plan is in comments at the start of compiler/GHC/Parser/PostProcess/Haddock.hs<https://gitlab.haskell.org/ghc/ghc/-/merge_requests/2377/diffs#diff-content…>

1 0

Querying Core for information about variables
by Siddharth Bhat 13 Jul '20

13 Jul '20

Hello, I'm trying to understand how to query information about `Var`s from a Core plugin. Consider the snippet of haskell: ``` {-# LANGUAGE MagicHash #-} import GHC.Prim fib :: Int# -> Int# fib i = case i of 0# -> i; 1# -> i; _ -> (fib i) +# (fib (i -# 1#)) main :: IO (); main = let x = fib 10# in return () ``` That compiles to the following (elided) GHC Core, dumped right after desugar: ``` Rec { fib [Occ=LoopBreaker] :: Int# -> Int# [LclId] fib = ... end Rec } Main.$trModule :: GHC.Types.Module [LclIdX] Main.$trModule = GHC.Types.Module (GHC.Types.TrNameS "main"#) (GHC.Types.TrNameS "Main"#) -- RHS size: {terms: 7, types: 3, coercions: 0, joins: 0/0} main :: IO () [LclIdX] main = case fib 10# of { __DEFAULT -> return @ IO GHC.Base.$fMonadIO @ () GHC.Tuple.() } -- RHS size: {terms: 2, types: 1, coercions: 0, joins: 0/0} :Main.main :: IO () [LclIdX] :Main.main = GHC.TopHandler.runMainIO @ () main ``` I've been using `occNameString . getOccName` to manipulate names of `Var`s from the Core module. I'm rapidly finding this insufficient, and want more information about a variable. In particular, How to I figure out: 1. When I see the Var with occurence name `fib`, that it belongs to module `Main`? 2. When I see the Var with name `main`, whether it is `Main.main` or `:Main.main`? 3. When I see the Var with name `+#`, that this is an inbuilt name? Similarly for `-#` and `()`. 4. When I see the binder $trModule, that it is added by GHC and has type `GHC.Types.Module`? 5. In general, given a Var, how do I decide where it comes from, and whether it is user-defined or something GHC defined ('wired-in' I believe is the term I am looking for)? 6. When I see a `Var`, how do I learn its type? 7. In general, is there a page that tells me how to 'query' Core/`ModGuts` from within a core plugin? Pointers on how to get this information is much appreciated. Also, pointers on "learning how to learn" --- that is, how I could have figured this out on my own / RTFMing better are also very appreciated! Thanks a lot, ~Siddharth -- https://bollu.github.io/

2 1