June 2018 - ghc-devs - Haskell.org

Should we have primitive fill-once variables?
by David Feuer 01 Jul '18

01 Jul '18

IVars (write-once variables) can be useful for various purposes. In Haskell, they can be implemented using MVars. But MVars are not the lightest things in the world. I'm wondering if it might pay to implement something much lighter primitively. Here's a sketch of some things I'll tentatively call QVars. A QVar has two fields: a value (possibly null) and a stack (no need for a queue) of waiting threads. newEmptyQVar: Create a QVar with a null value and an empty queue. tryReadQVar: Just look at the value. readQVar: Check if the value is null (simple memory read). If not, use it. If so, push yourself onto the waiting stack (CAS loop). The code that will run when you're awakened will try to awaken the next thread if there is one (CAS loop). putQVar: Install a new value and get the old one (exchange). If the old value was null, mark the QVar dirty. Awaken the first thread if there is one (CAS loop). Return the old value if it was non-null (this can be used in library code to make duplicate writes, or non-equal duplicate writes, an error). I think we'd probably also want atomic modification operations, but I haven't figured out which ones yet. Implementation differences from MVars: * We have two fields instead of three (because we can get away with a stack instead of a queue). * We never need to lock the QVar closure. MVars do: they can change freely between empty and full, so it's necessary to coordinate between value and queue modifications.

5 11

Re: Plan for GHC 8.6.1
by Ben Gamari 29 Jun '18

29 Jun '18

Moritz Angermann <moritz(a)lichtzwerge.com> writes: > Hi, > > I would almost go as far as saying 8.6 will work with LLVM4-6. GHC llvm codegen > is really only dependent on the textual IR, and even there only on the parts we > use. This used to be an issue, where LLVMs textual IR changed quite a bit, but > it looks like it hasn't for the last few releases. > > This of course does not insulate us from bugs in LLVM itself, which might or > might not affect us. > > Maybe we can be a bit more lenient with respect to LLVM versions? > I would really prefer not to be. We put in place the current policy for a few good reasons: not only was keeping up with the syntactic changes in a compatible way tiresome, but various sets of LLVM bugs meant significantly more work during ticket triage. While the relative syntactic stability of the last few LLVM releases may reduce the relevance of the first reason, the second reason holds just as well today as it did two years ago. Cheers, - Ben

3 3

Harbormaster: Build failure on OS/X build. How to fix it as a non Mac user
by Roland Senn 29 Jun '18

29 Jun '18

Hi all, Phabricator informed me, that the builds on Harbourmaster for my changes failed. Looking at the log at https://phabricator.haskell.org/B21368 I can see, that the builds on Linux and Windows succeeded, however the tests on OS/X failed. Looking at the failed tests at https://phabricator.haskell.org/harborma ster/build/48354/ it tells me that the 2 tests TEST="T5631 T6048" failed. Now, I don't own a machine with OS/X. How can I look, what really failed, and how can I try to fix it? Many thanks Roland

3 2

RE: GHC.Prim.Int# is not at TyThing?
by Simon Peyton Jones 29 Jun '18

29 Jun '18

Well what is lexically in scope is, well, whatever should be lexically in scope at that point. Yes, I suppose your lexicial environment might have changed, but I can’t speculate as to why. Starting with Strings makes you vulnerable to this. Starting with an “Orig” RdrName would be more robust. Simon From: Ranjit Jhala <jhala(a)cs.ucsd.edu> Sent: 29 June 2018 15:26 To: Simon Peyton Jones <simonpj(a)microsoft.com> Cc: ghc-devs(a)haskell.org Subject: Re: GHC.Prim.Int# is not at TyThing? Dear Simon, Yes I expect that the notion of what is in scope in the GlobalRdrEnv has changed across the GHC versions? Earlier, these lookups would succeed but now (likely due to some artifact of how we are using the API) they fail. You are right that we can get Int# from the wiredInTyCons; but the issue arises with other names (eg “Fractional”) Which are not (?) wiredIn. The silver lining is this may force us to redo the LH name resolution entirely in a “lazy” fashion only using those Var and Type That actually appear in the core being analyzed (as opposed to the current “eager” fashion where we use the hscenv to lookup all names in the LH prelude...) Thanks! Ranjit. On Fri, Jun 29, 2018 at 12:21 AM Simon Peyton Jones <simonpj(a)microsoft.com<mailto:simonpj@microsoft.com>> wrote: Are you sure that Int# is in (lexical scope) in the GlobalRdrEnv of the HscEnv? If not, looking up the String in the lexicial environment will fail. You can always just grab TysPrim.intPrimTyCon. Simon From: Ranjit Jhala <jhala(a)cs.ucsd.edu<mailto:jhala@cs.ucsd.edu>> Sent: 29 June 2018 00:55 To: Simon Peyton Jones <simonpj(a)microsoft.com<mailto:simonpj@microsoft.com>> Cc: ghc-devs(a)haskell.org<mailto:ghc-devs@haskell.org> Subject: Re: GHC.Prim.Int#<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> is not at TyThing? Dear Simon (and all), Thanks! Then it seems my problem is much worse: somehow the code I had that used the `HscEnv` to "resolve" names (i.e. get `Name` and then `TyThing` and then `TyCon`) from plain strings is no longer working with the GHC 8.4.3. My efforts to distill the relevant LH code into a simple test that shows the difference between the two versions (GHC 8.2 and 8.4) have proven fruitless so far. Can anyone point me to a (small?) example of using the GHC API that implements something like: lookupVarType :: String -> IO String which takes a `String` corresponding to the name of a top-level binder as input, and returns a `String` containing the TYPE of the binder as output? Thanks! - Ranjit. On Thu, Jun 28, 2018 at 3:48 AM Simon Peyton Jones <simonpj(a)microsoft.com<mailto:simonpj@microsoft.com>> wrote: Does the above indicate that in fact, `GHC.Prim.Int#`<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> DOES NOT (any longer) correspond to a `TyCon`? No, it does not indicate that! There still is a TyCon for Int#. Indeed you can see it defined in TysPrim.intPrimTyCon. Simon From: Ranjit Jhala <jhala(a)cs.ucsd.edu<mailto:jhala@cs.ucsd.edu>> Sent: 28 June 2018 06:08 To: ghc-devs(a)haskell.org<mailto:ghc-devs@haskell.org>; Simon Peyton Jones <simonpj(a)microsoft.com<mailto:simonpj@microsoft.com>> Subject: GHC.Prim.Int#<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> is not at TyThing? Hi all, I am trying to update LiquidHaskell to GHC 8.4. In doing so, I find that I can no longer resolve (i.e. get the `TyThing`, and hence `TyCon`) corresponding to the name: GHC.Prim.Int#<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> It seems like in older versions, we had λ> :i GHC.Prim.Int#<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> data GHC.Prim.Int#<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> -- Defined in ‘GHC.Prim’ λ> :k GHC.Prim.Int#<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> GHC.Prim.Int#<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> :: # but in GHC 8.4 this is changed so: λ> :i GHC.Prim.Int#<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> data GHC.Prim.Int#<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> :: TYPE 'GHC.Types.IntRep -- Defined in ‘GHC.Prim’ λ> :k GHC.Prim.Int#<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> GHC.Prim.Int#<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> :: TYPE 'GHC.Types.IntRep Does the above indicate that in fact, `GHC.Prim.Int#`<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> DOES NOT (any longer) correspond to a `TyCon`? If so, what does it correspond to? i.e. how is it represented as a `Type` in GHC? Any pointers would be most appreciated! Thanks! - Ranjit.

1 0

RE: GHC.Prim.Int# is not at TyThing?
by Simon Peyton Jones 29 Jun '18

29 Jun '18

Are you sure that Int# is in (lexical scope) in the GlobalRdrEnv of the HscEnv? If not, looking up the String in the lexicial environment will fail. You can always just grab TysPrim.intPrimTyCon. Simon From: Ranjit Jhala <jhala(a)cs.ucsd.edu> Sent: 29 June 2018 00:55 To: Simon Peyton Jones <simonpj(a)microsoft.com> Cc: ghc-devs(a)haskell.org Subject: Re: GHC.Prim.Int# is not at TyThing? Dear Simon (and all), Thanks! Then it seems my problem is much worse: somehow the code I had that used the `HscEnv` to "resolve" names (i.e. get `Name` and then `TyThing` and then `TyCon`) from plain strings is no longer working with the GHC 8.4.3. My efforts to distill the relevant LH code into a simple test that shows the difference between the two versions (GHC 8.2 and 8.4) have proven fruitless so far. Can anyone point me to a (small?) example of using the GHC API that implements something like: lookupVarType :: String -> IO String which takes a `String` corresponding to the name of a top-level binder as input, and returns a `String` containing the TYPE of the binder as output? Thanks! - Ranjit. On Thu, Jun 28, 2018 at 3:48 AM Simon Peyton Jones <simonpj(a)microsoft.com<mailto:simonpj@microsoft.com>> wrote: Does the above indicate that in fact, `GHC.Prim.Int#`<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> DOES NOT (any longer) correspond to a `TyCon`? No, it does not indicate that! There still is a TyCon for Int#. Indeed you can see it defined in TysPrim.intPrimTyCon. Simon From: Ranjit Jhala <jhala(a)cs.ucsd.edu<mailto:jhala@cs.ucsd.edu>> Sent: 28 June 2018 06:08 To: ghc-devs(a)haskell.org<mailto:ghc-devs@haskell.org>; Simon Peyton Jones <simonpj(a)microsoft.com<mailto:simonpj@microsoft.com>> Subject: GHC.Prim.Int#<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> is not at TyThing? Hi all, I am trying to update LiquidHaskell to GHC 8.4. In doing so, I find that I can no longer resolve (i.e. get the `TyThing`, and hence `TyCon`) corresponding to the name: GHC.Prim.Int#<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> It seems like in older versions, we had λ> :i GHC.Prim.Int#<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> data GHC.Prim.Int#<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> -- Defined in ‘GHC.Prim’ λ> :k GHC.Prim.Int#<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> GHC.Prim.Int#<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> :: # but in GHC 8.4 this is changed so: λ> :i GHC.Prim.Int#<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> data GHC.Prim.Int#<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> :: TYPE 'GHC.Types.IntRep -- Defined in ‘GHC.Prim’ λ> :k GHC.Prim.Int#<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> GHC.Prim.Int#<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> :: TYPE 'GHC.Types.IntRep Does the above indicate that in fact, `GHC.Prim.Int#`<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> DOES NOT (any longer) correspond to a `TyCon`? If so, what does it correspond to? i.e. how is it represented as a `Type` in GHC? Any pointers would be most appreciated! Thanks! - Ranjit.

1 0

How do you build base with your own compiler?
by Christopher Done 28 Jun '18

28 Jun '18

I've built the GHC compiler along with libraries/base in the canonical way. Now, I want to compile base with my own compiler frontend which will do some analysis. Here's what I've done so far: 1) I've compiled my frontend with the ghc-stage2 compiler and registered it. That works. 2) I've found the right GHC invocation which looks like this: "inplace/bin/ghc-stage1" -hisuf hi -osuf o -hcsuf hc -static -O0 -H64m -Wall -this-unit-id base-4.9.1.0 -hide-all-packages -i -ilibraries/base/. -ilibraries/base/dist-install/build -ilibraries/base/dist-install/build/autogen -Ilibraries/base/dist-install/build -Ilibraries/base/dist-install/build/autogen -Ilibraries/base/include -optP-DOPTIMISE_INTEGER_GCD_LCM -optP-include -optPlibraries/base/dist-install/build/autogen/cabal_macros.h -package-id ghc-prim-0.5.0.0 -package-id integer-gmp-1.0.0.1 -package-id rts -this-unit-id base -XHaskell2010 -O0 -no-user-package-db -rtsopts -Wno-trustworthy-safe -Wno-deprecated-flags -Wnoncanonical-monad-instances -odir libraries/base/dist-install/build -hidir libraries/base/dist-install/build -stubdir libraries/base/dist-install/build -dynamic-too $hsfile So I (1) changed stage1 to stage2, so that I could use plugins, (2) simply added --frontend GhcFrontendPlugin -package frontend and ran that on every file under base/ in a loop: https://gist.github.com/chrisdone/3ca64592aed2053606d8814f2fa5d772 That seems to work. I basically have what I wanted. But I'd rather be able to invoke make with e.g. GHC_COMPILER=inplace/ghc/stage2 and EXTRA_HC_OPTS=" --frontend GhcFrontendPlugin -package frontend". Is there an easy flag to do that? If not, can someone point me where in the makefile I could tweak this? Cheers

3 3

RE: GHC.Prim.Int# is not at TyThing?
by Simon Peyton Jones 28 Jun '18

28 Jun '18

Does the above indicate that in fact, `GHC.Prim.Int#`<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> DOES NOT (any longer) correspond to a `TyCon`? No, it does not indicate that! There still is a TyCon for Int#. Indeed you can see it defined in TysPrim.intPrimTyCon. Simon From: Ranjit Jhala <jhala(a)cs.ucsd.edu> Sent: 28 June 2018 06:08 To: ghc-devs(a)haskell.org; Simon Peyton Jones <simonpj(a)microsoft.com> Subject: GHC.Prim.Int# is not at TyThing? Hi all, I am trying to update LiquidHaskell to GHC 8.4. In doing so, I find that I can no longer resolve (i.e. get the `TyThing`, and hence `TyCon`) corresponding to the name: GHC.Prim.Int#<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> It seems like in older versions, we had λ> :i GHC.Prim.Int#<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> data GHC.Prim.Int#<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> -- Defined in ‘GHC.Prim’ λ> :k GHC.Prim.Int#<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> GHC.Prim.Int#<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> :: # but in GHC 8.4 this is changed so: λ> :i GHC.Prim.Int#<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> data GHC.Prim.Int#<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> :: TYPE 'GHC.Types.IntRep -- Defined in ‘GHC.Prim’ λ> :k GHC.Prim.Int#<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> GHC.Prim.Int#<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> :: TYPE 'GHC.Types.IntRep Does the above indicate that in fact, `GHC.Prim.Int#`<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2FGHC.Prim.In…> DOES NOT (any longer) correspond to a `TyCon`? If so, what does it correspond to? i.e. how is it represented as a `Type` in GHC? Any pointers would be most appreciated! Thanks! - Ranjit.

1 0

How do I add ghc-prim as a dep for ghc?
by Ömer Sinan Ağacan 27 Jun '18

27 Jun '18

I'm trying to add ghc-prim as a dependency to the ghc package. So far I've done these changes: diff --git a/compiler/ghc.cabal.in b/compiler/ghc.cabal.in index 01628dcad1..b9c3b3d02b 100644 --- a/compiler/ghc.cabal.in +++ b/compiler/ghc.cabal.in @@ -65,7 +65,8 @@ Library ghc-boot == @ProjectVersionMunged@, ghc-boot-th == @ProjectVersionMunged@, ghc-heap == @ProjectVersionMunged@, - ghci == @ProjectVersionMunged@ + ghci == @ProjectVersionMunged@, + ghc-prim if os(windows) Build-Depends: Win32 >= 2.3 && < 2.7 diff --git a/ghc.mk b/ghc.mk index c0b99c00f4..26c6e86c02 100644 --- a/ghc.mk +++ b/ghc.mk @@ -420,7 +420,8 @@ else # CLEANING # programs such as GHC and ghc-pkg, that we do not assume the stage0 # compiler already has installed (or up-to-date enough). -PACKAGES_STAGE0 = binary text transformers mtl parsec Cabal/Cabal hpc ghc-boot-th ghc-boot template-haskell ghc-heap ghci +PACKAGES_STAGE0 = binary text transformers mtl parsec Cabal/Cabal hpc \ + ghc-boot-th ghc-boot template-haskell ghc-heap ghci ghc-prim ifeq "$(Windows_Host)" "NO" PACKAGES_STAGE0 += terminfo endif But I'm getting this error: ghc-cabal: Encountered missing dependencies: ghc-prim ==0.5.3 Any ideas what else to edit? Thanks, Ömer

2 3

Update on HIE Files
by Zubin Duggal 26 Jun '18

26 Jun '18

Hello all, I've been working on the HIE File ( https://ghc.haskell.org/trac/ghc/wiki/HIEFiles) GSOC project, The design of the data structure as well as the traversal of GHCs ASTs to collect all the relevant info is mostly complete. We traverse the Renamed and Typechecked AST to collect the following info about each SrcSpan 1) Its type, if it corresponds to a binding, pattern or expression 2) Details about any tokens in the original source corresponding to this span(keywords, symbols, etc.) 3) The set of Constructor/Type pairs that correspond to this span in the GHC AST 4) Details about all the identifiers that occur at this SrcSpan For each occurrence of an identifier(Name or ModuleName), we store its type(if it has one), and classify it as one of the following based on how it occurs: 1) Use 2) Import/Export 3) Pattern Binding, along with the scope of the binding, and the span of the entire binding location(including the RHS) if it occurs as part of a top level declaration, do binding or let/where binding 4) Value Binding, along with whether it is an instance binding or not, its scope, and the span of its entire binding site, including the RHS 5) Type Declaration (class or regular) (foo :: ...) 6) Declaration(class, type, instance, data, type family etc.) 7) Type variable binding, along with its scope(which takes into account ScopedTypeVariables) I have updated the wiki page with more details about the Scopes associated with bindings: https://ghc.haskell.org/trac/ghc/wiki/HIEFiles#Scopeinformationaboutsymbols These annotated SrcSpans are then arranged into a interval/rose tree to aid lookups. We assume that no SrcSpans ever partially overlap, for any two SrcSpans that occur in the Renamed/Typechecked ASTs, either they are equal, disjoint, or strictly contained in each other. This assumption has mostly held out so far while testing on the entire ghc:HEAD tree, other than one case where the typechecker strips out parenthesis in the original source, which has been patched(see https://ghc.haskell.org/trac/ghc/ticket/15242) I have also written functions that lookup the binding site(including RHS) and scope of an identifier from the tree. Testing these functions on the ghc:HEAD tree, it succeeds in looking up scopes for almost all symbol occurrences in all source files, and I've also verified that the calculated scope always contains all the occurrences of the symbol. The few cases where this check fails is where the SrcSpans have been mangled by CPP(see https://ghc.haskell.org/trac/ghc/ticket/15279) The code for this currently lives here: https://github.com/haskell/haddock/compare/ghc-head...wz1000:hiefile-2 Moving forward, the plan for the rest of the summer is 1) Move this into the GHC tree and add a flag that controls generating this 2) Write serializers and deserializers for this info 3) Teach the GHC PackageDb about .hie files 4) Rewrite haddocks --hyperlinked-source to use .hie files.

3 3

Tracking down instances from use-sites
by Christopher Done 26 Jun '18

26 Jun '18

Hi all, Given a TypecheckedModule, what's the most direct way given a Var expression retrieved from the AST, to determine: 1) that it's a class method e.g. `read` 2) that it's a generic call (no instance chosen) e.g. `Read a => a -> String` 3) or if it's a resolved instance, then which instance is it and which package, module and declaration is that defined in? Starting with this file that has a TypecheckedModule in it: https://gist.github.com/chrisdone/6fcb9f1cba6324148d481fcd4eab6af6#file-ghc… I presume at this point that instance resolution has taken place. I'm not sure that dictionaries or chosen instances are inserted into the AST, or whether just the resolved types are inserted e.g. `Int -> String`, where I want e.g. `Read Int`, which might lead me to finding the matching instance from an InstEnv or so. I'd like to do some analyses of Haskell codebases, and the fact that calls to class methods are opaque is a bit of a road-blocker. Any handy tips? Prior work? It'd be neat in tooling to just hit a goto-definition key on `read` and be taken to the instance implementation rather than the class definition. Also, listing all functions that use throw# or functions defined in terms of throw# or FFI calls would be helpful, especially for doing audits. If I could immediately list all partial functions in a project, then list all call-sites, it would be a very convenient way when doing an audit to see whether partial functions (such as head) are used with the proper preconditions or not. Any tips appreciated, Chris

3 5