- ghc-devs - Haskell.org

Attempting to Create a semi-official Haskell layer for Open Embedded/Yocto
by drquigl 24 Sep '15

24 Sep '15

Hello Everyone, I had asked about cross compilers about 5 months back and other priorities have put that work on hold. However I am once again tasked with trying to get Haskell to build properly within an embedded environment we use. I found the additional pages on the Haskell build system and its architecture and everything it uses and read through all of them. I was wondering if anyone would be interested in helping me work through the problems I'm having with the build of Haskell in Open Embedded. I want to use the most up to date release of Haskell and if we need to push any changes to the build system to facilitate getting this working for everyone I'm happy to do so. I can set up a repository on github with all the appropriate sub modules to make it easy for someone to check out and start trying to build GHC. If anyone is interested in tackling this problem with me I'm happy to arrange a phone call or some other form of meeting to go over the specifics of OE and fill that person or persons in on what I hope to accomplish. If you are interested please let me know and we can try to work on getting better build support for Embedded toolchains. Dave

1 0

[RFC] Moving user's guide to ReStructuredText
by Ben Gamari 23 Sep '15

23 Sep '15

Hello all! Last year there was a brief thread [1] on this list discussing the choice of markup language used for GHC's users guide. At this point DocBook is showing signs of age, * The format's documentation is ancient and isn't terribly approachable. * Writing XML by hand is a terrible, terrible experience. For this reason a shocking fraction of the users' guide isn't even valid DocBook (although is still accepted by the tools) * The tooling surrounding the format is challenging to bring up on non-Linux platforms * Getting even a simple image displayed consistently in the PDF and HTML output is an exercise in futility [2] There are a few alternatives that we could switch to, * Markdown: While ubiquitous, its syntax isn't nearly expressive enough to accomodate the users guide. * asciidoc: This was the front-runner in [1]. Unfortunately, when I tried to use it in anger on the users guide things pretty quickly fell apart start to come apart. The syntax is sadly not very composable: tasks like nesting a code block inside a list becomes fragile and quite unreadable due to the need for continuation characters (as delimited blocks like code blocks must begin at column 0). Despite this I did manage to get much of the way through an asciidoc-ification of the users guide [2] but only through a great deal of manual fixing. While asciidoc does strive to map one-to-one onto DocBook, my experience is that the converse is not true; a conversion to asciidoc require that we drop some of the finer distinctions between code-like inline elements. For an example of the continuation character issue, see [3]. * ReStructuredText: this was a close second-place in the thread and has a fairly wide user base. The primary implementation, Sphinx, is used by Python, MathJAX, LLVM, Ubuntu, Ceph, Blender, and others. The syntax is fairly similar to Markdown and is at least as expressive as asciidoc. I have converted the entire users guide to ReStructuredText with a modified Pandoc. While some tweaking is still certainly necessary the output from the most-mechanical conversion looks quite good, * HTML (using a modified version of LLVM's theme), http://smart-cactus.org/~ben/ghc-user-manual/html/index.html * PDF produced by xetex (used to get convenient Unicode support), http://smart-cactus.org/~ben/ghc-user-manual/xetex/GHCUsersGuide.pdf * ePub (I know nothing about this format) http://smart-cactus.org/~ben/ghc-user-manual/epub/GHCUsersGuide.epub * Even Github's rendering of the source looks reasonable good, https://github.com/bgamari/ghc/blob/doc-rst/docs/users_guide/ghci.rst Of course, there are a few annoyances: the doctree construct doesn't quite work how one might expect, requiring one to split up files a bit more than one might like. Like asciidoc, there is no good way to express nested inlines, so we still lose some of the expressiveness of DocBook. Another nice advantage here is that Trac has native support [5] for rendering RST which could come in handy when pasting between documents. At this point we are leaning towards going with ReStructuredText: the tooling is much better the DocBook, the format reasonably easy to grok and expressive enough to accomodate the majority of the users guide unmodified. However, we would love to hear what others think. Are there any formats we have overlooked? Are there any objections to going ahead with this? If we want to move forward with ReStructuredText I think we will want to move quickly. While the conversion is mostly automated, there is some amount of manual fiddling necessary to get things formatted nicely. There are a few open Differentials that would need to be amended after the change but I would be happy to help authors through this if necessary. Cheers, - Ben [1] https://mail.haskell.org/pipermail/ghc-devs/2014-October/006599.html [2] https://ghc.haskell.org/trac/ghc/ticket/10416 [3] https://github.com/bgamari/ghc/blob/asciidoc/docs/users_guide/ [4] https://github.com/bgamari/ghc/blame/asciidoc/docs/users_guide/ghci.asciido… [5] http://trac.edgewall.org/wiki/WikiRestructuredText

3 3

[RFC] Moving user's guide to ReStructuredText
by harold_carr＠hotmail.com 23 Sep '15

23 Sep '15

You might consider the Racket documentation ecosystem: http://pkg-build.racket-lang.org/doc/scribble/index.html or http://pkg-build.racket-lang.org/doc/pollen/

3 2

ANN: CfN for new Haskell Prime language committee
by Herbert Valerio Riedel 23 Sep '15

23 Sep '15

Dear Haskell Community, In short, it's time to assemble a new Haskell Prime language committee. Please refer to the CfN at https://mail.haskell.org/pipermail/haskell-prime/2015-September/003936.html for more details. Cheers, hvr -- PGP fingerprint: 427C B69A AC9D 00F2 A43C AF1C BA3C BA3F FE22 B574

1 0

question about GHC API on GHC plugin
by Mike Izbicki 22 Sep '15

22 Sep '15

I'm trying to write a GHC plugin. The purpose of the plugin is to provide Haskell bindings to Herbie. Herbie (https://github.com/uwplse/herbie) is a program that takes a mathematical statement as input, and gives you a numerically stable formula to compute it as output. The plugin is supposed to automate this process for Haskell programs. I can convert the core expressions into a format for Herbie just fine. Where I'm having trouble is converting the output from Herbie back into core. Given a string that represents a numeric operator (e.g. "log" or "+"), I can get that converted into a Name that matches the Name of the version of that operator in scope at the location. But in order to create an Expr, I need to convert the Name into a Var. All the functions that I can find for this (e.g. mkGlobalVar) also require the type of the variable. But I can't find a way to figure out the Type given a Name. How can I do this?

4 20

Converting unboxed sum types in StgCmm
by Johan Tibell 22 Sep '15

22 Sep '15

Hi! The original idea for implementing the backend part of the unboxed sums proposal was to convert from the core representation to the actual data representation (i.e. a tag followed by some pointer and non-pointer fields) in the unarise stg-to-stg <https://github.com/ghc/ghc/blob/master/compiler/simplStg/UnariseStg.hs> pass. I have now realized that this won't work. The problem is that stg is too strongly typed. When we "desugar" sum types we need to convert functions receiving a value e.g. from f :: (# Bool | Char #) -> ... to f :: NonPointer {-# tag#-} -> Pointer {-# Bool or Char #-} -> ... Since stg is still typed with normal Haskell types (e.g. Bool, Char, etc), this is not possible, as we cannot represent an argument which has two different types. It seems to me that we will have to do the conversion in the stg-to-cmm <https://github.com/ghc/ghc/blob/master/compiler/codeGen/StgCmm.hs> pass, which is quite a bit more involved. For example, StgCmmEnv.idToReg function will have to change from idToReg :: DynFlags -> NonVoid Id -> LocalReg to idToReg :: DynFlags -> NonVoid Id -> [LocalReg] to accommodate the fact that we might need more than one register to store a binder. Any ideas for a better solution? -- Johan

5 12

Proposal: Associating pattern synonyms with types
by Matthew Pickering 19 Sep '15

19 Sep '15

Dear devs, There has been a bit of discussion on #10653 about extending exports to allow users to associate pattern synonyms with types (just like ordinary data constructors). There is a section on the wiki which describes the problem and the proposed behaviour. https://ghc.haskell.org/trac/ghc/wiki/PatternSynonyms#Associatingsynonymswi… Any comments would be appreciated before I start the implementation. Please read all the way to the bottom past the examples as their are a few sections of clarification which might answer your question. Matt

1 0

Haskell Error Messages
by Alan & Kim Zimmerman 18 Sep '15

18 Sep '15

Is there currently any planned work around making the haskell error messages able to support something like the ones in IDRIS, as shown in David Christianson's talk "A Pretty printer that says what it means" at HIW? https://www.youtube.com/watch?v=m7BBCcIDXSg&list=PLnqUlCo055hVfNkQHP7z43r10… Alan

4 5

Renaming InstalledPackageId
by Edward Z. Yang 18 Sep '15

18 Sep '15

Hello friends, During discussions with many people about Nix-like Cabal, it has emerged that InstalledPackageId is /really/ bad name. Consider: the commonly accepted definition of an InstalledPackageId in Nix is that it is morally a hash of all the inputs to compilation: the source code, the dependencies of the package, and the build configuration. However, a Cabal package can have *multiple* components (e.g. the library, the test suite, etc), each of which has its own 'build-depends' field. The concept of the "dependencies of a package" is simply not well-defined! The "simplification" that Cabal has adopted for a long time is to say that the installed package ID always refers to the library component of a package. [1] But the name InstalledPackageId has caused countless misunderstandings about how dependency resolution is done, even in Cabal's code. [2] I propose that we change the name of InstalledPackageId. The new name should have the following properties: 1. It should not say anything about packages, because it is not well-defined for a package, e.g. it should be something like "ComponentId". 2. It should not say anything about installation, because it is well-defined even before a package is even built. 3. It should some how communicate that it is a hash of the transitive source code (e.g. including dependencies) as well as build parameters. SPJ likes "SourceHash" because it's evocative of this (though I don't like it as much); there may also be a Nix-y term like "Derivation" we can use here. My proposed new name is "ComponentBuildHash", however I am open to other suggestions. I might also be convinced by "InstalledComponentId" (which runs aground (2) but is fairly similar to the old name, and gains points for familiarity.) However, I would like to hear your comments: have a better name? Think this is unnecessary? Please let me know. Edward P.S. With Backpack, the ComponentBuildHash won't even be the primary key into the installed package (to be renamed to a component/unit) database, because a single ComponentBuildHash can be rebuilt multiple times with different instantiations of its holes. So GHC will have some identifier, which we will probably continue to call the 'UnitKey', which is the ComponentBuildHash (entirely Cabal generated) plus extra information about how holes are instantiated (entirely GHC generated). [1] Except when it doesn't: cabal-install currently merges all the dependencies of all components that are being built for a package together and treats that as the sum total dependencies of the package. This causes problems when the test suite for text depends on a testing library which in turn depends on text, c.f. https://github.com/haskell/cabal/issues/960 [2] Here are some bugs caused by confusion of package dependencies versus component dependency: https://github.com/haskell/cabal/issues/2802 Specify components when configuring, not building https://github.com/haskell/cabal/issues/2623 `-j` should build package components in parallel https://github.com/haskell/cabal/issues/1893 Use per-component cabal_macros.h https://github.com/haskell/cabal/issues/1575 Do dependency resolution on a per component basis https://github.com/haskell/cabal/issues/1768 The "benchmark" target dependencies conflict with "executable" targets https://github.com/haskell/cabal/issues/960 Can't build with --enable-tests in presence of circular dependencies

2 1

Making compilation results deterministic (#4012)
by Bartosz Nitka 18 Sep '15

18 Sep '15

Hello, For the past couple of weeks I've been working on making compilation results deterministic. What I'm focusing on right now is the interface file determinism, I don't care about binaries being deterministic. I'd like to give a status update and ask for some advice, since I'm running into issues that I don't have a good way of solving. The first question anyone might ask is how did nondeterminism creep into the compiler. If we're compiling with a single thread there's no reason for the computation to proceed in non deterministic way. I'm fairly certain that the issue originates from lazy loading of interface files. Relevant function: https://phabricator.haskell.org/diffusion/GHC/browse/master/compiler/typech…. What happens is that if you already have an interface file for a target you're trying to build the computation will proceed differently. Why does lazy loading matter? As you load the interface file it needs to get type-checked and that means it needs to pull some Uniques from a global UniqSupply. It does that in different order resulting in different Unique assignment. As far as I can tell, lazy loading is required for performance, so I abandoned the idea of fixing it. I haven't looked at parallel compilation yet, but I'd expect it to result in different Unique assignment as well. I believe up to this point we're ok. Uniques are non-deterministic, but it shouldn't be a big deal. Uniques should be opaque enough to not affect the order of computation, for example the order of binds considered. But they aren't. Uniques are used in different ways throughout the compiler and they end up reordering things: 1) They have an `Ord` instance: https://phabricator.haskell.org/diffusion/GHC/browse/master/compiler/basicT… . So far the places it impacts the most are places that use `stronglyConnCompFromEdgedVertices`, because Unique is used as a Node key and the result depends on the order of Nodes being considered. Some examples: https://phabricator.haskell.org/diffusion/GHC/browse/master/compiler/simplC… https://phabricator.haskell.org/diffusion/GHC/browse/master/compiler/rename… (because Ord for Name uses Unique https://phabricator.haskell.org/diffusion/GHC/browse/master/compiler/basicT… ) I've tried to see what removing it would entail and the changes would be far reaching: https://phabricator.haskell.org/P62. 2) VarEnv, NameEnv are implemented in terms of UniqFM, which is just Data.IntMap with keys being the Unique integer values. The way this bites us is that when UniqFM's get converted to a list they end up being sorted on Unique value. This problem is more widespread than the `stronglyConnCompFromEdgedVertices` issue, there's even a place where it's implicitly depended on: https://phabricator.haskell.org/diffusion/GHC/browse/master/compiler/native… . I've tried to fix it by making `toList` return the elements in the order of insertion (https://phabricator.haskell.org/P63) but that turned out to have significant cost. My unscientific benchmark on aeson and text showed 10% compilation time increase. It's possible it can be done in less expensive way, I've tried a couple of approaches, but all of them resulted in 10% time increase. I've also considered to split UniqFM to two different types, one that keeps the ordering, and one that can't `toList`, but I suspect that the cut will not be clean, so I haven't tried that. In some cases we got away with ordering things by OccName where needed: ( https://phabricator.haskell.org/D1073, https://phabricator.haskell.org/D1192) but OccName's don't have to be unique in every case and if we try to make them unique that would make them longer and probably result in even greater slowdown. The instance I've recently looked at doesn't look like it can be solved by sorting by OccName. The code that triggers the problem (simplified from haskell-src-exts): data Decl l = Boring l deriving (Eq) data Binds l = BDecls l [Decl l] -- ^ An ordinary binding group | IPBinds l [IPBind l] -- ^ A binding group for implicit parameters deriving (Eq) data IPBind l = Boring2 l deriving (Eq) The end result is: 4449fe3f8368a2c47b2499a1fb033b6a $fEqBinds_$c==$Binds :: Eq l => Binds l -> Binds l -> Bool {- Arity: 1, HasNoCafRefs, Strictness: <L,U(C(C1(U)),A)>, Unfolding: (\ @ l $dEq :: Eq l -> let { $dEq1 :: Eq (Decl l) = $fEqDecl @ l $dEq } in let { $dEq2 :: Eq (IPBind l) = $fEqIPBind @ l $dEq } in \ ds :: Binds l ds1 :: Binds l -> case ds of wild { BDecls a1 a2 -> case ds1 of wild1 { BDecls b1 b2 -> case == @ l $dEq a1 b1 of wild2 { False -> False True -> $fEq[]_$c== @ (Decl l) $dEq1 a2 b2 } IPBinds ipv ipv1 -> False } IPBinds a1 a2 -> case ds1 of wild1 { BDecls ipv ipv1 -> False IPBinds b1 b2 -> case == @ l $dEq a1 b1 of wild2 { False -> False True -> $fEq[]_$c== @ (IPBind l) $dEq2 a2 b2 } } }) -} vs bb525bf8c0145a5379b3c29e8adb4b18 $fEqBinds_$c==$Binds :: Eq l => Binds l -> Binds l -> Bool {- Arity: 1, HasNoCafRefs, Strictness: <L,U(C(C1(U)),A)>, Unfolding: (\ @ l $dEq :: Eq l -> let { $dEq1 :: Eq (IPBind l) = $fEqIPBind @ l $dEq } in let { $dEq2 :: Eq (Decl l) = $fEqDecl @ l $dEq } in \ ds :: Binds l ds1 :: Binds l -> case ds of wild { BDecls a1 a2 -> case ds1 of wild1 { BDecls b1 b2 -> case == @ l $dEq a1 b1 of wild2 { False -> False True -> $fEq[]_$c== @ (Decl l) $dEq2 a2 b2 } IPBinds ipv ipv1 -> False } IPBinds a1 a2 -> case ds1 of wild1 { BDecls ipv ipv1 -> False IPBinds b1 b2 -> case == @ l $dEq a1 b1 of wild2 { False -> False True -> $fEq[]_$c== @ (IPBind l) $dEq1 a2 b2 } } }) -} This happens because when desugaring dictionaries we do an SCC on Uniques that ends up reordering lets ( https://phabricator.haskell.org/diffusion/GHC/browse/master/compiler/deSuga… ) Now all of the dictionaries have OccName of Eq. I could probably reach deeper into the term to extract enough information (in this instance, the constructor name) to get deterministic ordering, but this feels very ad-hoc and I don't expect it to scale to the whole codebase. Another problem with fixing things in this ad-hoc manner is keeping them fixed. There's nothing preventing people from introducing nondeterminism. One idea that makes it more testable is to test it with different UniqSupply allocation patterns. I've found it useful to compare against UniqSupply that starts at a big number and allocates in decreasing order. Gray codes could be used to generate non-sequential order. The reason I'm posting this is to get some ideas, because at this point I feel stuck, I don't see a good way of achieving the end goal. I hope someone with more intimate GHC knowledge can point out a wrong assumption I've made or suggest an approach I haven't thought of. Cheers, Bartosz

5 8