- ghc-devs - Haskell.org

Trac
by Simon Peyton Jones 28 Jul '14

28 Jul '14

Trac is taking minutes to display a page. Also spam is still coming Simon

2 1

RE: [commit: ghc] master: Module reexports, fixing #8407. (7f5c1086)
by Simon Peyton Jones 28 Jul '14

28 Jul '14

Edward Great stuff. Is this documented somewhere, notably in http://www.haskell.org/ghc/docs/latest/html/users_guide/packages.html for GHC, and somewhere in Cabal? And perhaps somewhere on the wiki https://ghc.haskell.org/trac/ghc/wiki/Commentary/Packages Thanks Simon | -----Original Message----- | From: ghc-commits [mailto:ghc-commits-bounces@haskell.org] On Behalf Of | git(a)git.haskell.org | Sent: 26 July 2014 02:08 | To: ghc-commits(a)haskell.org | Subject: [commit: ghc] master: Module reexports, fixing #8407. | (7f5c1086) | | Repository : ssh://git@git.haskell.org/ghc | | On branch : master | Link : | http://ghc.haskell.org/trac/ghc/changeset/7f5c10864e7c26b90c7ff4ed09d00 | c8a09aa4349/ghc | | >--------------------------------------------------------------- | | commit 7f5c10864e7c26b90c7ff4ed09d00c8a09aa4349 | Author: Edward Z. Yang <ezyang(a)cs.stanford.edu> | Date: Fri Jul 4 17:01:08 2014 +0100 | | Module reexports, fixing #8407. | | The general approach is to add a new field to the package database, | reexported-modules, which considered by the module finder as | possible | module declarations. Unlike declaring stub module files, multiple | reexports of the same physical package at the same name do not | result in an ambiguous import. | | Has submodule updates for Cabal and haddock. | | NB: When a reexport renames a module, that renaming is *not* | accessible | from inside the package. This is not so much a deliberate design | choice | as for implementation expediency (reexport resolution happens only | when | a package is in the package database.) | | TODO: Error handling when there are duplicate reexports/etc is not | very | well tested. | | Signed-off-by: Edward Z. Yang <ezyang(a)cs.stanford.edu> | | Conflicts: | compiler/main/HscTypes.lhs | testsuite/.gitignore | utils/haddock | | | >--------------------------------------------------------------- | | 7f5c10864e7c26b90c7ff4ed09d00c8a09aa4349 | compiler/main/DynFlags.hs | 1 + | compiler/main/Finder.lhs | 25 +++-- | compiler/main/GHC.hs | 12 ++- | compiler/main/HscTypes.lhs | 6 +- | compiler/main/PackageConfig.hs | 4 + | compiler/main/Packages.lhs | 109 | ++++++++++++++++----- | ghc/InteractiveUI.hs | 8 +- | libraries/Cabal | 2 +- | .../Distribution/InstalledPackageInfo/Binary.hs | 8 ++ | testsuite/.gitignore | 8 ++ | testsuite/tests/cabal/Makefile | 15 +++ | testsuite/tests/cabal/all.T | 6 ++ | testsuite/tests/cabal/cabal05/Makefile | 69 +++++++++++++ | .../{driver/T3007/A => cabal/cabal05}/Setup.hs | 0 | testsuite/tests/cabal/{cabal03 => cabal05}/all.T | 4 +- | .../tests/cabal/cabal05/p/LICENSE | 0 | testsuite/tests/cabal/cabal05/p/P.hs | 3 + | testsuite/tests/cabal/cabal05/p/P2.hs | 1 + | .../{driver/T3007/A => cabal/cabal05/p}/Setup.hs | 0 | testsuite/tests/cabal/cabal05/p/p.cabal | 11 +++ | .../tests/cabal/cabal05/q/LICENSE | 0 | testsuite/tests/cabal/cabal05/q/Q.hs | 4 + | .../{driver/T3007/A => cabal/cabal05/q}/Setup.hs | 0 | testsuite/tests/cabal/cabal05/q/q.cabal | 29 ++++++ | .../tests/cabal/cabal05/r/LICENSE | 0 | testsuite/tests/cabal/cabal05/r/R.hs | 11 +++ | .../{driver/T3007/A => cabal/cabal05/r}/Setup.hs | 0 | testsuite/tests/cabal/cabal05/r/r.cabal | 32 ++++++ | .../tests/cabal/cabal05/s/LICENSE | 0 | testsuite/tests/cabal/cabal05/s/S.hs | 18 ++++ | .../{driver/T3007/A => cabal/cabal05/s}/Setup.hs | 0 | testsuite/tests/cabal/cabal05/s/s.cabal | 11 +++ | testsuite/tests/cabal/ghcpkg07.stdout | 11 +++ | .../{test4.pkg => recache_reexport_db/a.conf} | 20 ++-- | testsuite/tests/cabal/{test4.pkg => test7a.pkg} | 20 ++-- | testsuite/tests/cabal/test7b.pkg | 17 ++++ | utils/ghc-cabal/ghc-cabal.cabal | 3 +- | utils/ghc-pkg/Main.hs | 55 ++++++++++- | utils/ghc-pkg/ghc-pkg.cabal | 4 +- | utils/ghctags/ghctags.cabal | 3 +- | utils/haddock | 2 +- | 41 files changed, 453 insertions(+), 79 deletions(-) | | Diff suppressed because of size. To see it, use: | | git diff-tree --root --patch-with-stat --no-color --find-copies- | harder --ignore-space-at-eol --cc | 7f5c10864e7c26b90c7ff4ed09d00c8a09aa4349 | _______________________________________________ | ghc-commits mailing list | ghc-commits(a)haskell.org | http://www.haskell.org/mailman/listinfo/ghc-commits

2 2

Understanding core2core optimisation pipeline
by Jan Stolarek 28 Jul '14

28 Jul '14

Devs, I'm trying to understand how the core2core pipeline works. Sadly, we don't have a wiki page about this so the "only" source of information are the papers and the source code. Papers give pretty much detail about each transformation in separate but none of the papers gives a comprehensive and up-to-date overview of how the whole pipeline is structured. My questions are based on reading the user documentation and the following papers: [1] - "The Glasgow Haskell Compiler" from The Architecture of Open Source Application, vol. 2 [2] - "Compilation by Transformation in Non-Strict Functional Languages", PhD by Santos [3] - "Secrets of the Glasgow Haskell Compiler inliner" [4] - "A transformation-based optimiser for Haskell" [5] - "Modular, Higher-Order Cardinality Analysis in Theory and Practice" [6] - "Let-floatig: moving bindings to give faster programs" [7] - "Playing by the Rules: Rewriting as a practical optimisation technique in GHC" I know there are several papers missing from this list, eg. "Constructed Product Result Analysis for Haskell" or "Call-pattern specialisation for Haskell programs". The reason is that these optimisations are beyond the scope of what I'm doing at the moment (or so I believe). This mail basically asks just one question: what is the order of optimizations pefromed on Core? Since this question is pretty big and general I've separated it into smaller questions that arose from reading the above papers, documentation, and experimenting with GHC. Now the detailed questions: 1. What is the difference between a "simplifier iteration" and "simplifier phase"? Section 7.20.6.5 of the user guide mentions phases but I believe that iterations are not explained anywhere. My best guess, expressed in pseudo-code, is this (sorry about the imperative style): foreach (i in iterations) { // some optimisations here? foreach (p in phases) { //...optimisations here } // some optimisations here? } 1a. What is the default maximum iterations count? User documentation does not specify that. 2. How can I observe the effects of `-ddump-simpl-phases`. I tried compiling several different programs and this flag seems to have no effect (ie. nothing gets printed). 3. Cardinality anlaysis and inlining: cardinality analysis can determine that a let binding is used exactly once. Can the inliner re-use this information from the cardinality analysis or does it recompute it per [3], section 3.1? 4. I've compiled a sample program using `-dverbose-core2core` and got the following phases: - Desugar (after optimization) - Simplifier (Phase = InitialPhase [Gentle]) - Specialise - Levels added - Float out - Float inwards - Simplifier (Phase = 2 [main]) - Simplifier (Phase = 1 [main]) - Simplifier (Phase = 0 [main]) - Demand analysis - Worker Wrapper binds - Simplifier (Phase = 0 [post-worker-wrapper]) - Levels added - Float out - Common sub-expression - Float inwards - Simplifier (Phase = 0 [final]) - Tidy Core - CorePrep This raises lots of questions: 4a. The first phase is "Desugar (after optimization)". What optimizations are performed during desugaring? 4b. I'm not sure whether I'm looking at a single iteration of core2core transformation or at multiple ones. Some passes are performed several times (Float out, Float inwards), which suggests that there might be many iterations here. On the other hand simplifier phases are decreasing towards 0, which looks as if it was one core2core iteration. My assumption here is that every time a new core2core iteration starts the simplifier phases are counted anew from 2 towards 0. Is that correct? 4c. Why are there several 0 phases of the Simplifier? I find it confusing. 4d. I understand that some passes can be enabled or disabled using command-line options. Can the decission to run some passes be made dynamically by the compiler (eg. to run extra simplifier passes)? 4e. Are there more phases that could appear here, ie. they were ommited with -O? 4f. "Levels added" pass before the "Float out" pass: my guess is that this is preparation for the full laziness transform. So, is full laziness performed as part of "Float out" pass? A general note is that I am confused by many Simplifier passes being interleaved with other passes. I expected that simplifier phases will grouped into a single pass, as speculated in question 1. 5. What optimizations *exactly* are performed by the Simplifier? I assume that most of what's described in chapter 3 of [2]: beta reduction, let elimination, case elimination, case floating, constant folding and eta expansion. I'm not sure about floating let outwards and inwards - [1], pg. 7, says these are in a pass separate from the simplifier. `-dverbose-core2core` seems to confirm that since it reveals separate "Float out" and "Float inwards" passes. 6. [4], pg. 31, mentions the Deforestation optimisation. Is everything described in that "Deforestation" section subsumed by cardinality analysis ([5], end of section 2.1 and section 7.1)? If not then when is deforestation performed? 7. [5], section 6.1 says: "We run the analysis twice, once in the middle of the optimisation pipeline and once near the end". When exactly in the middle of the pipeline? Between which passes? This does not show up with `-dverbose-core2core` (or at least it is not explicitly named). 8. How does the rules rewriting fit into the picture? Section 7.20.6.5 of the User Guide and section 4.1 of [7] explain the interaction between rules and inlining and my guess is that both are performed by the Simplifier. Again the "simplifier phases/iterations" distinction puzzles me as to what exactly is happening when. Within a single phase is the inlining happening before rewriting or vice versa? I know that all of the above questions can be answered by looking at the source code for sufficiently long. This is actually what I'm planning to do next but if anyone could help me by answering some of these questions this would certainly save me some time. My plan is to gather up the answers on a wiki page. Janek

2 2

phabricator issue with git submodules.
by Karel Gardas 27 Jul '14

27 Jul '14

Hi, just fixing few warning issues on Solaris/x86. The changes spread over main ghc tree and libraries/primitive and libraries/unix. I already commited changes and pushed to my github.com's forks of libraries/primitive and libraries/unix. The git status looks then: $ git status On branch master Your branch is ahead of 'origin/master' by 2 commits. (use "git push" to publish your local commits) Changes not staged for commit: (use "git add <file>..." to update what will be committed) (use "git checkout -- <file>..." to discard changes in working directory) modified: libraries/primitive (new commits) modified: libraries/unix (new commits) no changes added to commit (use "git add" and/or "git commit -a") and yet phabricator still complains about it: $ arc diff You have unstaged changes in this working copy. Working copy: /export/home/karel/vcs/ghc-src/validate-fixes/ Unstaged changes in working copy: libraries/primitive libraries/unix Do you want to amend these files to the commit? [y/N] Usage Exception: Stage and commit (or revert) them before proceeding. I pressed enter in question above. Is that a known issue or am I doing something wrong here? Thanks! Karel

2 3

Extending ghc-pkg to handle installed package IDs
by Edward Z. Yang 26 Jul '14

26 Jul '14

Hello all, Since we're relaxing the constraint that a package ID correspond to exactly one installed package in the package database, it will now sometimes be necessary to query the package database using ghc-pkg for a specific installed package ID. At the moment, none of the commands support this, and I'd like to request comments on proposed ways of adding this functionality. Here are the proposals: 1. All command line modes stay the same, but anywhere a package ID was previously accepted, now we also accept an installed package ID. We distinguish the two by using the IPID parser from Cabal, and falling back the the package ID parser (assuming that version tags are not used, this is unambiguous). 2. Add a new flag --id which changes the meaning of a package ID field to be an installed package ID field. 3. Add new variants of all commands such as 'ghc-pkg unregister-id' which accept package IDs. 4. ... 5. Profit!! Cheers, Edward

1 0

Heads up: binary package db format changed (EOM)
by Edward Z. Yang 26 Jul '14

26 Jul '14

1 0

Fatal git error on .git/modules/libffi-tarballs
by Karel Gardas 25 Jul '14

25 Jul '14

Hello, while working on HEAD and after a few iteraction of ./validate my git complains with: $ git status fatal: Not a git repository: /export/home/karel/vcs/ghc-src/ghc-git-test-2/.git/modules/libffi-tarballs karel@silence:~/vcs/ghc-src/ghc-solaris-validate-fix$ Is this a known error or is there any known workaround for this issue? Thanks! Karel

2 2

At long last: 2014.2.0.0 Release Candidate 1
by Mark Lentczner 25 Jul '14

25 Jul '14

*Even I find it hard to believe that it's finally here!* *I'm happy to announce the RC1 release of Haskell Platform 2014.2.0.0.* You can find builds for Windows (32bit and 64bit), Mac (64bit), as well as a source tarball, and a tag in the repo: - source tarball: haskell-platform-2014.2.0.0-RC1.tar.gz <http://www.ozonehouse.com/mark/platform/haskell-platform-2014.2.0.0-RC1.tar…> - source repo: https://github.com/haskell/haskell-platform/tree/2014.2.0.0-RC1 - windows 32bit: HaskellPlatform-2014.2.0.0-i386-RC1-setup.exe <http://www.ozonehouse.com/mark/platform/HaskellPlatform-2014.2.0.0-i386-RC1…> - windows 64bit: HaskellPlatform-2014.2.0.0-x86_64-RC1-setup.exe <http://www.ozonehouse.com/mark/platform/HaskellPlatform-2014.2.0.0-x86_64-R…> - os x 64bit: Haskell Platform 2014.2.0.0 64bit RC1.signed.pkg <http://www.ozonehouse.com/mark/platform/Haskell%20Platform%202014.2.0.0%206…> - travis-ci build: 20 <https://travis-ci.org/haskell/haskell-platform/builds/30723884> *Notes:* General - Built with the new shake based build system - cgi package not included as it doesn't build under 7.8, and no word from the maintainer in quite some time - hscolour is included as a tool, mostly as it is used to build the platform itself on win and mac... but we should probably officially decide to include it - the haskell-platform package itself is not in these releases... not sure if we actually need it as it doesn't contain anything other than dependencies. Windows - The Haskell Platform now provides a native Windows 64-bit installation (haskell-platform issue #54 <https://github.com/haskell/haskell-platform/issues/54>) - All included packages built without --enable-split-objs (GHC 7.8 FAQ <https://ghc.haskell.org/trac/ghc/wiki/GHC-7.8-FAQ#Cabal>) - All included packages built without --enable-shared (GHC ticket #8228 <https://ghc.haskell.org/trac/ghc/ticket/8228>) - All html document links are relative and everything links nicely together now, including the master index and cross-package links - If other Haskell Platform installations are detected during the installation of Haskell Platform 2014.2.0.0, a warning is displayed to the user that this is not recommended since problems will arise due to how the PATH is used in many cases to find ancillary build tools - Using ghci to build an executable that links against a DLL may result in numerous warnings about symbols (GHC ticket #9297 <https://ghc.haskell.org/trac/ghc/ticket/9297>) - On Windows 64-bit, some unneeded python files are included (GHC ticket #9014 <https://ghc.haskell.org/trac/ghc/ticket/9014>) - The Haskell Platform for Windows, both 32-bit and 64-bit, now includes an updated version of the OpenGL Utility Toolkit (GLUT) from the FreeGLUT project, utilizing the pre-built distribution from http://www.transmissionzero.co.uk/software/freeglut-devel/ with the MinGW build (freeglut-MinGW-2.8.1-1.mp.zip). (haskell-platform issue #81 <https://github.com/haskell/haskell-platform/issues/81>) Mac OS X - Distributed with a build of 7.8.3 that differs from the released bindist in two ways: a) it was built split-objs for smaller resulting executables, b) it includes Cabal-1.18.1.4 which fixes a particularly nasty problem with haddock, -XCPP, and clang based systems. This ghc-7.8.3 bindist is available as well: - ghc-7.8.3-x86_64-apple-darwin-r3.tar.bz2 <http://www.ozonehouse.com/mark/platform/ghc-7.8.3-x86_64-apple-darwin-r3.ta…> - haddocks finally cross link between packages correctly - includes a new experimental activate-hs command that can switch between multiple installed versions of the platform - includes a slightly updated uninstall-hs command - the cabal command is wrapped to provide a smoother file layout on the Mac... the wrapping only updates the ~/.cabal/config file the first time you run it.. please pay attention to its output - if you have a custom config file, you'll want to update it, as Cabal's defaults have changed - works on 10.7 (tested), 10.8 (assumed!), 10.9 (tested) - both with gcc and clang based Xcode installs. (Next RC will support back to 10.6!) - 64bit only build this time... anyone still want the 32bit build? Source tarball - Quite some changes from last time - notably now includes the new build machinery, not the old. - The build machinery is well tested (see the travis-ci link above) from a repo... not tested much from a source repo - It isn't clear the source repo is of much interest... anyone need it? anyone need anything beyond the haskell-platform.cabal file in it? Timetable - These can "soak" amongst the intreped on these lists for 48 hours. - On Saturday, I'll release any needed updated RCs, and announce to a wider audience - End of next week (from my vacation, I'll point out), we'll declare success and ship. — Mark P.S.: sha256 sums for the reasonably wary: 860cba2d65bc1790ec3b8a0a62c8aaf19f5b18e9070aa1ec3c468b7c6ca82697 Haskell Platform 2014.2.0.0 64bit RC1.signed.pkg 8ca8a994334bf846ac6a41ada381c51634b2fce0d68dc70b2dadf59cb8748397 haskell-platform-2014.2.0.0-RC1.tar.gz b034e2daa595e0d3827860b062afca05f62df64434b0923559acc99bd82ef1e9 HaskellPlatform-2014.2.0.0-i386-RC1-setup.exe e31fe5e0cb9dca330e6f0c62ac49bf70b79f555bfe11e8f04c47739fef224145 HaskellPlatform-2014.2.0.0-x86_64-RC1-setup.exe 8e479d9dd504b1c603cd51f3be0fa57ecdc996e842d655e39d97faafff3c2d31 ghc-7.8.3-x86_64-apple-darwin-r3.tar.bz2

3 9

Early draft spec of Strict language pragma
by Johan Tibell 25 Jul '14

25 Jul '14

Hi! I started a draft spec for the Strict language pragma we chatted about during our call a while ago. I made the big mistake of writing it much after the actual discussion, so I forgot most of the details. https://ghc.haskell.org/trac/ghc/wiki/StrictPragma Perhaps if you could ask questions that you don't think are answered in the wiki page I can start fleshing it out more? P.S. I've included our chat log at the bottom of the wiki page. -- Johan

3 5

RE: GhcPlugin-writing and "finding things"
by Simon Peyton Jones 25 Jul '14

25 Jul '14

Philip You are right: there are some missing pieces. * First you need to ask where your plugin's special library module "Foo" is in the file system. This is what findImportedModule is for, and it seems quite reasonable. However, it (or some variant) should be available to you in CoreM. * Next, suppose you special library module defines a special type "T". You need to get its Name. For this you a CoreM variant of IfaceEnv.lookupOrig. The function CoreMonad.getOrigNameCache is far too low level and should be killed. Instead, CoreMonad should expose lookupOrig :: Module -> OccName -> CoreM Name It should be an easy function to write, using IfaceEnv.lookupOrig; maybe a tiny bit of refactoring. * Next you want to get from T's Name to T's TyCon. Here CoreMonad is fine: it offers lookupThing :: Name -> CoreM TyThing This function calls TcEnv.tcLookupGlobal, which will automatically load Foo.hi if need be. So your code should look like foo_mod <- findImportedModule "Foo" t_name <- lookupOrig foo_mod (mkTcOcc "T") t_tycon <- lookupThing t_name corresponding to these three steps. I suspect that the error cases of findImported module should be dealt with via exceptions in CoreM, to de-clutter the code. Some of the above suggests a bit of cleaning up of the CoreM API. Would someone like to undertake that? I can advise, but I don't want to lead. Simon | -----Original Message----- | From: Glasgow-haskell-users [mailto:glasgow-haskell-users- | bounces(a)haskell.org] On Behalf Of p.k.f.holzenspies(a)utwente.nl | Sent: 23 July 2014 17:07 | To: glasgow-haskell-users(a)haskell.org | Subject: GhcPlugin-writing and "finding things" | | Dear GHC-ers, | | I'm working on a plugin for GHC that should help compile the library | with which this plugin is to ship. What this plugin does is traverse | the CoreProgram(s) to find things of types defined in my library and | optimizes them. I have worked out how to "find" things, but I was | wondering whether the API could be improved for plugin-writers. | | For the sake of argument, I have the following: | - module Foo: library for users to import, containing functions, ADTs | etc | - module Foo.Plugin: GhcPlugin that compiles out all uses of things in | Foo | | > module Foo where | > | > data Foo x = Foo x | > | > runFoo :: Foo x -> x | > runFoo (Foo x) = x | | | This example is trivial and I imagine GHC will have no trouble | eliminating most cases of this, but imagine more complex stuff. Now, if | I want to traverse the CoreProgram in my plugin, I need to find | occurrences of these, so somewhere there's stuff like: | | > pass tcFoo _ _ (NonRec b expr) | > | varType b `containsTyConAnywhere` tcFoo | > = {- clever stuff to compile out Foo -} | | My problem is "getting" tcFoo in this example. Below is how I do it | now. Maybe I'm being thick, or maybe there's just no simpler way. This | is my 'plugin' function in Foo.Plugin: | | > plugin = Plugin $ \opts todo -> do | > hsc <- getHscEnv | > dfs <- getDynFlags | > fr <- liftIO $ findImportedModule hsc (mkModuleName "Foo") Nothing | > mod <- case fr of | > Found ml m -> return m | > _ -> panic "Failed to (unambiguously) find 'Foo' (using | findImportedModule)" | > onc <- getOrigNameCache | > let nms = lookupWithDefaultModuleEnv nms (panic "No names defined | for module 'Foo'") mod | > find_ d occ fnd nm | > = maybe | > (fail $ "Failed to find " ++ d ++ " '" ++ nm ++ "'") | > fnd | > (lookupOccEnv nms $ occ nm) | > tcFind = find_ "TyCon" mkTcOcc lookupTyCon | > dcFind = find_ "DataCon" mkDataOcc lookupDataCon | > idFind = find_ "Id" mkVarOcc lookupId | > tcFoo <- tcFind "Foo" | > dcFoo <- dcFind "Foo" | > idRunFoo <- idFind "runFoo" | > return $ CoreDoPluginPass "Foo optimisation" (pass tcFoo dcFoo | > idRunFoo) : todo | | I have the following questions: | | 1) Is this a/the right way to "find" those things in the plugin? | 2) There seems to be a lot to gain with quasi-quoting a la Template | Haskell for people writing plugins to go with a library that they | wrote. Can such QQ be done? Has it been considered? | 3) Is findImportedModule the right function to find my starting point | to begin with? | 4) What is the 'Maybe FastString' argument in findImportedModule for? | I've been trying to put in the FSs of PackageIDs, but they make the | lookup fail. This (dumb) example really made me nervous: | | > fr <- liftIO $ findImportedModule hsc (mkModuleName "Foo") Nothing | > mod <- case fr of | > Found ml m -> do | > fr' <- liftIO $ findImportedModule hsc (moduleName m) | > (packageIdFS $ packageId m) | | Here, fr' should always be a "Found ml' m'" such that ml == ml' and m | == m', but... it consistently results in NotFound{} for me. Also, I | find this especially round-about. Shouldn't Paths_Foo.hs (the Cabal- | generated file) maybe contain variables for every module in the | package? In my case it would thus contain some "modFoo :: Module" | | Comments and suggestions more than welcome! | | Regards, | Philip | | | | | | | | _______________________________________________ | Glasgow-haskell-users mailing list | Glasgow-haskell-users(a)haskell.org | http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

1 0