December 2015 - ghc-devs

Improving trac notifications
by Matthew Pickering 12 Feb '17

12 Feb '17

I subscribe to the ghc-tickets[1] mailing list which provides updates to all trac tickets. This is very useful, however when a ticket description is updated the whole description is included in the email which makes it hard to see what actually changed. The web interface shows a nice diff[2] of the changes, I think it would be good if emails could also include a diff rather than the current quite useless output. After a bit of investigation, it appears that the easiest way to achieve this is apply a simple patch to our copy of trac. The current format is hard coded on line 558 in this module, it seems easy to modify this section to instead provide a diff. http://www.edgewall.org/docs/trac-trunk/epydoc/trac.ticket.notification-pys… There is also already a module which has the diff operations implemented which could be used. http://www.edgewall.org/docs/trac-trunk/epydoc/trac.versioncontrol.diff-mod… Another option would be to use something like the `AnnouncerPlugin`[3] but that seems like much more work and would still require someone to write a custom formatter. Any thoughts? Matt [1]: https://mail.haskell.org/mailman/listinfo/ghc-tickets [2]: https://ghc.haskell.org/trac/ghc/ticket/10873?action=diff&version=6 [3]: https://trac-hacks.org/wiki/AnnouncerPlugin

3 8

RE: vectorisation code?
by Simon Peyton Jones 12 Sep '16

12 Sep '16

Austin, (or anyone else) Manuel says: | > Would it be ok if we left it in the repo, but CPP'd it out so that | we | > didn't compile everything? (The DPH library is in the same state at | > the moment.) | > | > It might suffer bit-rot, but it’d still be there for resurrection. | | Sure, that’s ok. Could you action this? Just avoid compiling anything in 'vectorise/', using (I suppose) cpp to create a stub where necessary. Leave enough comments to explain! Simon | | I hope everything is fine in Cambridge! | Manuel | | > | -----Original Message----- | > | From: ghc-devs [mailto:ghc-devs-bounces@haskell.org] On Behalf Of | > | Manuel M T Chakravarty | > | Sent: 16 January 2015 02:58 | > | To: Richard Eisenberg | > | Cc: ghc-devs(a)haskell.org Devs | > | Subject: Re: vectorisation code? | > | | > | [Sorry, sent from the wrong account at first.] | > | | > | We currently don’t have the resources to work on DPH. I would | > | obviously prefer to leave the code in, in the hope that we will be | > | able to return to it. | > | | > | Manuel | > | | > | > Richard Eisenberg <eir(a)cis.upenn.edu>: | > | > | > | > Hi devs, | > | > | > | > There's a sizable number of modules in the `vectorise` | > | subdirectory of GHC. I'm sure these do all sorts of wonderful | > | things. But what, exactly? And, does anyone make use of these | wonderful things? | > | > | > | > A quick poking through the code shows a tiny link between the | > | vectorise code and the rest of GHC -- the function `vectorise` | > | exported from the module `Vectorise`, which is named in exactly | one | > | place from SimplCore. From what I can tell, the function will be | > | called only when `-fvectorise` is specified, and then it seems to | > | interact with a {-# VECTORISE #-} pragma. However, `{-# VECTORISE | > | #-}` doesn't appear in the manual at all, and `-fvectorise` is | > | given only a cursory explanation. It seems these work with DPH... | > | which has been disabled, no? Searching online finds several hits, | > | but nothing more recent than 2012. | > | > | > | > I hope this question doesn't offend -- it seems that | > | vectorisation probably has amazing performance gains. Yet, the | > | feature also seems unloved. In the meantime, compiling (and | > | recompiling, and | > | recompiling...) the modules takes time, as does going through | them | > | to propagate changes from elsewhere. If this feature is truly | > | orphaned, unloved, and unused at the moment, is it reasonable to | > | consider putting it on furlough? | > | > | > | > Thanks, | > | > Richard | > | > _______________________________________________ | > | > ghc-devs mailing list | > | > ghc-devs(a)haskell.org | > | > http://www.haskell.org/mailman/listinfo/ghc-devs | > | | > | _______________________________________________ | > | ghc-devs mailing list | > | ghc-devs(a)haskell.org | > | http://www.haskell.org/mailman/listinfo/ghc-devs

13 35

Re: Questions concerning the LLVM backend - i.e. 'proc point splitting'
by David Spitzenberg 04 Mar '16

04 Mar '16

Simon > | Could you explain the need of further info-tables for 'inner' > | proc-points (those, which are not the entry-block of a function) to > | me, please? > > The main purpose of inner info tables is this. Given (case e of blah) we push a return address (for 'blah') on the stack and evaluate 'e'. During evaluation of 'e', garbage collection may take place. So the garbage collector needs to walk the stack to follow live pointers. Who knows what pointers are live? Answer: the code for 'blah' clearly knows what pointers are alive in the current stack frame, because it is going to access them. So the return address gets an info table that makes that implicit knowledge explicit, as a bitmap describing the liveness/pointer-hood of the stack frame. > > Addresses that are only jumped to don’t need info tables. > > It would be great if, as you learn about this stuff, you could add to the Commentary > https://ghc.haskell.org/trac/ghc/wiki/Commentary > to capture what you have learned. Thank you very much! In cases I find the current description imprecise or incomplete I can add what I've learned to the Commentary of course. > | The documentation states, that proc-point splitting is only required > | by the LLVM backend (see [2]). Does the cmm-pipeline perform proc- > | point splitting even when using the native code generator? > > I'm not certain, but I don't think so. > > | As stated above, the major problem of LLVM seems to be inferring > | control flow when it comes to call proc-points. As it's hard to avoid > | proc-point splitting, we leave the cmm-pipeline 'as-is'. We modify the > > I regard the whole proc-point thing as a major wart. I think we should work hard to eliminate it. So: another line of thought might be: how could we influence LLVM so that we didn't need proc-point splitting? I totally agree with you here. However, determining how LLVM needs to be influenced to eliminate proc-point splitting is a thing I won't be capable of doing mid term because of my current knowledge and lack of time. > | To make control flow more apparent to LLVM, they came up with the > | following idea. Remark before showing the idea: I know that it greatly > > I'm sorry to say that I didn't really understand the idea. Maybe someone else did? David Terei? My apologies for being imprecise. I hope to find the time to work out a little demonstration during the upcoming holidays. I'm currently fully occupied by university because of upcoming exams this week. > If you are seriously about investing effort in the back end, I'd be happy to help; in that case a Skype call is probably the best way. That is a great idea. Due to the above mentioned time constraints I propose that we follow up in January. Regards, David > Simon > > | interferes with CPS. It leaves parts of the continuation-handling to > | the implicit LLVM call-stack. Further, this does not solve the problem > | of proc-point splitting at all. It is more a workaround than a > | solution. > | But it would greatly increase LLVM's possibilities to optimize code > | later on. That's why I found this idea worth mentioning. Especially > | because after reading your answer, Ben, I'm somewhat convinced that > | solving the problem of proc-point splitting is nearly impossible with > | the capabilities of LLVM IR available at the moment. Now back to the > | idea. > | > | As stated above, the major problem of LLVM seems to be inferring > | control flow when it comes to call proc-points. As it's hard to avoid > | proc-point splitting, we leave the cmm-pipeline 'as-is'. We modify the > | code, llvmGen produces when it comes to call proc-points instead. All > | other types of proc-points remain unchanged. > | > | Initial situation in pseudo cmm: > | > | foo() { > | ... > | call bar() returns to cont // bar defined externally > | > | cont: > | ... > | } > | > | After proc-point splitting, this is converted into LLVM IR similar to > | the following: > | > | @foo() { > | ... > | store @cont(), %Sp ; push continuation on the stack tail call @bar() ; > | @bar defined externally } > | > | @cont() { > | ... > | } > | > | To fix the above issue, we introduce a method, which restores the > | pointers Sp, Hp and the register R1 (among everything else, what has > | to be restored) and 'fakes' the continuation. Note, that '@restore' > | returns without calling into the continuation. This way, the call into > | the continuation can be made direct. The above code would be > | transformed into something similar to the following: > | > | @foo() { > | ... > | store @restore(), %Sp ; 'fake' continuation call @bar() ; br label > | restore > | > | restore: > | %Sp = load (getelementptr @environment 0, 0) ; restore what has to be > | ... ; restored tail call @cont() } > | > | @cont() { > | ... > | } > | > | @restore(%Sp, %Hp, %R1, ...) { > | store %Sp, (getelementptr @environment 0, 0) ; store everything into > | ... ; global variable ret void } > | > | @environment ; structure of type similar to {i64*, i64*, i64*, i64, > | i64, ..} > | > | As an alternative, the above transformation could be done by a LLVM > | pass. llvmGen could then annotate the continuation of calls call cite. > | This would greatly simplify the detection of the correct continuation. > | > | What do you think about this idea? Especially when thinking about the > | interference with CPS? > | > | Regards, > | > | David > | > | > | > | > | [1] > | https://ghc.haskell.org/trac/ghc/wiki/Commentary/Compiler/GeneratedCod > | e#Importantconceptsinthemachine > | > | [2] > | https://ghc.haskell.org/trac/ghc/wiki/Commentary/Compiler/CodeGen#Seco > | ndstage:theCmmpipeline > | > | > | > | > | On 11/21/2015 12:00 AM, Ben Gamari wrote: > | > Simon Peyton Jones <simonpj(a)microsoft.com> writes: > | > > | >> David > | >> > | >> All this stuff is terribly paged-out for me. But I'd love someone > | to > | >> pay attention to some of this back-end stuff, so if you'd like to > | >> work there, I'd like to help you. > | >> > | >> David Terei was also thinking of getting rid of proc point > | splitting > | >> for LLVM (see attached email and the notes attached to it) > | >> > | >> Simon Marlow also was thinking about this; see his attached email. > | >> > | >> The main *reason* for establishing a proc-point is so that, when > | >> generating C code (which we don't do any more) you can take its > | >> address. E.g. > | >> > | >> foo() { ... Push &bar onto the stack (effectively a return address) > | >> Jump to thingumpy } > | >> > | >> bar() { ... } > | >> > | >> Really bar is part of foo, but we have make it a separate top level > | >> thing so that we can take the address of bar and push it on the > | stack. > | >> > | >> The trouble is that if there is a label inside bar that foo wants > | to > | >> jump to (i.e. without pushing &bar etc) then we have to make that > | >> label too into a proc-point, so that both foo and bar can see it. > | >> Urgh. > | >> > | >> In LLVM this probably is not true; presumably you can take the > | >> address of any label? > | >> > | > This is true. While labels themselves have function-local scope in > | > LLVM, there is an expression, `blockaddress`, which allows you to > | take > | > an address to a label in another function [1]. That being said, in > | > reading through the documentation it's not at all clear to me that > | it > | > would be safe to jump to such an address. In fact, given that the > | > instruction that this expression is supposed to be used with, > | > `indirectbr`, can only be used for local blocks, I suspect it is > | not. > | > More information about this feature can be found here [2]. > | > > | > The jump issue aside, I don't know how you would deal with > | > tables-next-to-code. The prefix data support that currently > | available > | > in LLVM is attached to functions and I unfortunately don't see that > | > changing any time soon. > | > > | > Ultimately it seems that trying to refer to labels defined in other > | > functions is using LLVM against the way it was intended. One > | > alternative would be to teach llvmGen to merge mutually recusive > | > top-level functions into a single LLVM function during code > | > generation. Otherwise I'm afraid you are stuck with either the > | status > | > quo or attempting to improve on LLVM's own cross-procedure > | optimization ability. > | > > | > That being said, it sounds as though eliminating proc-point > | splitting > | > would make for quite a nice project in the native code generator. > | > > | > Cheers, > | > > | > - Ben > | > > | > > | > [1] > | > > | https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fllvm.o > | > rg%2fdocs%2fLangRef.html%23addresses-of-basic- > | blocks&data=01%7c01%7csi > | > > | monpj%40064d.mgd.microsoft.com%7c7fdbfda0b4514a098c4d08d2f74448f1%7c72 > | > > | f988bf86f141af91ab2d7cd011db47%7c1&sdata=nKVLgmZezUdEYL%2fma1exMMPLbcT > | > RkaE1JrvVaR1EesY%3d [2] > | > > | https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fblog.l > | > lvm.org%2f2010%2f01%2faddress-of-label-and-indirect- > | branches.html&data > | > > | =01%7c01%7csimonpj%40064d.mgd.microsoft.com%7c7fdbfda0b4514a098c4d08d2 > | > > | f74448f1%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=BTMdZESwYS4tZa5W > | > sOAqyFutoI5xNNFDe0b%2bdKC3ODs%3d [3] > | > > | https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fllvm.o > | > rg%2fdocs%2fLangRef.html%23prefix- > | data&data=01%7c01%7csimonpj%40064d.m > | > > | gd.microsoft.com%7c7fdbfda0b4514a098c4d08d2f74448f1%7c72f988bf86f141af > | > > | 91ab2d7cd011db47%7c1&sdata=rdo4ZLjjE%2bWxNykVL%2bSuToWioY6nzQpflSk296E > | > 8W70%3d > | > >

3 4

"Excuse me, I think this i my stop..." - Resigning from the Platform
by Mark Lentczner 22 Feb '16

22 Feb '16

I think this is the right time for me to exit: The truth is, I still can't bring myself to use a version of Haskell post the Foldable-Traversable-aPocalypse, let alone some future Haskell after the changes now in the works. My personal machines are all still 7.8. My personal projects are all pre-FTP. The Haskell I love to code in, the Haskell I'm passionate about, the Haskell I've advocated for real world use, and the Haskell I like to teach, is 7.8, pre-FTP. It's not that I'm dead set against change and evolution in a language, or even breaking changes. But FTP and beyond are changes that have lost the balance that Haskell had between abstraction and clarity, between elaborate and practical engineering. I don't see any restraint going forward, so I'm getting off the train. This puts me in an odd position with respect to Haskell Platform: I find myself building the Platform for a version of Haskell that I don't use. This isn't healthy for either the Platform or me. Hence, I'm resigning as release manager. I am sad because I believed that Haskell's path to much wider adoption was within reach. Now, especially with the ramping up of the Haskell Prime committee, which seems preordained to codify FTP and beyond into standard, we are approaching our Algol 68 moment: Building a major language revision with less opportunity than it's predecessor. I'll still see you 'round at meet-ups and conferences. I'll just be coding with an older accent. - Mark "mzero" Lentczner

7 8

Fwd: Compiling Core Haskell using GHC API
by Marek Wawrzos 20 Feb '16

20 Feb '16

Hello, I'm trying to compile some module with GHC API. I'm going to feed GHC with Core generated by me. So far is noticed, that function compileCoreToObj ends with an runtime error on each call. In spide of this function produces some *.o and *.hi files. However, produced interface file does not contain any item in exports field, what makes compiled module unusable. Because of this problem, I tried to achieve my goal by using steps described here: https://wiki.haskell.org/GHC/As_a_library (the second example) I was able to print out Core produced form file. If I'm not wrong, modifying structures passing on between sequent calls does not effect on produced *.hi and *.o files. Both files are generated by `load LoadAllTargets' call, and further calls (parseModule, loadModule, ...) does not effect on generated files. Could You give me any advices in that topic? Am I right about compileCoreToObj and functions mentioned above? Best Regards, Marek

2 3

Explanation of a core-lint warning (Bad getNth)
by Jan Bracker 15 Jan '16

15 Jan '16

Hi, I am using the type checker plugin interface and I am trying to produce some evidence for type class instances. During compilation of one of my examples I get this core-lint error: *** Core Lint errors : in result of Simplifier *** <no location info>: Warning: [RHS of ds_a6bY :: (Set '["thres" :-> Int], Set (Unit Reader))] Bad getNth: Nth:0 (Nth:2 (Sub (Sym (TFCo:R:Inv[]Readerfg[0] <'["thres" :-> Int]>_N <'[]>_N)) ; (Inv <Reader>_N <'["thres" :-> Int]>_N (Sym TFCo:R:Unit[]Reader[0]))_R ; Sub (TFCo:R:Inv[]Readerfg[0] <'["thres" :-> Int]>_N <Unit Reader>_N))) Split '["thres" :-> Int] '[] (Union '["thres" :-> Int] '[]) Split '["thres" :-> Int] (Unit Reader) (Union '["thres" :-> Int] (Unit Reader)) I suppose "getNth" refers to the constructor "EvTupleSel" from "EvTerm", "TcNthCo" from "TcCoercion" or to "NthCo" from "Coercion". But I never produce evidence of the shape "getNth". My evidence production code can be found at [1] and the only place where evidence of this shape can come from is my "evaluateType" function [2] that calls "normaliseType" from the GHC module "FamInstEnv". You can reproduce the error by checking out commit 144525886ec107af6f1283b26b19f8125c980aa4 from [3] and running "make effect-example" in the top directory of the repository (GHC 7.10 or better is required and a sandbox is automatically created). The core-lint error does not seem to have any negative consequences when ignored. The produced executable works fine. Can somebody explain why it appears and maybe how I can fix it? Thank you! Jan [1] https://github.com/jbracker/polymonad-plugin/blob/144525886ec107af6f1283b26… [2] https://github.com/jbracker/polymonad-plugin/blob/144525886ec107af6f1283b26… [3] https://github.com/jbracker/polymonad-plugin

4 12

Flag warnings show intermediate hscpp filenames on SmartOS
by Alain O'Dea 10 Jan '16

10 Jan '16

On SmartOS GHC flag warnings show up like this: /tmp/ghc72341_0/ghc_1.hscpp:7:16: Warning: -fffi is deprecated: use -XForeignFunctionInterface or pragma {-# LANGUAGE ForeignFunctionInterface #-} instead It appears that this line in DriverPipeline leads to the correct filename in warnings on Ubuntu, but not on SmartOS: src_opts <- liftIO $ getOptionsFromFile dflags0 output_fn https://github.com/ghc/ghc/blob/bb7f2e33197e667eb694bd1243f125c722a0a868/co… How does the intermediate hscpp filename get restored to a correct filename if warnings are generated during a re-read of the pragmas after having preprocessed the file? Full investigation notes are here (rambling and unedited): https://gist.github.com/AlainODea/98141991849093285c52

3 16

Remote GHCi
by Simon Marlow 08 Jan '16

08 Jan '16

Hi folks - I've been thinking about changing the way we run interpreted code so that it would be run in a separate process. It turns out this has quite a few benefits, and would let us kill some of the really awkward hacks we have in GHC to work around problems that arise because we're running interpreted code and the compiler on the same runtime. I summarised the idea here: https://ghc.haskell.org/trac/ghc/wiki/RemoteGHCi I'd be interested to hear if anyone has any thoughts around this, particularly if doing this would make your life difficult in some way. Are people relying on dynCompileExpr for anything? Cheers, Simon

9 27

Implementation idea for unboxed polymorphic types
by Francesco Mazzoli 05 Jan '16

05 Jan '16

(A nicely rendered version of this email can be found at <https://gist.github.com/bitonic/52cfe54a2dcdbee1b7f3>) ## Macro types I very often find myself wanting unboxed polymorphic types (e.g. types that contain `UNPACK`ed type variables). I find it extremely frustrating that it's easier to write fast _and_ generic code in C++ than in Haskell. I'd like to submit to the mailing list a very rough proposal on how this could be achieved in a pretty straightforward way in GHC. The proposal is meant to be a proof of concept, just to show that this could be done rather easily. I did not think about a nice interface or the implementation details in GHC. My goal is to check the feasibility of this plan with GHC developers. I'll call such types "macro types", since their effect is similar to defining a macro that defines a new type for each type variable instantiation. Consider ``` data #Point a = Point { x :: {-# UNPACK #-} !a , y :: {-# UNPACK #-} !a } ``` This definition defines the macro type `#Point`, with one parameter `a`. Macro types definition would be allowed only for single-constructor records. The intent is that if we mention `#Point Double`, it will be equivalent to ``` data PointDouble = PointDouble { x :: {-# UNPACK #-} !Double , y :: {-# UNPACK #-} !Double } ``` To use `#Point` generically, the following type class would be generated: ``` class PointFamily a where data #Point a :: * -- Family of types generated by @data #Point a@. #Point :: a -> a -> #Point a -- Constructor. #x :: #Point a -> a -- Projection @x@. #y :: #Point a -> a -- Projection @y@. ``` Thi type class lets us work with `#Point`s generically, for example ``` distance :: (PointFamily a, Fractional a) => #Point a -> #Point a -> a distance p1 p2 = let dx = #x p1 - #x p2 dy = #y p1 - #y p2 in sqrt (dx*dx + dy*dy) ``` Internally, for every type appearing for `a`, e.g. `#Point Double`, a new type equivalent to the `PointDouble` above would be generated by GHC, with the corresponding instance ``` instance PointFamily Double where data #Point Double = PointDouble #x = x #y = x ``` If it's not possible to instantiate `#Point` with the provided type (for example because the type is not `UNPACK`able, e.g. `#Point (Maybe A)`), GHC would throw an error. Note that we can compile `distance` in its polymorphic version (as opposed to C++ templates, where template functions _must_ be instantiated at every use). The polymorphic `distance` would require a call to "virtual functions" `#x` and `#y`, as provided by the `PointFamily` dictionary. But if we use `INLINE` or `SPECIALIZE` pragmas the virtual calls to `#x` and `#y` would disappear, making this as efficient as if we were to define `distance` on the manually defined `PointDouble`. Compiler hints would be put in place to always inline functions using macro types, if possible. Note that the inlining is only important so that the `PointFamily` dictionary disappears, e.g. functions containing recursive helpers are fine, such as ``` {-# INLINE leftmost #-} leftmost :: forall a. (PointFamily a, Ord a) => [#Point a] -> #Point a leftmost [] = error "leftmost: no points" leftmost (p0 : ps0) = go p0 ps0 where go :: #Point a -> [#Point a] -> Point# a go candidate (p : ps) = if #x p < #x candidate then go p ps else go candidate ps ``` It might be worth considering throwing a warning when a top-level definition whose type contains a macro type cannot be inlined, since the main performance benefit of using macro types would be lost. We can define instances for these types as normal, for instance ``` instance (Show a, PointFamily a) => Show (#Point a) where {-# INLINE show #-} show pt = "Point{x = " ++ #x pt ++ ", y = " ++ #y pt ++ "}" ``` `deriving` support could also be added. ## Further ideas ### Hide or remove `PointFamily` from the user In the examples above `PointFamily` is manipulated explicitely (e.g. in the type signature for `distance`). In most cases the right constraint could be generated automatically by GHC, although I think direct access to the type class would be beneficial (history shows that direct access to these facilities is good, see upcoming explicit type applications). Maybe the type class associated to a macro type should not even exist -- for example we could simply represent `#Point` as a type family and treat construction and destruction as built-in syntax (the `#` prefix). ### Pattern matching Sugar could be added to pattern match, e.g. ``` foo :: Point# a -> ... distance (Point# x1 y1) = ... or dinstance Point#{..} = ... -- #x and #y are available ``` ### No "record types" limitation Instead of restricting ourselves to single-constructor records, we could simply generate ``` data Point a = Point a { x :: !a , y :: !a } class PointFamily a where data Point# a :: * destruct :: Point# a -> Point a construct :: Point a -> Point# a ``` However, I think it would be harder to guarantee the well-behavedness of the inlined functions if we had this intermediate type. I also don't think macro types would be very useful beyond polymorphic unboxed types.

4 5

Open data kinds
by Jan Stolarek 03 Jan '16

03 Jan '16

Devs, I plan to work on implementing open data kinds (#11080). The idea is that users will be allowed to declare open kinds and then populate them with member types. Perhaps I will also implement closed data kinds. This is already possible using DataKinds, but the idea is to declare a data kind without corresponding data type - see #6024. Now, consider this declaration (syntax subject to bikeshedding): kind K = T In what namespace should T go: type namespace or data constructor namespace? If we put it in type namespace then it is possible for the user to declare a data constructor T that is completely unrelated to type T belonging to kind K. This might be confusing. If we put T in the data namespace then we miss the point of #6024. Thoughts? Janek --- Politechnika Łódzka Lodz University of Technology Treść tej wiadomości zawiera informacje przeznaczone tylko dla adresata. Jeżeli nie jesteście Państwo jej adresatem, bądź otrzymaliście ją przez pomyłkę prosimy o powiadomienie o tym nadawcy oraz trwałe jej usunięcie. This email contains information intended solely for the use of the individual to whom it is addressed. If you are not the intended recipient or if you have received this message in error, please notify the sender and delete it from your system.

3 4