- ghc-devs - Haskell.org

Restrictions of proc-notation with RebindableSyntax
by Jan Bracker 24 Dec '16

24 Dec '16

Hello, I want to use the proc-notation together with RebindableSyntax. So far what I am trying to do is working fine, but I would like to know what the exact restrictions on the supplied functions are. I am introducing additional indices and constraints on the operations. The documentation [1] says the details are in flux and that I should ask directly. Best, Jan [1] https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_ext…

10 14

Re: Trac to Phabricator (Maniphest) migration prototype
by Matthew Pickering 24 Dec '16

24 Dec '16

I wondered if someone would ask me about this. In principle I don't see why not but I don't immediately know how to get the correct label. In fact, this is an interesting example as "comment:21" refers to a commit comment (https://ghc.haskell.org/trac/ghc/ticket/10547#comment:21) which I have filtered out whilst doing the conversion and replaced by actual links to the commits in a more idiomatic style. Thus "comment:21" should actually point to #178376 (http://ec2-52-213-249-242.eu-west-1.compute.amazonaws.com/T10547#178376) On Wed, Dec 21, 2016 at 1:54 PM, Sylvain Henry <hsyl20(a)gmail.com> wrote: > Nice work! > > Would it be possible to convert comment references too? For instance in > http://ec2-52-213-249-242.eu-west-1.compute.amazonaws.com/T10547#182793 > "comment:21" should be a link to the label #178747 > > If we do the transfer, we should redirect: > https://ghc.haskell.org/trac/ghc/ticket/{NN}#comment:{CC} > to > phabricator.haskell.org/T{NN}#{tracToPhabComment(NN,CC)} > where "tracToPhabComment" function remains to be written ;-) > > Thanks, > Sylvain > > > On 21/12/2016 11:12, Matthew Pickering wrote: >> >> Dear devs, >> >> I have completed writing a migration which moves tickets from trac to >> phabricator. The conversion is essentially lossless. The trac >> transaction history is replayed which means all events are transferred >> with their original authors and timestamps. I welcome comments on the >> work I have done so far, especially bugs as I have definitely not >> looked at all 12000 tickets. >> >> http://ec2-52-213-249-242.eu-west-1.compute.amazonaws.com >> >> All the user accounts are automatically generated. If you want to see >> the tracker from your perspective then send me an email or ping me on >> IRC and I can set the password of the relevant account. >> >> NOTE: This is not a decision, the existence of this prototype is to >> show that the migration is feasible in a satisfactory way and to >> remove hypothetical arguments from the discussion. >> >> I must also thank Dan Palmer and Herbert who helped me along the way. >> Dan was responsible for the first implementation and setting up much >> of the infrastructure at the Haskell Exchange hackathon in October. We >> extensively used the API bindings which Herbert had been working on. >> >> Further information below! >> >> Matt >> >> ===================================================================== >> >> Reasons >> ====== >> >> Why this change? The main argument is consolidation. Having many >> different services is confusing for new and old contributors. >> Phabricator has proved effective as a code review tool. It is modern >> and actively developed with a powerful feature set which we currently >> only use a small fraction of. >> >> Trac is showing signs of its age. It is old and slow, users regularly >> lose comments through accidently refreshing their browser. Further to >> this, the integration with other services is quite poor. Commits do >> not close tickets which mention them and the only link to commits is a >> comment. Querying the tickets is also quite difficult, I usually >> resort to using google search or my emails to find the relevant >> ticket. >> >> >> Why is Phabricator better? >> ==================== >> >> Through learning more about Phabricator, there are many small things >> that I think it does better which will improve the usability of the >> issue tracker. I will list a few but I urge you to try it out. >> >> * Commits which mention ticket numbers are currently posted as trac >> comments. There is better integration in phabricator as linking to >> commits has first-class support. >> * Links with differentials are also more direct than the current >> custom field which means you must update two places when posting a >> differential. >> * Fields are verified so that mispelling user names is not possible >> (see #12623 where Ben mispelled his name for example) >> * This is also true for projects and other fields. Inspecting these >> fields on trac you will find that the formatting on each ticket is >> often quite different. >> * Keywords are much more useful as the set of used keywords is >> discoverable. >> * Related tickets are much more substantial as the status of related >> tickets is reflected to parent ticket. >> (http://ec2-52-213-249-242.eu-west-1.compute.amazonaws.com/T7724) >> >> Implementation >> ============ >> >> Keywords are implemented as projects. A project is a combination of a >> tag which can be used with any Phabricator object, a workboard to >> organise tasks and a group of people who care about the topic. Not all >> keywords are migrated. Only keywords with at least 5 tickets were >> added to avoid lots of useless projects. The state of keywords is >> still a bit unsatisfactory but I wanted to take this chance to clean >> them up. >> >> Custom fields such as architecture and OS are replaced by *projects* >> just like keywords. This has the same advantage as other projects. >> Users can be subscribed to projects and receive emails when new >> tickets are tagged with a project. The large majority of tickets have >> very little additional metadata set. I also implemented these as >> custom fields but found the the result to be less satisfactory. >> >> Some users who have trac accounts do not have phab accounts. >> Fortunately it is easy to create new user accounts for these users >> which have empty passwords which can be recovered by the appropriate >> email address. This means tickets can be properly attributed in the >> migration. >> >> The ticket numbers are maintained. I still advocate moving the >> infrastructure tickets in order to maintain this mapping. Especially >> as there has been little activity in thr the last year. >> >> Tickets are linked to the relevant commits, differentials and other >> tickets. There are 3000 dummy differentials which are used to test >> that the linking works correctly. Of course with real data, the proper >> differential would be >> linked.(http://ec2-52-213-249-242.eu-west-1.compute.amazonaws.com/T11044) >> >> There are a couple of issues currently with the migration. There are a >> few issues in the parser which converts trac markup to remarkup. Most >> comments have very simple with just paragraphs and code blocks but >> complex items like lists are sometimes parsed incorrectly. Definition >> lists are converted to tables as there are no equivalent in remarkup. >> Trac ticket links are converted to phab ticket links. >> >> The ideal time to migrate is before the end of January The busiest >> time for the issue tracker is before and after a new major release. >> With 8.2 planned for around April this gives the transition a few >> months to settle. We can close the trac issue tracker and continue to >> serve it or preferably redirect users to the new ticket. I don't plan >> to migrate the wiki at this stage as I do not feel that the parser is >> robust enough although there are now few other technical challenges >> blocking this direction. >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs(a)haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > >

2 3

Competing with C in a simple loop
by Christopher Done 23 Dec '16

23 Dec '16

Purely as an experiment, I've written a function that uses ByteString to simply elemIndex it's way across a string here. Look for <, then look for >. Repeat until done. https://github.com/chrisdone/xeno (under src/Xeno.hs) But if you scroll down the README to the 182kb file example, you see that hexml takes 33us and xeno takes 111us. That's surprising to me because I'm doing just a walk across a string and hexml is doing a full parse. It's written in C, but still, 3x faster AND doing allocations and more work. I tried replacing the ByteString with a raw Ptr Word8 and it didn't make a difference, actually increased time a little bit. My weigh results indicate that it's not doing any allocations during the process, at least nothing linear or above. So, I haven't looked at the core or asm yet, but I'm guessing it's simply doing more instructions and/or indirections than necessary. You can reproduce this with stack build --bench xeno Can anyone make an improvement to the speed? I already nerd sniped myself enough with this, so I'm spreading the bug elsewhere. I think it's a pretty good "raw" performance exercise and possibly something that could serve as a tutorial on haskell performance. Ciao!

3 3

User manual broken
by Simon Peyton Jones 22 Dec '16

22 Dec '16

I’m getting tons of this stuff from the user manual type setting. Might someone fix it? Simon /5playpen/simonpj/HEAD-4/docs/users_guide/8.0.2-notes.rst:162: WARNING: Inline interpreted text or phrase reference start-string without end-string. /5playpen/simonpj/HEAD-4/docs/users_guide/8.2.1-notes.rst:242: WARNING: Bullet list ends without a blank line; unexpected unindent. /5playpen/simonpj/HEAD-4/docs/users_guide/editing-guide.rst:350: ERROR: Error in "ghci-cmd" directive: invalid option block. .. ghci-cmd:: :module [*] <file> Load a module /5playpen/simonpj/HEAD-4/docs/users_guide/eventlog-formats.rst:35: ERROR: Unexpected indentation. /5playpen/simonpj/HEAD-4/docs/users_guide/eventlog-formats.rst:41: WARNING: Block quote ends without a blank line; unexpected unindent. /5playpen/simonpj/HEAD-4/docs/users_guide/eventlog-formats.rst:58: ERROR: Unexpected indentation. /5playpen/simonpj/HEAD-4/docs/users_guide/eventlog-formats.rst:104: ERROR: Unexpected indentation. /5playpen/simonpj/HEAD-4/docs/users_guide/ghci.rst:509: ERROR: Error in "warning" directive: invalid option block. .. warning:: Temporary bindings introduced at the prompt only last until the next :ghci-cmd:`:load` or :ghci-cmd:`:reload` command, at which time they will be simply lost. However, they do survive a change of context with :ghci-cmd:`:module`: the temporary bindings just move to the new location. /5playpen/simonpj/HEAD-4/docs/users_guide/ghci.rst:515: ERROR: Error in "hint" directive: invalid option block.

2 2

Work on mail.haskell.org beginning, please report any problems
by John Wiegley 22 Dec '16

22 Dec '16

Hello Haskellers, Beginning today, I am upgrading our Postfix installation on mail.haskell.org, and introducing some new options to reduce the amount of spam that hits our mailman server. If you experience delivery problems, or any bounced mail, please send a copy of the full bounce message to my address: jwiegley(a)gmail.com. I'll be making more changes gradually over the next few days, and watching the mail logs, but it's possible that mail accepted before will suddenly start getting rejected, depending on how well-behaved your sending mail server is. Activities planned for this Christmas break are: - [x] Upgrade Postfix to 2.11 - [X] Enable postscreen for pre-queue RBL filtering - [ ] DKIM sign messages sent from mailman - [ ] Implement DMARC policy (i.e., reject incoming messages improperly DKIM signed, or failing SPF check) - [ ] Prevent mail being spoofed from haskell.org addresses - [ ] Tighten sender and recipient restrictions - [ ] Re-assess inbound and outbound rate limits - [ ] Use SpamAssassin for post-queue filtering - [ ] If helpful, enable deep protocol pre-filtering - [ ] Document all the above, so others can help with e-mail admin Thank you, John Wiegley Haskell.org, infrastructure team

1 0

Trying to resurrect nofib/fibon
by Gracjan Polak 22 Dec '16

22 Dec '16

Hi all, I went into the rabbit hole that starts here https://ghc.haskell.org/trac/ghc/ticket/11501 So far I know that: 1. fibon is not built regularly by any CI and it bitrotted significantly. 2. Makefile's for fibon use undefined variables like INPLACE_HSC2HS_PGM. Probably those were sourced from ghc/mk/*.mk, but not now. 3. Haskell courses are pre-AMP (this is easiest to fix). 4. Makefile's build object files one by one and that fails in case modules depend on each other. ghc-paths.mk sorts objects and that gives wrong compile order. Does anybody know a commit-hash or date when it did compile last time? Knowing that this module (fibno) wasn't used for so long does it make sense to fix it? How about removing it? -- Gracjan

1 0

Retro-Haskell: can we get seq somewhat under control?
by David Feuer 22 Dec '16

22 Dec '16

In the Old Days (some time before Haskell 98), `seq` wasn't fully polymorphic. It could only be applied to instances of a certain class. I don't know the name that class had, but let's say Seq. Apparently, some people didn't like that, and now it's gone. I'd love to be able to turn on a language extension, use an alternate Prelude, and get it back. I'm not ready to put up a full-scale proposal yet; I'm hoping some people may have suggestions for details. Some thoughts: 1. Why do you want that crazy thing, David? When implementing general-purpose lazy data structures, a *lot* of things need to be done strictly for efficiency. Often, the easiest way to do this is using either bang patterns or strict data constructors. Care is necessary to only ever force pieces of the data structure, and not the polymorphic data a user has stored in it. 2. Why does it need GHC support? It would certainly be possible to write alternative versions of `seq`, `$!`, and `evaluate` to use a user-supplied Seq class. It should even be possible to deal with strict data constructors by hand or (probably) using Template Haskell. For instance, data Foo a = Foo !Int !a would translate to normal GHC Haskell as data Foo a = Seq a => Foo !Int !a But only GHC can extend this to bang patterns, deal with the interactions with coercions, and optimize it thoroughly. 3. How does Seq interact with coercions and roles? I believe we'd probably want a special rule that (Seq a, Coercible a b) => Seq b Thanks to this rule, a Seq constraint on a type variable shouldn't prevent it from having a representational role. The downside of this rule is that if something *can* be forced, but we don't *want* it to be, then we have to hide it a little more carefully than we might like. This shouldn't be too hard, however, using a newtype defined in a separate module that exports a pattern synonym instead of a constructor, to hide the coercibility. 4. Optimize? What? Nobody wants Seq constraints blocking up specialization. Today, a function foo :: (Seq a, Foldable f) => f a -> () won't specialize to the Foldable instance if the Seq instance is unknown. This is lousy. Furthermore, all Seq instances are the same. The RTS doesn't actually need a dictionary to force something to WHNF. The situation is somewhat similar to that of Coercible, *but more so*. Coercible sometimes needs to pass evidence at runtime to maintain type safety. But Seq carries no type safety hazard whatsoever--when compiling in "production mode", we can just *assume* that Seq evidence is valid, and erase it immediately after type checking; the worst thing that could possibly happen is that someone will force a function and get weird semantics. Further, we should *unconditionally* erase Seq evidence from datatypes; this is necessary to maintain compatibility with the usual data representations. I don't know if this unconditional erasure could cause "laziness safety" issues, but the system would be essentially unusable without it. 4. What would the language extension do, exactly? a. Automatically satisfy Seq for data types and families. b. Propagate Seq constraints using the usual rules and the special Coercible rule. c. Modify the translation of strict fields to add Seq constraints as required. David Feuer

3 3

Confused about the sub-modules
by Erik de Castro Lopo 22 Dec '16

22 Dec '16

Hi all, I'm a bit confused about how the GHC dev tree handles submodules like libraries/Cabal, libraries/process, libraries/directory and libraries/containers. All of these libraries/submodules seem to have their own github projects where people can submit PRs, but once the commits have been made there, what is the process to get submodules updated in the GHC tree? Any light people can shed on this process would be appreciated. Erik -- ---------------------------------------------------------------------- Erik de Castro Lopo http://www.mega-nerd.com/

6 8

Improving DeriveTraversable
by David Feuer 22 Dec '16

22 Dec '16

The role system is not currently able to use GND to derive Traversable instances. While we wait for future research to solve that problem, I think it would be nice to address a problem that can arise with DeriveTraversable: when newtypes stack up, fmaps also stack up. I've come up with a trick that I think could help solve the problem in at least some important cases. There may be a nicer solution (perhaps using associated types?), but I haven't found it yet. What I don't know is whether this arrangement works for all important "shapes" of newtypes, or what might be involved in automating it. -- Represents a traversal that may come up with a type that's -- a bit off, but not too far off. If you think about Coyoneda, this type -- might make more sense. Whereas Coyoneda builds up larger and -- larger *function compositions*, we just keep changing the coercion -- types. data Trav t b where Trav :: Coercible x (t b) => (forall f a . Applicative f => (a -> f b) -> t a -> f x) -> Trav t b class (Foldable t, Functor t) => Traversable t where traverse :: Applicative f => (a -> f b) -> t a -> f (t b) -- This new method is not intended to be exported by Data.Traversable, -- but only by some ghc-special module. trav :: Trav t b trav = Trav traverse {-# INLINE trav #-} Here are some sample newtype instances. -- Convenience function from Data.Profunctor.Unsafe (#.) :: Coercible b c => (b -> c) -> (a -> b) -> a -> c _ #. g = coerce g {-# INLINE (#.) #-} -- Convenience function for changing a Trav type retrav :: Coercible u t => (forall a . u a -> t a) -> Trav t b -> Trav u b retrav extr (Trav t) = Trav ((. extr) #. t) -- Function for defining traverse proper. Note that this should -- *only* be used to define traverse for newtype wrappers; -- for other types, it will add an unnecessary fmap. travTraverse :: forall f t a b . (Traversable t, Applicative f) => (a -> f b) -> t a -> f (t b) travTraverse = case trav :: Trav t b of Trav t -> \f xs -> fmap coerce (t f xs) {-# INLINE travTraverse #-} -- Sample types newtype F t x = F {getF :: t x} deriving (Functor, Foldable) newtype G t x = G {getG :: t x} deriving (Functor, Foldable) newtype H f x = H {getH :: F (G f) x} deriving (Functor, Foldable) instance Traversable t => Traversable (F t) where traverse = travTraverse trav = retrav getF trav instance Traversable t => Traversable (G t) where traverse = travTraverse trav = retrav getG trav instance Traversable t => Traversable (H t) where traverse = travTraverse trav = retrav getH trav With these instances, traversing H t a will perform one fmap instead of three. David Feuer

1 0

Re: Help needed: Restrictions of proc-notation with RebindableSyntax
by Edward Kmett 21 Dec '16

21 Dec '16

The S&D parser I was referring to was based on tracking FIRST sets, and provided a nice linear time parsing bound for (infinite) LL(1) grammars. (You can't really compute FOLLOW sets without knowing the grammar has a finite number of productions, but FIRST sets work perfectly well with infinite grammars.) By doing so you can transform parsing into more or less a series of map lookups for dispatch. You need to carry a set of all characters that a parser will consume in the case of legal parses, and whether or not the parser accepts the empty parse. http://www.cse.chalmers.se/~rjmh/afp-arrows.pdf mentions this style of FIRST-set tracking parser as the original motivation for arrows. Of course, they didn't see fit to stop puttering around with parsers after 1998, so referring to "the S&D parser" is quite ambiguous! =) -Edward On Wed, Dec 21, 2016 at 4:00 AM, Henrik Nilsson < Henrik.Nilsson(a)nottingham.ac.uk> wrote: > Hi Edward, > CC Others, > > On 12/21/2016 05:15 AM, Edward Kmett wrote: > >> Arrows haven't seen much love for a while. In part this is because many >> of the original applications for arrows have been shown to be perfectly >> suited to being handled by Applicatives. e.g. the Swiestra/Duponcheel >> parser that sort of kickstarted everything. >> > > Thanks for a very thorough reply. > > A quick side-remark: a parser library due to Sweistra (and maybe > Dupncheel, I can't remember) used an applicative structure a long time > before applicatives became apkicatives and even idioms. (I used a > variation of this library myself for the Freja compiler around 1995. > Freja was part of my PhD work and was close to what Haskell looked like at > the time.) > > I've never used arrows for parsing, or seen the need for arrows in that > context, but find arrows a very good fit for many EDSLs, including > stream-processing/FRP/Yampa of course, along with other circuit-like > abstractions, which I'd say were the original motivation for arrows. > Altenkirch have also used arrow-like notions in the context of quantum > computation. More recently for probabilistic programming and > Bayesian inference. Except then that the current hard-wired "pseudo- > product" in particular often gets in the way. Along with the fact > that there is no good support for constrained arrows (or monads). > > Best, > > /Henrik > > > > > > This message and any attachment are intended solely for the addressee > and may contain confidential information. If you have received this > message in error, please send it back to me, and immediately delete it. > Please do not use, copy or disclose the information contained in this > message or in any attachment. Any views or opinions expressed by the > author of this email do not necessarily reflect the views of the > University of Nottingham. > > This message has been checked for viruses but the contents of an > attachment may still contain software viruses which could damage your > computer system, you are advised to perform your own checks. Email > communications with the University of Nottingham may be monitored as > permitted by UK legislation. > >

1 0