December 2016 - ghc-devs

Trac to Phabricator (Maniphest) migration prototype
by Matthew Pickering 11 Jan '17

11 Jan '17

Dear devs, I have completed writing a migration which moves tickets from trac to phabricator. The conversion is essentially lossless. The trac transaction history is replayed which means all events are transferred with their original authors and timestamps. I welcome comments on the work I have done so far, especially bugs as I have definitely not looked at all 12000 tickets. http://ec2-52-213-249-242.eu-west-1.compute.amazonaws.com All the user accounts are automatically generated. If you want to see the tracker from your perspective then send me an email or ping me on IRC and I can set the password of the relevant account. NOTE: This is not a decision, the existence of this prototype is to show that the migration is feasible in a satisfactory way and to remove hypothetical arguments from the discussion. I must also thank Dan Palmer and Herbert who helped me along the way. Dan was responsible for the first implementation and setting up much of the infrastructure at the Haskell Exchange hackathon in October. We extensively used the API bindings which Herbert had been working on. Further information below! Matt ===================================================================== Reasons ====== Why this change? The main argument is consolidation. Having many different services is confusing for new and old contributors. Phabricator has proved effective as a code review tool. It is modern and actively developed with a powerful feature set which we currently only use a small fraction of. Trac is showing signs of its age. It is old and slow, users regularly lose comments through accidently refreshing their browser. Further to this, the integration with other services is quite poor. Commits do not close tickets which mention them and the only link to commits is a comment. Querying the tickets is also quite difficult, I usually resort to using google search or my emails to find the relevant ticket. Why is Phabricator better? ==================== Through learning more about Phabricator, there are many small things that I think it does better which will improve the usability of the issue tracker. I will list a few but I urge you to try it out. * Commits which mention ticket numbers are currently posted as trac comments. There is better integration in phabricator as linking to commits has first-class support. * Links with differentials are also more direct than the current custom field which means you must update two places when posting a differential. * Fields are verified so that mispelling user names is not possible (see #12623 where Ben mispelled his name for example) * This is also true for projects and other fields. Inspecting these fields on trac you will find that the formatting on each ticket is often quite different. * Keywords are much more useful as the set of used keywords is discoverable. * Related tickets are much more substantial as the status of related tickets is reflected to parent ticket. (http://ec2-52-213-249-242.eu-west-1.compute.amazonaws.com/T7724) Implementation ============ Keywords are implemented as projects. A project is a combination of a tag which can be used with any Phabricator object, a workboard to organise tasks and a group of people who care about the topic. Not all keywords are migrated. Only keywords with at least 5 tickets were added to avoid lots of useless projects. The state of keywords is still a bit unsatisfactory but I wanted to take this chance to clean them up. Custom fields such as architecture and OS are replaced by *projects* just like keywords. This has the same advantage as other projects. Users can be subscribed to projects and receive emails when new tickets are tagged with a project. The large majority of tickets have very little additional metadata set. I also implemented these as custom fields but found the the result to be less satisfactory. Some users who have trac accounts do not have phab accounts. Fortunately it is easy to create new user accounts for these users which have empty passwords which can be recovered by the appropriate email address. This means tickets can be properly attributed in the migration. The ticket numbers are maintained. I still advocate moving the infrastructure tickets in order to maintain this mapping. Especially as there has been little activity in thr the last year. Tickets are linked to the relevant commits, differentials and other tickets. There are 3000 dummy differentials which are used to test that the linking works correctly. Of course with real data, the proper differential would be linked.(http://ec2-52-213-249-242.eu-west-1.compute.amazonaws.com/T11044) There are a couple of issues currently with the migration. There are a few issues in the parser which converts trac markup to remarkup. Most comments have very simple with just paragraphs and code blocks but complex items like lists are sometimes parsed incorrectly. Definition lists are converted to tables as there are no equivalent in remarkup. Trac ticket links are converted to phab ticket links. The ideal time to migrate is before the end of January The busiest time for the issue tracker is before and after a new major release. With 8.2 planned for around April this gives the transition a few months to settle. We can close the trac issue tracker and continue to serve it or preferably redirect users to the new ticket. I don't plan to migrate the wiki at this stage as I do not feel that the parser is robust enough although there are now few other technical challenges blocking this direction.

8 25

OpenSearch with GHC manual
by Sylvain Henry 05 Jan '17

05 Jan '17

Hi, Search engines often reference old versions of the GHC user guide. For instance with Google and the request "ghc unboxed tuples" I get the manual for 7.0.3, 5.04.1 and 6.8.2 as first results. With DuckDuckGo I get 6.12.3 and then "latest" versions of the manual. So I have made a custom search engine for the latest manual (using OpenSearch spec). You can install it from the following page: http://haskus.fr/ghc/index.html like any other search engine. Sphinx supports automatic generation of OpenSearch spec: http://www.sphinx-doc.org/en/1.4.8/config.html#confval-html_use_opensearch Maybe we should use this to make the search engine easier to find and use. As a side note, would it be possible to have a nicer URI for the latest doc? Currently it is: https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/index.html Something like https://haskell.org/ghc/doc/index.html would be better IMO. Sylvain

2 1

Lightweight Concurrency Branch
by Daniel Bennet 04 Jan '17

04 Jan '17

The lightweight concurrency branch is highly interesting and relevant to my interests, however, the ghc-lwc2 branch hasn't been updated in several years even though it's listed as an active branch at https://ghc.haskell.org/trac/ghc/wiki/ActiveBranches The wiki page for the work hasn't been updated in almost two years either, https://ghc.haskell.org/trac/ghc/wiki/LightweightConcurrency Relevant papers: Composable Scheduler Activations for Haskell (2014) https://timharris.uk/papers/2014-composable-tr.pdf Composable Scheduler Activations for Haskell (2016) http://kcsrk.info/papers/schedact_jfp16.pdf What remains for integrating this branch into GHC?

2 1

[ANNOUNCE] GHC 8.0.2 release candidate 2
by Ben Gamari 27 Dec '16

27 Dec '16

Hello everyone, The GHC team is happy to announce the second candiate of the 8.0.2 release of the Glasgow Haskell Compiler. Source and binary distributions are available at http://downloads.haskell.org/~ghc/8.0.2-rc2/ This is the second and likely final release candidate leading up the 8.0.2 release. This release will fix a number of bugs found in 8.0.1 including, * Interface file build determinism (#4012). * Compatibility with macOS Sierra and GCC compilers which compile position-independent executables by default * Runtime linker fixes on Windows (see #12797) * A compiler bug which resulted in undefined reference errors while compiling some packages (see #12076) * Compatability with systems which use the gold linker * A number of memory consistency bugs in the runtime system * A number of efficiency issues in the threaded runtime which manifest on larger core counts and large numbers of bound threads. * A typechecker bug which caused some programs using -XDefaultSignatures to be incorrectly accepted. * More than two-hundred other bugs. See Trac [1] for a complete listing. This release candidate fixes a number of issues present in -rc1, * #12757, which lead to broken runtime behavior and even crashes in the presence of primitive strings. * #12844, a type inference issue affecting partial type signatures. * A bump of the `directory` library, fixing buggy path canonicalization behavior (#12894). Unfortunately this required a major version bump in `directory` and minor bumps in several other libraries. * #12912, where use of the `select` system call would lead to runtime system failures with large numbers of open file handles. If all goes well we should have a final 8.0.2 release out shortly after the new year. As always, let us know if you encounter trouble. Thanks to everyone who has contributed so far! Happy testing, - Ben [1] https://ghc.haskell.org/trac/ghc/query?status=closed&milestone=8.0.2&col=id…

4 4

Can the definition of alwaysSucceeds be streamlined?
by David Feuer 27 Dec '16

27 Dec '16

Currently, `GHC.Conc` has alwaysSucceeds :: STM a -> STM () alwaysSucceeds i = do ( i >> retry ) `orElse` ( return () ) checkInv i If I understand what's going on here (which I may not), I think this should be equivalent to alwaysSucceeds i = (i >> retry) `orElse` checkInv i David Feuer

1 0

Re: [Haskell-cafe] many haskell's mails are detected as spam on gmail
by Takenobu Tani 25 Dec '16

25 Dec '16

Hi Arian , Thank you for information. At least from about 11th December, detection of the spam have been increasing. I'll report them after I understand it. Regards, Takenobu 2016-12-25 18:34 GMT+09:00 Arian van Putten <aeroboy94(a)gmail.com>: > If I recall correctly it's being worked on. There is a plan to harden the > haskell.org domain during the holidays by introducing DKIM and setting up > DMARC. There is a thread in haskell-cafe titled "[Haskell-cafe] Work on > mail.haskell.org beginning, please report any problems" with more info. > > On Sun, 25 Dec 2016, 09:35 Takenobu Tani, <takenobu.hs(a)gmail.com> wrote: > > Hi, > > I'm using gmail. > Recently, many haskell's mails are detected as spam on gmail. > (ghc-devs, haskell-cafe, ghc-commit, ...) > > Does anyone know why? > Do you know the workaround? > > Regards, > Takenobu > > _______________________________________________ > Haskell-Cafe mailing list > To (un)subscribe, modify options or view archives go to: > http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe > Only members subscribed via the mailman list are allowed to post. > >

1 0

I have Aphronted Phab
by Richard Eisenberg 24 Dec '16

24 Dec '16

Hi Austin, Ben, In trying to write a comment on Phab, I got this: Unhandled Exception ("AphrontCSRFException") You are trying to save some data to Phabricator, but the request your browser made included an incorrect token. Reload the page and try again. You may need to clear your cookies. This was a Web request. This request had an invalid CSRF token. I followed the suggestions, and even tried a different browser (one I never use), to no avail. This still may very well be my fault, but it seems something is awry somewhere. Do you have any help you can offer? Thanks! Richard

3 2

Re: [GHC] #13005: Mac OSX uses MAP_ANON in place of MAP_ANONYMOUS
by Gracjan Polak 24 Dec '16

24 Dec '16

No, `master`. 2016-12-23 23:53 GMT+01:00 GHC <ghc-devs(a)haskell.org>: > #13005: Mac OSX uses MAP_ANON in place of MAP_ANONYMOUS > -------------------------------------+---------------------- > --------------- > Reporter: gracjan | Owner: > Type: bug | Status: merge > Priority: high | Milestone: 8.0.2 > Component: Compiler | Version: 8.0.1 > (Linking) | > Resolution: | Keywords: > Operating System: MacOS X | Architecture: > Type of failure: Building GHC | Unknown/Multiple > failed | Test Case: > Blocked By: | Blocking: > Related Tickets: | Differential Rev(s): Phab:D2881 > Wiki Page: | > -------------------------------------+---------------------- > --------------- > Changes (by bgamari): > > * status: patch => merge > * milestone: => 8.0.2 > > > Comment: > > Did you really observe this on 8.0.1, gracjan? > > -- > Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13005#comment:5> > GHC <http://www.haskell.org/ghc/> > The Glasgow Haskell Compiler >

1 0

Restrictions of proc-notation with RebindableSyntax
by Jan Bracker 23 Dec '16

23 Dec '16

Hello, I want to use the proc-notation together with RebindableSyntax. So far what I am trying to do is working fine, but I would like to know what the exact restrictions on the supplied functions are. I am introducing additional indices and constraints on the operations. The documentation [1] says the details are in flux and that I should ask directly. Best, Jan [1] https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_ext…

10 14

Re: Trac to Phabricator (Maniphest) migration prototype
by Matthew Pickering 23 Dec '16

23 Dec '16

I wondered if someone would ask me about this. In principle I don't see why not but I don't immediately know how to get the correct label. In fact, this is an interesting example as "comment:21" refers to a commit comment (https://ghc.haskell.org/trac/ghc/ticket/10547#comment:21) which I have filtered out whilst doing the conversion and replaced by actual links to the commits in a more idiomatic style. Thus "comment:21" should actually point to #178376 (http://ec2-52-213-249-242.eu-west-1.compute.amazonaws.com/T10547#178376) On Wed, Dec 21, 2016 at 1:54 PM, Sylvain Henry <hsyl20(a)gmail.com> wrote: > Nice work! > > Would it be possible to convert comment references too? For instance in > http://ec2-52-213-249-242.eu-west-1.compute.amazonaws.com/T10547#182793 > "comment:21" should be a link to the label #178747 > > If we do the transfer, we should redirect: > https://ghc.haskell.org/trac/ghc/ticket/{NN}#comment:{CC} > to > phabricator.haskell.org/T{NN}#{tracToPhabComment(NN,CC)} > where "tracToPhabComment" function remains to be written ;-) > > Thanks, > Sylvain > > > On 21/12/2016 11:12, Matthew Pickering wrote: >> >> Dear devs, >> >> I have completed writing a migration which moves tickets from trac to >> phabricator. The conversion is essentially lossless. The trac >> transaction history is replayed which means all events are transferred >> with their original authors and timestamps. I welcome comments on the >> work I have done so far, especially bugs as I have definitely not >> looked at all 12000 tickets. >> >> http://ec2-52-213-249-242.eu-west-1.compute.amazonaws.com >> >> All the user accounts are automatically generated. If you want to see >> the tracker from your perspective then send me an email or ping me on >> IRC and I can set the password of the relevant account. >> >> NOTE: This is not a decision, the existence of this prototype is to >> show that the migration is feasible in a satisfactory way and to >> remove hypothetical arguments from the discussion. >> >> I must also thank Dan Palmer and Herbert who helped me along the way. >> Dan was responsible for the first implementation and setting up much >> of the infrastructure at the Haskell Exchange hackathon in October. We >> extensively used the API bindings which Herbert had been working on. >> >> Further information below! >> >> Matt >> >> ===================================================================== >> >> Reasons >> ====== >> >> Why this change? The main argument is consolidation. Having many >> different services is confusing for new and old contributors. >> Phabricator has proved effective as a code review tool. It is modern >> and actively developed with a powerful feature set which we currently >> only use a small fraction of. >> >> Trac is showing signs of its age. It is old and slow, users regularly >> lose comments through accidently refreshing their browser. Further to >> this, the integration with other services is quite poor. Commits do >> not close tickets which mention them and the only link to commits is a >> comment. Querying the tickets is also quite difficult, I usually >> resort to using google search or my emails to find the relevant >> ticket. >> >> >> Why is Phabricator better? >> ==================== >> >> Through learning more about Phabricator, there are many small things >> that I think it does better which will improve the usability of the >> issue tracker. I will list a few but I urge you to try it out. >> >> * Commits which mention ticket numbers are currently posted as trac >> comments. There is better integration in phabricator as linking to >> commits has first-class support. >> * Links with differentials are also more direct than the current >> custom field which means you must update two places when posting a >> differential. >> * Fields are verified so that mispelling user names is not possible >> (see #12623 where Ben mispelled his name for example) >> * This is also true for projects and other fields. Inspecting these >> fields on trac you will find that the formatting on each ticket is >> often quite different. >> * Keywords are much more useful as the set of used keywords is >> discoverable. >> * Related tickets are much more substantial as the status of related >> tickets is reflected to parent ticket. >> (http://ec2-52-213-249-242.eu-west-1.compute.amazonaws.com/T7724) >> >> Implementation >> ============ >> >> Keywords are implemented as projects. A project is a combination of a >> tag which can be used with any Phabricator object, a workboard to >> organise tasks and a group of people who care about the topic. Not all >> keywords are migrated. Only keywords with at least 5 tickets were >> added to avoid lots of useless projects. The state of keywords is >> still a bit unsatisfactory but I wanted to take this chance to clean >> them up. >> >> Custom fields such as architecture and OS are replaced by *projects* >> just like keywords. This has the same advantage as other projects. >> Users can be subscribed to projects and receive emails when new >> tickets are tagged with a project. The large majority of tickets have >> very little additional metadata set. I also implemented these as >> custom fields but found the the result to be less satisfactory. >> >> Some users who have trac accounts do not have phab accounts. >> Fortunately it is easy to create new user accounts for these users >> which have empty passwords which can be recovered by the appropriate >> email address. This means tickets can be properly attributed in the >> migration. >> >> The ticket numbers are maintained. I still advocate moving the >> infrastructure tickets in order to maintain this mapping. Especially >> as there has been little activity in thr the last year. >> >> Tickets are linked to the relevant commits, differentials and other >> tickets. There are 3000 dummy differentials which are used to test >> that the linking works correctly. Of course with real data, the proper >> differential would be >> linked.(http://ec2-52-213-249-242.eu-west-1.compute.amazonaws.com/T11044) >> >> There are a couple of issues currently with the migration. There are a >> few issues in the parser which converts trac markup to remarkup. Most >> comments have very simple with just paragraphs and code blocks but >> complex items like lists are sometimes parsed incorrectly. Definition >> lists are converted to tables as there are no equivalent in remarkup. >> Trac ticket links are converted to phab ticket links. >> >> The ideal time to migrate is before the end of January The busiest >> time for the issue tracker is before and after a new major release. >> With 8.2 planned for around April this gives the transition a few >> months to settle. We can close the trac issue tracker and continue to >> serve it or preferably redirect users to the new ticket. I don't plan >> to migrate the wiki at this stage as I do not feel that the parser is >> robust enough although there are now few other technical challenges >> blocking this direction. >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs(a)haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > >

2 3