- ghc-devs - Haskell.org

Trac to Phabricator (Maniphest) migration prototype
by Matthew Pickering 11 Jan '17

11 Jan '17

Dear devs, I have completed writing a migration which moves tickets from trac to phabricator. The conversion is essentially lossless. The trac transaction history is replayed which means all events are transferred with their original authors and timestamps. I welcome comments on the work I have done so far, especially bugs as I have definitely not looked at all 12000 tickets. http://ec2-52-213-249-242.eu-west-1.compute.amazonaws.com All the user accounts are automatically generated. If you want to see the tracker from your perspective then send me an email or ping me on IRC and I can set the password of the relevant account. NOTE: This is not a decision, the existence of this prototype is to show that the migration is feasible in a satisfactory way and to remove hypothetical arguments from the discussion. I must also thank Dan Palmer and Herbert who helped me along the way. Dan was responsible for the first implementation and setting up much of the infrastructure at the Haskell Exchange hackathon in October. We extensively used the API bindings which Herbert had been working on. Further information below! Matt ===================================================================== Reasons ====== Why this change? The main argument is consolidation. Having many different services is confusing for new and old contributors. Phabricator has proved effective as a code review tool. It is modern and actively developed with a powerful feature set which we currently only use a small fraction of. Trac is showing signs of its age. It is old and slow, users regularly lose comments through accidently refreshing their browser. Further to this, the integration with other services is quite poor. Commits do not close tickets which mention them and the only link to commits is a comment. Querying the tickets is also quite difficult, I usually resort to using google search or my emails to find the relevant ticket. Why is Phabricator better? ==================== Through learning more about Phabricator, there are many small things that I think it does better which will improve the usability of the issue tracker. I will list a few but I urge you to try it out. * Commits which mention ticket numbers are currently posted as trac comments. There is better integration in phabricator as linking to commits has first-class support. * Links with differentials are also more direct than the current custom field which means you must update two places when posting a differential. * Fields are verified so that mispelling user names is not possible (see #12623 where Ben mispelled his name for example) * This is also true for projects and other fields. Inspecting these fields on trac you will find that the formatting on each ticket is often quite different. * Keywords are much more useful as the set of used keywords is discoverable. * Related tickets are much more substantial as the status of related tickets is reflected to parent ticket. (http://ec2-52-213-249-242.eu-west-1.compute.amazonaws.com/T7724) Implementation ============ Keywords are implemented as projects. A project is a combination of a tag which can be used with any Phabricator object, a workboard to organise tasks and a group of people who care about the topic. Not all keywords are migrated. Only keywords with at least 5 tickets were added to avoid lots of useless projects. The state of keywords is still a bit unsatisfactory but I wanted to take this chance to clean them up. Custom fields such as architecture and OS are replaced by *projects* just like keywords. This has the same advantage as other projects. Users can be subscribed to projects and receive emails when new tickets are tagged with a project. The large majority of tickets have very little additional metadata set. I also implemented these as custom fields but found the the result to be less satisfactory. Some users who have trac accounts do not have phab accounts. Fortunately it is easy to create new user accounts for these users which have empty passwords which can be recovered by the appropriate email address. This means tickets can be properly attributed in the migration. The ticket numbers are maintained. I still advocate moving the infrastructure tickets in order to maintain this mapping. Especially as there has been little activity in thr the last year. Tickets are linked to the relevant commits, differentials and other tickets. There are 3000 dummy differentials which are used to test that the linking works correctly. Of course with real data, the proper differential would be linked.(http://ec2-52-213-249-242.eu-west-1.compute.amazonaws.com/T11044) There are a couple of issues currently with the migration. There are a few issues in the parser which converts trac markup to remarkup. Most comments have very simple with just paragraphs and code blocks but complex items like lists are sometimes parsed incorrectly. Definition lists are converted to tables as there are no equivalent in remarkup. Trac ticket links are converted to phab ticket links. The ideal time to migrate is before the end of January The busiest time for the issue tracker is before and after a new major release. With 8.2 planned for around April this gives the transition a few months to settle. We can close the trac issue tracker and continue to serve it or preferably redirect users to the new ticket. I don't plan to migrate the wiki at this stage as I do not feel that the parser is robust enough although there are now few other technical challenges blocking this direction.

8 25

Pattern checker status
by David Feuer 11 Jan '17

11 Jan '17

Could you possibly give us an update about the current status of your pattern checker work? I know the number of tickets may seem a bit overwhelming; please reach out to the ghc-devs list, or individual developers, to get whatever help you need. Knocking out some of these tickets is a priority for 8.2, and the freeze is coming up fast. In my mind, the top priorities for GHC 8.2 should probably be #10746 and #12949. We really want to get those squashed. I think #11195 is also a fairly high priority (if it's still an issue!). #10746 is a serious correctness issue, and Simon's suggested fix sounds straightforward. Have you run into trouble? #12949 is also a serious correctness issue. You're right that desugaring doesn't happen till after type checking, but I believe the type checker should have already worked out what it needs to be able to desugar the overloading. It would be worth asking how you might access that information. #11195 looks like a practically important and serious performance problem. I know you spent some time investigating it months ago; do you have any more recent progress to report? Do you know if the problem is still there? David Feuer

1 0

nofib on Shake
by Michal Terepeta 10 Jan '17

10 Jan '17

Hi all, While looking at nofib, I've found a blog post from Neil Mitchell [1], which describes a Shake build system for nofib. The comments mentioned that this should get merged, but it seems that nothing actually happened? Is there some fundamental reason for that? If not, I'd be interested picking this up - the current make-based system is pretty confusing for me and `runstdtest` looks simply terrifying ;-) We could also create a cabal and stack files for `nofib-analyse` (making it possible to use some libraries for it). Thanks, Michal [1] http://neilmitchell.blogspot.ch/2013/02/a-nofib-build-system-using-shake.ht…

4 8

Debugging GHC with GHCi
by Thomas Jakway 10 Jan '17

10 Jan '17

I want to be able to load certain GHC modules in interpreted mode in ghci so I can set breakpoints in them. I have tests in the testsuite that are compiled by inplace/bin/ghc-stage2 with -package ghc. I can load the tests with ghc-stage2 --interactive -package ghc but since ghc is compiled I can only set breakpoints in the tests themselves. Loading the relevant files by passing them as absolute paths to :l loads them but ghci doesn't stop at the breakpoints placed in them (I'm guessing because ghci doesn't realize that the module I just loaded is meant to replace the compiled version in -package ghc). So if I use inplace/bin/ghc-stage2 --interactive -package ghc mytest.hs then :l abs/path/to/AsmCodeGen.hs and set breakpoints, nothing happens. Ideally I'd only have to load the module I'm debugging (and it's dependencies?) but if that isn't possible how would I get ghci to load all of GHC in interpreted mode (intead of using -package ghc)? Currently I'm using trace & friends to do printf-style debugging but it's definitely not ideal. -Thomas Jakway

5 5

GHC 8.2.1 tree freeze timing
by Ben Gamari 10 Jan '17

10 Jan '17

tl;dr. The feature freeze for GHC 8.2 will happen on 30 January 2016. Get any patches you'd like to see in 8.2 up on Phabricator soon! Hello everyone, GHC 8.2.1 is quickly approaching. Our plan is to do a release candidate in February, so we would like to have the tree sorted out by the end of this month. Consequently, I would like to set a general feature freeze for 30 Janary 2016. After the freeze I will be much less likely to accept larger patches; ideally all work post-freeze should be in the name of preparing the tree for cutting the ghc-8.2 branch. If you are concerned that your work isn't positioned to meet this freeze date then please speak to me so we can make appropriate arrangements. Happy hacking! Cheers, - Ben

1 0

GHC 8.2.1 tree freeze timing
by Ben Gamari 10 Jan '17

10 Jan '17

Hello everyone, GHC 8.2.1 is quickly approaching. Our plan is to do a release candidate in February, so we would like to have the tree sorted out by the end of this month. Consequently, I would like to set a general feature freeze for 30 Janary 2016. If you are concerned that your work isn't positioned to meet this freeze date then please speak to me so we can make appropriate arrangements. Happy hacking! Cheers, - Ben

1 0

Navigating GHC proposals
by Simon Peyton Jones 09 Jan '17

09 Jan '17

Once I am looking the rendered form of a GHC proposal, eg https://github.com/ghc-proposals/ghc-proposals/blob/rae/constraint-vs-type/… how can I find my way to the “conversation” for that proposal, so I can comment on it? https://github.com/ghc-proposals/ghc-proposals/pull/32 Once more, I am lost in a maze of twisty little Githup passages. I clearly have not yet internalised an accurate model of what Github is thinking Thanks Simon

4 6

Phabricator upgrade underway
by Ben Gamari 09 Jan '17

09 Jan '17

Hello everyone, I'll be bringing down Phabricator for an upgrade in a few minutes. I'll let you know when things are back up. Cheers, - Ben

1 1

Lexical error in string continuation
by Harendra Kumar 07 Jan '17

07 Jan '17

Hi devs, I am making a change in runghc on the ghc master branch. When compiling the following code (edited/new code in utils/runghc): 208 splitGhcNonGhcArgs :: [String] -> IO ([String], [String]) 209 splitGhcNonGhcArgs args = do 210 let (ghc, other) = break notAFlag args 211 when (hasUnescapedGhcArgs ghc) $ 212 hPutStrLn stderr "yy\ 213 \ xx" I get an error because of the string continuation at line 212. If I put the backslashes on the same line I do not get any error. I have more string continuations in the same file and they all work fine. This snippet works fine with ghc-7.10.3/ghc-8.0.1 when compiled separately. Here is the error message that I get: utils/runghc/Main.hs:212:56: error: lexical error in string/character literal at character 'x' | 212 | hPutStrLn stderr "yy\ | ^ utils/runghc/ghc.mk:30: recipe for target 'utils/runghc/dist-install/build/Main.dyn_o' failed make[2]: *** [utils/runghc/dist-install/build/Main.dyn_o] Error 1 Makefile:122: recipe for target 'all_utils/runghc' failed make[1]: *** [all_utils/runghc] Error 2 make[1]: Leaving directory '/vol/hosts/cueball/workspace/play/ghc' ../../mk/sub-makefile.mk:50: recipe for target 'all' failed make: *** [all] Error 2 Any help will be appreciated. I can send the modified file if anyone wants to reproduce/debug. -harendra

1 1

GHC API 7.10 -> 8.0.1 (unusable due to missing or recursive dependencies)
by A.M. 07 Jan '17

07 Jan '17

Hello, I am migrating a use of the GHC API from GHC 7.10 to GHC 8.0.1 and have hit an issue whereby importing pre-compiled libraries spits out: package testhsscript-0.1-CQQKGzp5pWwBmTnAldv1Hk is unusable due to missing or recursive dependencies: Glob-0.7.13-2PN6d9dpHzz7DHotD0T0wu base-4.9.0.0 though these packages are indeed in the sandbox and listed with ExposePackage in the packageFlags. To be clear, a variant of this code worked as expected under GHC 7.10. I have created a simplified test case to demonstrate the issue: https://github.com/agentm/testhsscript Here is the meat of the problem: * lib/Test.hs - contains a module importing Glob with some functions of no consequence https://github.com/agentm/testhsscript/blob/master/lib/Test.hs * test.hs includes the GHC API call to load the precompiled Test module from the cabal sandbox https://github.com/agentm/testhsscript/blob/master/test.hs * cabal file has two targets: Library and executable "test"- the executable uses the GHC API to load the pacakge "testhsscript" https://github.com/agentm/testhsscript/blob/master/testhsscript.cabal For reference, here is the GHC API invocation which works under GHC 7.10: https://github.com/agentm/project-m36/blob/master/src/lib/ProjectM36/AtomFu… I suspect I am missing something basic that has changed in GHC 8 and would very much appreciate any tips. Here is the failing result of attempting to load the testhsscript package: Setting up HscEnv Using binary package database: /home/agentm/Dev/testhsscript/.cabal-sandbox/x86_64-linux-ghc-8.0.1-packages.conf.d/package.cache Using binary package database: /opt/ghc/8.0.1/lib/ghc-8.0.1/package.conf.d/package.cache loading package database /home/agentm/Dev/testhsscript/.cabal-sandbox/x86_64-linux-ghc-8.0.1-packages.conf.d package testhsscript-0.1-CQQKGzp5pWwBmTnAldv1Hk is unusable due to missing or recursive dependencies: Glob-0.7.13-2PN6d9dpHzz7DHotD0T0wu base-4.9.0.0 package Glob-0.7.13-2PN6d9dpHzz7DHotD0T0wu is unusable due to missing or recursive dependencies: base-4.9.0.0 containers-0.5.7.1 directory-1.2.6.2 dlist-0.8.0.2-GWAMmbX9rLg3tqrbOizHGv filepath-1.4.1.0 transformers-0.5.2.0 transformers-compat-0.5.1.4-G5tKvPrwhggJRvSwXNMs1N package ghc-paths-0.1.0.9-GIOnKzk0HmEBZ77Q1HsThK is unusable due to missing or recursive dependencies: base-4.9.0.0 package mtl-2.2.1-6qsR1PHUy5lL47Hpoa4jCM is unusable due to missing or recursive dependencies: base-4.9.0.0 transformers-0.5.2.0 package dlist-0.8.0.2-GWAMmbX9rLg3tqrbOizHGv is unusable due to missing or recursive dependencies: base-4.9.0.0 deepseq-1.4.2.0 package transformers-compat-0.5.1.4-G5tKvPrwhggJRvSwXNMs1N is unusable due to missing or recursive dependencies: base-4.9.0.0 ghc-prim-0.5.0.0 transformers-0.5.2.0 loading package database /opt/ghc/8.0.1/lib/ghc-8.0.1/package.conf.d test: <command line>: cannot satisfy dlist: dlist-0.8.0.2-GWAMmbX9rLg3tqrbOizHGv is unusable due to missing or recursive dependencies: base-4.9.0.0 deepseq-1.4.2.0 (use -v for more information) Cheers, M

2 3