January 2019 - ghc-devs

RE: [ANNOUNCE] You should try Hadrian
by Andrey Mokhov 29 Jan '19

29 Jan '19

Hi Tamar, Here is the relevant bullet point from the README: > On Windows, if you do not want to install MSYS, you can > use the Stack-based build script (Stack provides a managed > MSYS environment), as described in these instructions. > If you don't mind installing MSYS yourself or already have it, > you can use the Cabal-based build script. As you can see, `doc/windows.md` is recommended for people who “do not want to install MSYS”. Perhaps, the wording is not clear enough – please feel free to suggest improvements. > In order for ./boot and configure to work already you need to > be in an msys2 environment. Yes, and Stack provides it. This means, if you follow instructions in this file, you don’t need to install MSYS yourself. > There's a dubious claim there that using stack is > "more robust", what is this claim based on? This claim is based on my experience. Installing the MSYS environment has never worked out smoothly for me. Doing this via Stack was indeed more robust (especially, when struggling with building GHC on Windows CI!). Has this been different in your experience? > I'm just confused when it was decided to switch the defaults, > and why, without any consultation. I’m not sure what you mean. Could you clarify? The file `doc/windows.md` is 3 years old and hasn’t changed much since creation. The default build script `build.bat` currently uses Cabal: ``` rem By default on Windows we build Hadrian using Cabal hadrian/build.cabal.bat %* ``` P.S.: I’ve just noticed that `doc/windows.md` hasn’t been updated when moving to GitLab, and created this MR to fix this: https://gitlab.haskell.org/ghc/ghc/merge_requests/239 Please jump into the comments there if you’d like me to fix/clarify anything. Thanks for reaching out! Cheers, Andrey From: Phyx [mailto:lonetiger@gmail.com] Sent: 27 January 2019 21:11 To: Andrey Mokhov <andrey.mokhov(a)newcastle.ac.uk>; Ben Gamari <ben(a)well-typed.com> Cc: GHC developers <ghc-devs(a)haskell.org> Subject: Re: [ANNOUNCE] You should try Hadrian Hi Andrey, I'm looking at https://gitlab.haskell.org/ghc/ghc/blob/master/hadrian/README.md and https://gitlab.haskell.org/ghc/ghc/blob/master/hadrian/doc/windows.md wondering why the default instructions for Windows are using stack, this isn't currently the case. In order for ./boot and configure to work already you need to be in an msys2 environment. So having stack install its own, un-updated msys2 is not a workflow I would recommend. There's a dubious claim there that using stack is "more robust", what is this claim based on? I'm just confused when it was decided to switch the defaults, and why, without any consultation. Regards, Tamar

3 2

Re: Hadrian questions
by Andrey Mokhov 28 Jan '19

28 Jan '19

Hi Richard, First of all, thank you trying Hadrian and generating several new bug reports. This is very helpful! > I think it would be very convenient to make a script we can all install (in our PATH) that will > search for hadrian's build.sh and run it. Then, we can just say `build` (or whatever we name > the script) anywhere in a tree. Of course, I could just do this locally, but I doubt I'm the only > one who would enjoy it. This sounds like a good feature request to me. Could you please add it on Trac, so it's not lost? > Also, I just had a look at https://gitlab.haskell.org/ghc/ghc/blob/master/hadrian/doc/testsuite.md, > and I don't see a way to run just one directory of the testsuite. I use that ability currently quite > often (because I know that I've mucked with the typechecker, so I just run the typecheck tests > before doing full CI). Is this possible? Yes, this should be possible. I believe Alp is currently working on the testsuite with the goal of fully matching the functionality provided by the Make build system. I think this feature is already on his list, but I'm CC-ing him just in case. Cheers, Andrey ================================================ Date: Fri, 25 Jan 2019 08:18:08 -0500 From: Richard Eisenberg <rae(a)cs.brynmawr.edu> To: Matthew Pickering <matthewtpickering(a)gmail.com> Cc: GHC <ghc-devs(a)haskell.org> Subject: Re: Hadrian questions Maybe I'm just lazy, but would it be possible to shorten these? Specifically, could there be a root-level file `build` that triggers Hadrian? That way, I could just say ./build instead of hadrian/build.sh. Actually, even that isn't quite right. It is common, for example, for me to be deep in the testsuite, trying to fix a test. I twiddle something (say, the output file), and then I want to rerun the test. But now I have to go back out to the root of my tree to run the test, no? So: I think it would be very convenient to make a script we can all install (in our PATH) that will search for hadrian's build.sh and run it. Then, we can just say `build` (or whatever we name the script) anywhere in a tree. Of course, I could just do this locally, but I doubt I'm the only one who would enjoy it. Also, I just had a look at https://gitlab.haskell.org/ghc/ghc/blob/master/hadrian/doc/testsuite.md, and I don't see a way to run just one directory of the testsuite. I use that ability currently quite often (because I know that I've mucked with the typechecker, so I just run the typecheck tests before doing full CI). Is this possible? Thanks for all the work on this! Richard

2 1

[GitLab] Introducing Marge-bot
by Ben Gamari 28 Jan '19

28 Jan '19

Hi everyone, As you might have noticed there is a new face on GitLab: Meet @marge-bot. Marge will be helping us with the pain of merging merge requests: Currently the typical workflow to merge an accepted MR involves the following: 1. Rebase the MR on top of the current `master` branch 2. Click on the "Merge when pipeline succeeds" button 3. Wait. 4. If another MR is merged before yours, return to step (1) Given the volume of patches that we have, this process gets tiresome quite quickly. Upstream knows [1] about this issue and is actively working towards a solution which will likely be ready in a few months. In the meantime, Marge automates this currently-manual process. With Marge merging a patch involves just two steps: 1. Ensure that the MR has at least one approval. This should happen in the course of normal review but ping @bgamari, @alpmestan, @osa1, or @tdammers if this was forgotten. 2. Use the "Assignee" field in the sidebar on the right side of the MR to assign it to @marge-bot. Once Marge notices your MR she will dutifully watch over it, rebasing it as necessary until it is merged. If something goes awry, she will leave a (hopefully) helpful message and assign the MR back to you. So far Marge has been working out reasonably well and seems to be an improvement over the status quo. However, she still has some quirks so let me know if you think she is behaving erratically or otherwise have questions. Cheers, - Ben

5 4

CI request: a DEBUG compiler
by Richard Eisenberg 27 Jan '19

27 Jan '19

Hi devs, I almost exclusively work with a DEBUG compiler. It continually shocks me that I'm relatively alone in this stance, as the ASSERTs littered throughout GHC have caught many a bug in easier a manner than long stares at -ddump-tc-trace would do. In any case, my goal in this email is not to convince you to follow suit. My goal is to request that our CI infrastructure include at least one testsuite run in a DEBUG compiler. A patch I'm working on is failing a test because of an assertion failure. I doubt my patch is the culprit, but I also can't categorically rule it out. So, I'm left rebuilding HEAD with DEBUG turned on to see if I can repro the ASSERTion failure. Clearly, having a CI trace of this would simplify things -- not to mention catch bugs earlier. Thanks! Richard

3 2

[ANNOUNCE] You should try Hadrian
by Andrey Mokhov 27 Jan '19

27 Jan '19

Dear GHC developers, Summary: You should try to use Hadrian as the GHC build system, because it will (hopefully!) become the default around GHC 8.8. What is Hadrian and how can I try it? ===================================== Hadrian is a new build system for GHC written in Haskell. It lives in the directory "hadrian" in the GHC tree, and we have been actively developing it in the past year to reach feature and correctness parity with the existing Make-based build system. While we haven't quite reached this goal (more on this below), Hadrian is already working well and we run Hadrian jobs alongside the Make ones in our CI pipelines since the recent move to GitLab. At this point, we would like to encourage everyone to try using Hadrian for their usual GHC development tasks. Hadrian's documentation resides in GHC's source tree, and below are the documents you will be most interested in: * https://gitlab.haskell.org/ghc/ghc/blob/master/hadrian/README.md: The root of Hadrian's documentation. It explains the basics and points to more specific documents where appropriate. * https://gitlab.haskell.org/ghc/ghc/blob/master/hadrian/doc/make.md: A cheatsheet-style document for GHC developers used to the Make build system (that is, most/all of you), showing equivalent Make/Hadrian commands for many tasks. * https://gitlab.haskell.org/ghc/ghc/blob/master/hadrian/doc/user-settings.md: A description of the "user settings" mechanism in Hadrian, which is where you can customise the build flavour, choose the packages to build, add file/package/platform-specific command line flags, etc. * https://gitlab.haskell.org/ghc/ghc/blob/master/hadrian/doc/testsuite.md: A description of the "test" rule and all the options it supports. The documentation can surely be improved, so please do not hesitate to send us feedback and suggestions here, or even better on Trac: make sure you choose the component "Build System (Hadrian)" when creating a new ticket. You need Hadrian ================ Hadrian is new, requires time to learn, and still has rough edges, but it has been developed to make your lives better. Here are a few advantages of Hadrian over the Make-based build system: 1) Hadrian is more reliable. Hadrian can capture build dependencies more accurately, which means you rarely (if ever) need to do a clean rebuild. 2) Hadrian is faster. Hadrian is faster for two reasons: (i) more accurate build dependencies, (ii) tracking of file contents instead of file modification times. Both allow you to avoid a lot of unnecessary rebuilds. Building Hadrian itself may take a while but needs to be done only once. 3) Hadrian is easier to understand and modify. You no longer need to deal with Make's global namespace of mutable string variables. Hadrian is written in the language you love; it has modules, types and pure functions. If you come across a situation where Hadrian is worse than the Make build system in any of the above aspects, this is a bug and you should report it. Helping Hadrian =============== The best way to help is to try Hadrian, and let us know how it goes, what doesn't work, what's missing for you, what you think should be easier, and so on. Below is a list of known issues that we are in the process of fixing or that we will be tackling soon: * Stage 2 GHC should be dynamically linked most of the time, but it never is, currently. See https://ghc.haskell.org/trac/ghc/ticket/15837 * There are about a dozen of failing tests in the GHC testsuite, some related to #15837. * Binary distributions haven't been thoroughly tested on many platforms (only some Linux flavours). There will definitely be some issues here. For example, the binary distribution rule currently fails on Windows: https://ghc.haskell.org/trac/ghc/ticket/16073. * There is no "validate" rule yet, only "test", but we have all the pieces to make this happen and it has a very high priority. * There are issues with building cross compilers: see https://ghc.haskell.org/trac/ghc/ticket/16051. We are likely missing some features compared to the Make build system, but none of them should take a lot of time to implement at this point. If you spot one, let us know! We'll do our best to implement it (or help you do it) as soon as we can. It is useful to look at the existing Hadrian tickets before submitting new ones, to make sure that the issue or idea that you would like to talk about hasn't been brought up yet: https://ghc.haskell.org/trac/ghc/query?status=!closed&component=Build+Syste…. Of course, we welcome your code contributions too! Several GHC developers have a good understanding of Hadrian codebase and will be able to help you. To find their names, have a look at the list of recent Hadrian commits: https://gitlab.haskell.org/ghc/ghc/commits/master/hadrian. As you can see, Hadrian is actively developed by many people, and we hope you will join too. Cheers, Andrey

2 1

Using same stdout/stderr files for multiple tests?
by Ömer Sinan Ağacan 27 Jan '19

27 Jan '19

I have a test that I want to run with different compile and runtime parameters. I managed to reuse the source file across different tests by adding a extra_files(['source.hs']) to the tests, but I don't know how to do the same for stdout/stderr files. Any ideas? In more details, I have test.hs test.stdout and two tests test('test', [extra_run_opts('...')], compile_and_run, []) test('test_debug', [extra_run_opts('...'), extra_hc_opts('-debug'), extra_files(['test.hs'])], compile_and_run, []) The first test works fine, but the second test fails because I don't know how to tell it to use test.stdout as the stdout file and it looks for test_debug.stdout. Thanks Ömer

3 4

Profuse apologies for email spamming
by Ben Gamari 27 Jan '19

27 Jan '19

Hello everyone, Just a moment ago mistakes were made during a test import of Trac tickets. Consequently you may have found that you have received errant email notifications from gitlab.staging.haskell.org. I have killed the responsible job and ensured that this shouldn't happen again. My apologies for any inconvenience this may have caused. Cheers, - Ben

1 0

Purpose of the CODEOWNERS file
by Matthew Pickering 26 Jan '19

26 Jan '19

There have been a flurry of MRs to add ownership to the CODEOWNERS file but it's not clear to me what this file does. It links to this page: https://docs.gitlab.com/ee/user/project/code_owners.html Which explains the syntax quite thoroughly but doesn't as far as I can see explain how this information is used? So before I add myself to some files, what does adding ownership actually do? Cheers, Matt

2 1

Getting a profiled GHC with hadrian.
by Andreas Klebinger 25 Jan '19

25 Jan '19

Hello Devs, What's the proper way to get a profiled GHC in hadrian? I tried hadrian/build.cabal.sh -j8 --flavour=Prof but that didn't work out: $ _build/stage1/bin/ghc.exe +RTS -p ghc.exe: the flag -p requires the program to be built with -prof ghc.exe: ... What am I doing wrong here? Cheers Andreas

2 1

Hadrian questions
by Richard Eisenberg 25 Jan '19

25 Jan '19

As suggested, I'm trying out Hadrian. I have a few questions. - After building GHC the first time, I often go into the /ghc directory and then do `make 2` to build just the stage-2 compiler. Is that now the same as `build --freeze1`? It would seem not (I haven't tested), because running `make 2` builds only the compiler, not the libraries. Can this workflow be replicated in Hadrian? - I have `userDefaultFlavour = "devel2"` in my hadrian/UserSettings.hs file. But I see build artifacts during compilation of stage-2 that have dyn_o and p_o extensions. I don't want these. Have I done something wrong? Or are these artifacts now necessary? - I have quite a few ghc directories. What are the dependencies of Hadrian so that I can install these into my global package database and avoiding rebuilding them for each ghc tree? (Please don't tell me that the global package database is bad for my health. I know that, and I know why, but it's still terribly convenient, and I'm happy to pay the occasional price for that convenience.) I'm sure I'll have more questions. Thanks! Richard

3 4