- ghc-devs - Haskell.org

[ANNOUNCE] You should try Hadrian
by Andrey Mokhov 27 Jan '19

27 Jan '19

Dear GHC developers, Summary: You should try to use Hadrian as the GHC build system, because it will (hopefully!) become the default around GHC 8.8. What is Hadrian and how can I try it? ===================================== Hadrian is a new build system for GHC written in Haskell. It lives in the directory "hadrian" in the GHC tree, and we have been actively developing it in the past year to reach feature and correctness parity with the existing Make-based build system. While we haven't quite reached this goal (more on this below), Hadrian is already working well and we run Hadrian jobs alongside the Make ones in our CI pipelines since the recent move to GitLab. At this point, we would like to encourage everyone to try using Hadrian for their usual GHC development tasks. Hadrian's documentation resides in GHC's source tree, and below are the documents you will be most interested in: * https://gitlab.haskell.org/ghc/ghc/blob/master/hadrian/README.md: The root of Hadrian's documentation. It explains the basics and points to more specific documents where appropriate. * https://gitlab.haskell.org/ghc/ghc/blob/master/hadrian/doc/make.md: A cheatsheet-style document for GHC developers used to the Make build system (that is, most/all of you), showing equivalent Make/Hadrian commands for many tasks. * https://gitlab.haskell.org/ghc/ghc/blob/master/hadrian/doc/user-settings.md: A description of the "user settings" mechanism in Hadrian, which is where you can customise the build flavour, choose the packages to build, add file/package/platform-specific command line flags, etc. * https://gitlab.haskell.org/ghc/ghc/blob/master/hadrian/doc/testsuite.md: A description of the "test" rule and all the options it supports. The documentation can surely be improved, so please do not hesitate to send us feedback and suggestions here, or even better on Trac: make sure you choose the component "Build System (Hadrian)" when creating a new ticket. You need Hadrian ================ Hadrian is new, requires time to learn, and still has rough edges, but it has been developed to make your lives better. Here are a few advantages of Hadrian over the Make-based build system: 1) Hadrian is more reliable. Hadrian can capture build dependencies more accurately, which means you rarely (if ever) need to do a clean rebuild. 2) Hadrian is faster. Hadrian is faster for two reasons: (i) more accurate build dependencies, (ii) tracking of file contents instead of file modification times. Both allow you to avoid a lot of unnecessary rebuilds. Building Hadrian itself may take a while but needs to be done only once. 3) Hadrian is easier to understand and modify. You no longer need to deal with Make's global namespace of mutable string variables. Hadrian is written in the language you love; it has modules, types and pure functions. If you come across a situation where Hadrian is worse than the Make build system in any of the above aspects, this is a bug and you should report it. Helping Hadrian =============== The best way to help is to try Hadrian, and let us know how it goes, what doesn't work, what's missing for you, what you think should be easier, and so on. Below is a list of known issues that we are in the process of fixing or that we will be tackling soon: * Stage 2 GHC should be dynamically linked most of the time, but it never is, currently. See https://ghc.haskell.org/trac/ghc/ticket/15837 * There are about a dozen of failing tests in the GHC testsuite, some related to #15837. * Binary distributions haven't been thoroughly tested on many platforms (only some Linux flavours). There will definitely be some issues here. For example, the binary distribution rule currently fails on Windows: https://ghc.haskell.org/trac/ghc/ticket/16073. * There is no "validate" rule yet, only "test", but we have all the pieces to make this happen and it has a very high priority. * There are issues with building cross compilers: see https://ghc.haskell.org/trac/ghc/ticket/16051. We are likely missing some features compared to the Make build system, but none of them should take a lot of time to implement at this point. If you spot one, let us know! We'll do our best to implement it (or help you do it) as soon as we can. It is useful to look at the existing Hadrian tickets before submitting new ones, to make sure that the issue or idea that you would like to talk about hasn't been brought up yet: https://ghc.haskell.org/trac/ghc/query?status=!closed&component=Build+Syste…. Of course, we welcome your code contributions too! Several GHC developers have a good understanding of Hadrian codebase and will be able to help you. To find their names, have a look at the list of recent Hadrian commits: https://gitlab.haskell.org/ghc/ghc/commits/master/hadrian. As you can see, Hadrian is actively developed by many people, and we hope you will join too. Cheers, Andrey

2 1

Using same stdout/stderr files for multiple tests?
by Ömer Sinan Ağacan 27 Jan '19

27 Jan '19

I have a test that I want to run with different compile and runtime parameters. I managed to reuse the source file across different tests by adding a extra_files(['source.hs']) to the tests, but I don't know how to do the same for stdout/stderr files. Any ideas? In more details, I have test.hs test.stdout and two tests test('test', [extra_run_opts('...')], compile_and_run, []) test('test_debug', [extra_run_opts('...'), extra_hc_opts('-debug'), extra_files(['test.hs'])], compile_and_run, []) The first test works fine, but the second test fails because I don't know how to tell it to use test.stdout as the stdout file and it looks for test_debug.stdout. Thanks Ömer

3 4

Profuse apologies for email spamming
by Ben Gamari 27 Jan '19

27 Jan '19

Hello everyone, Just a moment ago mistakes were made during a test import of Trac tickets. Consequently you may have found that you have received errant email notifications from gitlab.staging.haskell.org. I have killed the responsible job and ensured that this shouldn't happen again. My apologies for any inconvenience this may have caused. Cheers, - Ben

1 0

Purpose of the CODEOWNERS file
by Matthew Pickering 26 Jan '19

26 Jan '19

There have been a flurry of MRs to add ownership to the CODEOWNERS file but it's not clear to me what this file does. It links to this page: https://docs.gitlab.com/ee/user/project/code_owners.html Which explains the syntax quite thoroughly but doesn't as far as I can see explain how this information is used? So before I add myself to some files, what does adding ownership actually do? Cheers, Matt

2 1

Getting a profiled GHC with hadrian.
by Andreas Klebinger 25 Jan '19

25 Jan '19

Hello Devs, What's the proper way to get a profiled GHC in hadrian? I tried hadrian/build.cabal.sh -j8 --flavour=Prof but that didn't work out: $ _build/stage1/bin/ghc.exe +RTS -p ghc.exe: the flag -p requires the program to be built with -prof ghc.exe: ... What am I doing wrong here? Cheers Andreas

2 1

Hadrian questions
by Richard Eisenberg 25 Jan '19

25 Jan '19

As suggested, I'm trying out Hadrian. I have a few questions. - After building GHC the first time, I often go into the /ghc directory and then do `make 2` to build just the stage-2 compiler. Is that now the same as `build --freeze1`? It would seem not (I haven't tested), because running `make 2` builds only the compiler, not the libraries. Can this workflow be replicated in Hadrian? - I have `userDefaultFlavour = "devel2"` in my hadrian/UserSettings.hs file. But I see build artifacts during compilation of stage-2 that have dyn_o and p_o extensions. I don't want these. Have I done something wrong? Or are these artifacts now necessary? - I have quite a few ghc directories. What are the dependencies of Hadrian so that I can install these into my global package database and avoiding rebuilding them for each ghc tree? (Please don't tell me that the global package database is bad for my health. I know that, and I know why, but it's still terribly convenient, and I'm happy to pay the occasional price for that convenience.) I'm sure I'll have more questions. Thanks! Richard

3 4

Marge: "CI is taking too long"
by Richard Eisenberg 25 Jan '19

25 Jan '19

Marge has complained that https://gitlab.haskell.org/rae/ghc/-/jobs/17206 is taking too long. And indeed it seems stuck. Help? Thanks, Richard

2 1

"resource exhausted" in CI
by Richard Eisenberg 24 Jan '19

24 Jan '19

Hi Ben, The pipeline for a recent patch (https://gitlab.haskell.org/rae/ghc/-/jobs/16793) says: HC [stage 1] compiler/stage2/build/TcForeign.o HC [stage 1] compiler/stage2/build/TcRules.o /tmp/ghc6477_0/ghc_3.s: hClose: resource exhausted (No space left on device) compiler/ghc.mk:445: recipe for target 'compiler/stage2/build/TcTyClsDecls.o' failed make[1]: *** [compiler/stage2/build/TcTyClsDecls.o] Error 1 make[1]: *** Waiting for unfinished jobs.... Is there a way to avoid this? Or should I just restart the CI? And what does Marge think of it all? (As in, if I ask her to take another spin, will she do so on the same commit?) Thanks, Richard

4 7

access failed CI artifacts
by Richard Eisenberg 24 Jan '19

24 Jan '19

A pipeline of mine failed: https://gitlab.haskell.org/rae/ghc/-/jobs/16807 The failures are just some output changes. I could delicately use the log to extract the changes and commit them, but it would be better to have direct access to, say, the *.run.stderr files and then commit those. Is this possible? Note that I do not have a build of this tree locally, as it's the result of a rebase that affected the output. Of course, I could build locally, but it would be nice to skip that step. Thanks! Richard

2 1

Groups and projects
by Simon Peyton Jones 24 Jan '19

24 Jan '19

> Alright, I believe I have found the issue: you are a member of > the GHC group and GitLab's default notification behavior is that > you will receive notifications for all events of repositories in > groups to which you belong. OK thanks, that's helpful. When you say "repository" could you also have said "project". That is, are "projects" and "repositories" in 1-1 correspondence? So, to understand the semantics, you can belong to * A project. * A group What's the difference? The group seems to be "Glasgow Haskell Compiler"; the project seems to be "Glasgow Haskell Compiler / GHC". Does that imply that * every project is part of a group; but * a group may have many projects? Or maybe a project can be part of more than one group? The "Custom settings" on a project seem to be identical to the "custom settings" for a group. If Gitlab is deciding whether to send me a notification N, is it clear whether it's a "project notification" or a "group notification"? If so, then it can consult the appropriate settings; but if not, it's not clear. Sorry for all these questions. It's astonishingly common for large software systems to describe the surface knobs without describing the underlying model that makes sense of all those knobs! Simon -----Original Message----- From: Ben Gamari <ben(a)well-typed.com> Sent: 24 January 2019 03:30 To: Simon Peyton Jones <simonpj(a)microsoft.com> Subject: RE: GHC | hWaitForInput-accurate-pipe test (!21) Simon Peyton Jones <simonpj(a)microsoft.com> writes: > Yes, tons of them! I attach a dozen that arrived in the last few hrs. > Alright, I believe I have found the issue: you are a member of the GHC group and GitLab's default notification behavior is that you will receive notifications for all events of repositories in groups to which you belong. You can view and modify these settings yourself here but I have changed them to be a bit more selective. You should now only receive notifications of new merge requests and new comments on merge requests on which you have previously commented. Do let me know if you receive any more unwanted mail. Cheers, - Ben

2 1