- ghc-devs - Haskell.org

Perf baselines
by Simon Peyton Jones 25 Aug '20

25 Aug '20

Ben CI is saying (on !3889) Missing Baseline Metrics these metrics trivially pass because a baseline (expected value) cannot be recovered from previous git commits. This may be due to HEAD having new tests or having expected changes, the presence of expected changes since the last run of the tests, and/or the latest test run being too old. T18223 T12425 T13253 Now it's true that T18223 is new, but T12425 and T13253 are not. What's wrong? Thanks Simon

1 0

Haskell Implementors' Workshop lightning talks
by Ben Gamari 24 Aug '20

24 Aug '20

Hello everyone, The Haskell Implementors' Workshop will be held this Friday, August 28th, colocated with virtual ICFP. As usual, we will have a number of slots for lightning-talks. These are five-minute-long talks where you can present libraries, language extension ideas, plugins, tools... anything that the Haskell implementor community might find interesting. We are collecting talk submissions via Google Forms [1] and will notify speakers of their slots on August 27th. We are looking forward to seeing you there! Cheers, - Ben [1] https://forms.gle/a8sZQNRRMe9H9YnX8

1 0

Breakage on master
by Ben Gamari 19 Aug '20

19 Aug '20

Hi all, It seems that a patch breaking master for DEBUG-enabled builds has snuck into `master`. I'm currently working on fixing the issue but in the meantime the issue can be worked around by reverting accbc242e555822a2060091af7188ce6e9b0144e. Cheers, - Ben

3 6

Call for GHC Maintainers
by Moritz Angermann 18 Aug '20

18 Aug '20

Hi there! As it stands right now, Ben is the one who works tirelessly trying to cut releases. Not just for the most recent version, but also for previous versions. Most recently 8.10.2, but we have 9.0 coming up as well. I know that there are some people who deeply care for personal or professional reasons for older releases, 8.4, 8.6, 8.8, ... Some of them have stacks of patches applied, or proprietary extensions. I'd argue that most of those applied patches are backports of bug fixes and rarely language features, as language features will break compatibility (due to ghc, base, and other library versions anyway). I would therefore like drum up a group of people who will take care (ideally 2+ per release) of backporting and making minor partch releases. This does not have to go on forever, but it would take much needed load off of Ben to focus on what ever happens in ghc HEAD. So what would this work actually look like? It would consist of - going through the list of MRs and tagging those which are relevant for backporting to a certain release. - backport MRs where the MR does not cleanly apply. - fixup any test-suite failures. - agree on a date to cut/make the release. This is not a permanent commitment. I hope we can attract more people to the ghc release managers. I'm looking forward to great many responses. And I'm sure Ben will be able to help mentor us through cutting the first releases. I'll volunteer to be part of the 8.6 branch maintainers for now. Cheers, Moritz PS: There is a slightly related discussion about release cadence and versions and how other projects deal with this in this ticket: https://gitlab.haskell.org/ghc/ghc/-/issues/18222

5 6

The curious case of #367: Infinite loops can hang Concurrent Haskell
by Moritz Angermann 17 Aug '20

17 Aug '20

Hi there! While working on a NCG, I eventually came across #367[0], which make GHC produce code that looks similar to this: ``` label: [non-branch-instructions]* brach-instruction label ``` so essentially an uninterruptible loop. The solution for GHC to produce code that can be interrupted is to pass -fno-omit-yields. So far so good. Out of curiosity, I did add a small piece of code to detect this to my NCG to complain if code like the above was generated[1]. Three weeks ago, I kind of maneuvered myself into a memory blow up corner, and then life happened, but this weekend I managed to find some time to revert some memory blow up and continue working on the NCG. Turns out I can build a stage2 "quick" flavour of the NCG without dynamic support just fine. I never saw the dead lock detection code fire. Now I did leave the test suite running yesterday night, and when looking through the test suite results, there were quite a few failure. Curiously a lot of them were due to ghc missing dynamic support (doh!). But also quite a few that failed due to the deadlock detection. T12485, hs_try_putmvar003, ds-wildcard, ds001, read029, T2817, tc011, tc021, T4524 So, my question then is this: are we fine with ghc generating this code? Or, if we are not, do we want to figure out if we can eliminate it? The issue 367 goes into quite a bit of detail why this is tricky to handle generally. Or should we add -fno-omit-yields to the test-cases? The ultimate option is to just turn of the detection, and I'm fine with doing so. However I'd rather ask if anyone sees value in detecting this or not. Cheers, Moritz -- [0]: https://gitlab.haskell.org/ghc/ghc/-/issues/367 [1]: https://gitlab.haskell.org/ghc/ghc/-/blob/46fba2c91e1c4d23d46fa2d9b18dcd000…

2 6

ApplicativeDo needs a bit of love
by Simon Marlow 12 Aug '20

12 Aug '20

Folks, ApplicativeDo does what it does quite well, indeed it's in heavy use doing what we built it for, and I was pleasantly surprised to see that 12% of people would even like it to be enabled by default <https://taylor.fausak.me/2019/11/16/haskell-survey-results/>. But a number of issues have arisen around it, some stemming from limitations in the original implementation and others due to interactions with other features that we didn't consider at the time. This message is just to point out that I'd be delighted if anyone wanted to jump on any of these tickets and work on them, as it's unlikely I'll find the bandwidth to work on them myself. All the tickets are tagged with ApplicativeDo: https://gitlab.haskell.org/ghc/ghc/-/issues?scope=all&utf8=%E2%9C%93&state=… Have at it! Cheers Simon

1 0

[ANNOUNCE] Glasgow Haskell Compiler 8.10.2 released
by Ben Gamari 08 Aug '20

08 Aug '20

Hello all, The GHC team is happy to announce the release of GHC 8.10.2. Source and binary distributions are available at the usual place: https://downloads.haskell.org/ghc/8.10.2/ GHC 8.10.2 fixes a number of issues in present in GHC 8.10.1 including: * Fixes a bug in process creation on Windows (#17926). * Fixes a code generation bug resulting in incorrect code generation for foreign calls with complex arguments (#18527) * Fixes a bug causing object merging to fail when the lld linker is in use (#17962) * Introduces a workaround for a Linux kernel bug in the implementation of the timerfd mechanism (#18033). * Fixes a few specialiser regressions (#17810, #18120) as well introduces a variety of miscellaneous specialiser improvements (#16473, #17930, #17966) * Fixes a potential loss of sharing due to left operator sections (#18151). * Fix bootstrapping of GHC with the LLVM backend on x86-64 (#17920). * A few important correctness fixes for the low-latency garbage collector. Users of `--nonmoving-gc` are strongly encouraged to upgrade promptly. Note that at the moment we still require that macOS Catalina users exempt the binary distribution from the notarization requirement by running `xattr -cr .` on the unpacked tree before running `make install`. This situation will hopefully be improved for GHC 9.0.1 with the resolution of #17418 [1]. Cheers, - Ben [1] https://gitlab.haskell.org/ghc/ghc/issues/17418

1 0

Question about binary distributions
by Mathieu Boespflug 07 Aug '20

07 Aug '20

Hi all, GHC currently has 3 tier-1 platforms: Linux, macOS and Windows. I'll focus the dicussion below on these three platforms. The binary distributions for Linux and macOS are designed to be unpacked, then the user types ./configure && make install. This is not the case for Windows. On all platforms it's possible to create "relocatable" installations, such that GHC doesn't really care where it's installed, and commands will still work if the install directory changes location on the filesystem. So my question is, why do we have a ./configure step on Linux and macOS? Why could we not have bindists for all platforms that work like the Windows one? I.e. a binary distribution that you just unpack, in any directory of your choice, without any configuration or installation step.

5 4

Re: [Haskell-cafe] STM friendly TreeMap (or similar with range scan api) ? WAS: Best ways to achieve throughput, for large M:N ratio of STM threads, with hot TVar updates?
by Compl Yue 06 Aug '20

06 Aug '20

Hi Devs & Cafe, I would report back my progress on it, actually I've got a rough conclusion that TL;DR: > For data-intensive workloads, x86_64 ISA has its cache of CPU chips being a hardware bottleneck, it's very hard to scale up with added number of cores, so long as they share the cache as being in a single chip. For the details - I developed a minimal script interpreter for diagnostic purpose, dependent only on libraries bundled with GHC, the source repository is at: https://github.com/complyue/txs <https://github.com/complyue/txs> I benchmarked on my machine with a single 6-core Xeon E5 CPU chip, for contention-free read/write performance scaling, got numbers at: https://github.com/complyue/txs/blob/master/results/baseline.csv <https://github.com/complyue/txs/blob/master/results/baseline.csv> conc thread avg tps scale eff populate 1 1741 1.00 1.00 2 1285 1.48 0.74 3 1028 1.77 0.59 4 843 1.94 0.48 5 696 2.00 0.40 6 600 2.07 0.34 scan 1 1565 1.00 1.00 2 1285 1.64 0.82 3 1018 1.95 0.65 4 843 2.15 0.54 5 696 2.22 0.44 6 586 2.25 0.37 The script is at: https://github.com/complyue/txs/blob/master/scripts/scan.txs <https://github.com/complyue/txs/blob/master/scripts/scan.txs> GHC cmdl is at: https://github.com/complyue/txs/blob/master/metric.bash <https://github.com/complyue/txs/blob/master/metric.bash> ghc --make -Wall -threaded -rtsopts -prof -o txs -outputdir . -stubdir . -i../src ../src/Main.hs && ( ./txs +RTS -N10 -A32m -H256m -qg -I0 -M5g -T -s <../scripts/"${SCRIPT}".txs ) I intended to use a single Haskell based process to handle meta data about many ndarrays being crunched, acting as a centralized graph database, as it turned out, many clients queued to query/insert meta data against a single database node, will create such high data throughput that just few CPU chips can't handle well, we didn't expect this but apparently we'll have to deploy more machines as for such a database instance, with data partitioned and distributed to more nodes for load balancing. (A single machine with many sockets for CPU thus many NUMA nodes is neither an option for us.) While the flexibility a central graph database would provide, is not currently a crucial requirement of our business, so we are not interested to further develop this database system. We currently have CPU intensive workloads handled by some cluster of machines running Python processes (crunching numbers with Numpy and C++ tensors), while some Haskell based number crunching software are still under development, it may turn out some day in the future, that some heavier computation be bound with the db access, effectively creating some CPU intensive workloads for the database functionality, then we'll have the opportunity to dive deeper into the database implementation. And in case more flexibility required in near future, I think I'll tend to implement embedded database instances in those worker processes, in contrast to centralized db servers. I wonder if ARM servers will have up scaling of data intensive workloads easier, though that's neither a near feasible option for us. Thanks for everyone that have been helpful! Best regards, Compl > On 2020-07-31, at 22:35, YueCompl via Haskell-Cafe <haskell-cafe(a)haskell.org> wrote: > > Hi Ben, > > Thanks as always for your great support! And at the moment I'm working on a minimum working example to reproduce the symptoms, I intend to work out a program depends only on libraries bundled with GHC, so it can be easily diagnosed without my complex env, but so far no reprod yet. I'll come with some piece of code once it can reproduce something. > > Thanks in advance. > > Sincerely, > Compl > > >> On 2020-07-31, at 21:36, Ben Gamari <ben(a)well-typed.com> wrote: >> >> Simon Peyton Jones via Haskell-Cafe <haskell-cafe(a)haskell.org> writes: >> >>>> Compl’s problem is (apparently) that execution becomes dominated by >>>> GC. That doesn’t sound like a constant-factor overhead from TVars, no >>>> matter how efficient (or otherwise) they are. It sounds more like a >>>> space leak to me; perhaps you need some strict evaluation or >>>> something. >>> >>> My point is only: before re-engineering STM it would make sense to get >>> a much more detailed insight into what is actually happening, and >>> where the space and time is going. We have tools to do this (heap >>> profiling, Threadscope, …) but I know they need some skill and insight >>> to use well. But we don’t have nearly enough insight to draw >>> meaningful conclusions yet. >>> >>> Maybe someone with experience of performance debugging might feel able >>> to help Compl? >>> >> Compl, >> >> If you want to discuss the issue feel free to get in touch on IRC. I >> would be happy to help. >> >> It would be great if we had something of a decision tree for performance >> tuning of Haskell code in the users guide or Wiki. We have so many tools >> yet there isn't a comprehensive overview of >> >> 1. what factors might affect which runtime characteristics of your >> program >> 2. which tools can be used to measure which factors >> 3. how these factors can be improved >> >> Cheers, >> >> - Ben >> _______________________________________________ >> Haskell-Cafe mailing list >> To (un)subscribe, modify options or view archives go to: >> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe >> Only members subscribed via the mailman list are allowed to post. > > _______________________________________________ > Haskell-Cafe mailing list > To (un)subscribe, modify options or view archives go to: > http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe > Only members subscribed via the mailman list are allowed to post.

1 0

Hi. I'm new to this mailing list and have a few questions.
by Anselm Schüler (conversations subemail) 04 Aug '20

04 Aug '20

8 13