March 2013 - ghc-devs

adding -ticky to, say, WAY_debug_HC_OPTS?
by Nicolas Frisby 27 Mar '13

27 Mar '13

(I admit that I remain intimidated by the build system.) I'm renabling some ticky counter bumps from within the RTS. The most direct path I see to this involves adding -ticky to makefile variables such as WAY_debug_HC_OPTS. (See the APPENDIX for why I need that.) It seems that the debug way is only used in the build system for building the RTS. Is that right? If so, will a flag added to the debug HC_OPTS vars in ways.mk only be used when compiling from .cmm sources? If so, my -ticky flag won't be putting ticky counters into any .o file where I'm not expecting them, right? So: 1) Is it a bad idea to add -ticky to the non-threaded and non-profiling WAY_debug*_HC_OPTS variables in ways.mk? 2) Alternative designs? A few come to mind, but they're more invasive/fragile. Thank you for your time. APPENDIX For example: * The (generated) definition of stg_ap_*_info uses BUILD_PAP * The definition of BUILD_PAP uses TICK_ALLOC_PAP * TICK_ALLOC_PAP is elaborated by CmmParse.y using StgCmmTicky.tickyAllocPAP * tickyAllocPAP emits no code unless Opt_Ticky is present. * -ticky is not currently in the relevant command lines

2 1

Could not find 'Prelude'
by Richard Eisenberg 26 Mar '13

26 Mar '13

On HEAD, commit b9d53..., a freshly built ghc-stage2 can't find its Prelude: > rae:22:06:54 ~/temp> ~/Documents/ghc-baseline/inplace/bin/ghc-stage2 Scratch.hs > > Scratch.hs:1:1: > Could not find module ‛Prelude’ > There are files missing in the ‛base’ package, > try running 'ghc-pkg check'. > Use -v to see a list of the files searched for. > I'm building on Mac OS 10.7.5, and my build.mk is set to 'quick'. Any ideas? Thanks, Richard

2 3

An idea for improving workflow
by Jan Stolarek 26 Mar '13

26 Mar '13

I've been reading the ghc-devs list for quite some time now and I noticed that it often happens that one of the developers pushes the changes to the ghc.git repo but forgets to push the corresponding changes to the testsuite, which results in people trying to figure out why their validation fails. I was thinking if this problem could be eliminated. Perhaps it would make sense to add a server-side hook to the ghc.git repo that would display a message reminding to push the changes to the testsuite? That's definitely not an ultimate solution to the problem, but I think it could reduce the number of such situations. I played a bit with server-side hooks in git to see how that looks. The message is interspersed with other output from git, but I think we can make it visible enough. The message can even be displayed in color, though I am not sure how portable is this accross different terminals. Janek

2 4

Compiling Prelude.hs or complete base to external core
by Carlos Lopez Camey 26 Mar '13

26 Mar '13

Hi, I have an interpreter for external core and I would like to read the definitions in base:Prelude. After jumping over some obstacles [0], I find myself here: > ghc --make -fext-core -I./include -cpp Prelude.hs GHC/Num.lhs:65:27: Could not deduce (base:GHC.Num.Num a) arising from the literal `0' from the context (Num a) bound by the class declaration for `Num' at GHC/Num.lhs:(42,1)-(65,31) Possible fix: add (base:GHC.Num.Num a) to the context of the class declaration for `Num' In the first argument of `(-)', namely `0' In the expression: 0 - x In an equation for `negate': negate x = 0 - x If I jump over this one with some nonsense fix (e.g. negate = id) then I get GHC/Exception.lhs:47:14: Cannot derive well-kinded instance of form `Typeable * (SomeException ...)' Class `Typeable' expects an argument of kind `k' In the data declaration for `SomeException' GHC/Exception.lhs:169:24: Cannot derive well-kinded instance of form `Typeable * (ErrorCall ...)' Class `Typeable' expects an argument of kind `k' In the newtype declaration for `ErrorCall' GHC/Exception.lhs:186:22: Cannot derive well-kinded instance of form `Typeable * (ArithException ...)' Class `Typeable' expects an argument of kind `k' In the data declaration for `ArithException' Do you have any tips that help me? I am using the HEAD version of base and GHC 7.6.2. I also tried base-4.6.0.0 but the file HsFFI.h is looked upon and is not found or is not generated by ./configure (ieee-flpt.h: is a similar case, although I could find this one on the web [1]). [0] Manually compiled GHC/Event/{Poll,Clock,EPoll} with hsc2hs [1] http://darcs.haskell.org/packages/base/include/ieee-flpt.h Thanks

2 1

Fwd: [Haddock] error: utils/haddock_dist_PROGNAME is not set
by Rob Stewart 25 Mar '13

25 Mar '13

Hi Ian, That may well be true. However, I ran multiple ./sync-all commands, due to the following error that I am seeing with the parallel library: $ ./sync-all -r http://darcs.haskell.org --extra get -b ghc-7.6 ... Cloning into 'libraries/process'... == libraries/process: running git config --local core.ignorecase true == running git clone http://darcs.haskell.org/packages/template-haskell.git libraries/template-haskell -b ghc-7.6 Cloning into 'libraries/template-haskell'... == libraries/template-haskell: running git config --local core.ignorecase true == running git clone http://darcs.haskell.org/packages/unix.git libraries/unix -b ghc-7.6 Cloning into 'libraries/unix'... == libraries/unix: running git config --local core.ignorecase true == running git clone http://darcs.haskell.org/packages/parallel.git libraries/parallel -b ghc-7.6 Cloning into 'libraries/parallel'... fatal: Remote branch ghc-7.6 not found in upstream origin Unexpected end of command stream git failed: 32768 at ./sync-all line 193. -- Rob On 24 March 2013 19:03, Ian Lynagh <ian(a)well-typed.com> wrote: > On Sun, Mar 24, 2013 at 07:50:34PM +0100, David Waern wrote: >> Not sure what's wrong here. Maybe someone on ghc-devs@ knows. >> >> 2013/3/24 Rob Stewart <robstewart57(a)gmail.com> >> >> > I am not able to build ghc 7.6 from source, and come across an error >> > in utils/haddock/ghc.mk >> > >> > utils/haddock/ghc.mk:28: *** utils/haddock_dist_PROGNAME is not set. > > I suspect that either your haddock repository is out-of-date, or that > some of your repositories are on the 7.6 branch and some are on master. > > > Thanks > Ian >

1 0

Re: [Haddock] error: utils/haddock_dist_PROGNAME is not set
by David Waern 24 Mar '13

24 Mar '13

Not sure what's wrong here. Maybe someone on ghc-devs@ knows. David 2013/3/24 Rob Stewart <robstewart57(a)gmail.com> > Hi, > > I am not able to build ghc 7.6 from source, and come across an error > in utils/haddock/ghc.mk > > $ cd ghc > $ ./sync-all -r http://darcs.haskell.org --extra get -b ghc-7.6 > $ ./sync-all -r http://darcs.haskell.org --extra get # get > libraries/parallel and libraries/xhtml > $ perl boot > $ ./configure --prefix=/home/foo/bar/ > $ make > + test -f mk/config.mk.old > + cp -p mk/config.mk mk/config.mk.old > touch -r mk/config.mk.old mk/config.mk > + test -f mk/project.mk.old > + cp -p mk/project.mk mk/project.mk.old > touch -r mk/project.mk.old mk/project.mk > + test -f compiler/ghc.cabal.old > + cp -p compiler/ghc.cabal compiler/ghc.cabal.old > touch -r compiler/ghc.cabal.old compiler/ghc.cabal > ===--- building phase 0 > make -r --no-print-directory -f ghc.mk phase=0 phase_0_builds > libraries/hpc/ghc.mk:3: libraries/hpc/dist-boot/package-data.mk: No > such file or directory > libraries/Cabal/Cabal/ghc.mk:3: > libraries/Cabal/Cabal/dist-boot/package-data.mk: No such file or > directory > libraries/bin-package-db/ghc.mk:3: > libraries/bin-package-db/dist-boot/package-data.mk: No such file or > directory > libraries/hoopl/ghc.mk:3: libraries/hoopl/dist-boot/package-data.mk: > No such file or directory > libraries/transformers/ghc.mk:3: > libraries/transformers/dist-boot/package-data.mk: No such file or > directory > libraries/terminfo/ghc.mk:3: > libraries/terminfo/dist-boot/package-data.mk: No such file or > directory > utils/haddock/ghc.mk:28: *** utils/haddock_dist_PROGNAME is not set. > Stop. > > -- > Rob > > _______________________________________________ > Haddock mailing list > Haddock(a)projects.haskell.org > http://projects.haskell.org/cgi-bin/mailman/listinfo/haddock >

2 1

I/O manager: relying solely upon kqueue is not a safe way to go
by PHO 24 Mar '13

24 Mar '13

I found the HEAD stopped working on MacOS X 10.5.8 since the parallel I/O manager got merged to HEAD. Stage-2 compiler successfully builds (including Language.Haskell.TH.Syntax contrary to the report by Kazu Yamamoto) but the resulting binary is very unstable especially for ghci: % inplace/bin/ghc-stage2 --interactive GHCi, version 7.7.20130313: http://www.haskell.org/ghc/ :? for help Loading package ghc-prim ... linking ... done. Loading package integer-gmp ... linking ... done. Loading package base ... linking ... done. Prelude> <stdin>: hGetChar: failed (Operation not supported) So I took a dtruss log and found it was kevent(2) that returned ENOTSUP. GHC.Event.KQueue was just registering the stdin for EVFILT_READ, whose type was of course tty, and then kevent(2) said "tty is not supported". Didn't the old I/O manager do the same thing? Why was it working then? After a hard investigation, I concluded that the old I/O manager was not really working. It just looked fine but in fact wasn't. Here's an explanation: If a fd to be registered is unsupported by kqueue, kevent(2) returns -1 iff no incoming event buffer is passed together. Otherwise it successfully returns with an incoming kevent whose "flags" is EV_ERROR and "data" contains an errno. The I/O manager has always been passing a non-empty event buffer until the commit e5f5cfcd, while it wasn't (and still isn't) checking if a received event in fact represents an error. That is, the KQueue backend asks the kernel to monitor the stdin's readability. The kernel then immediately delivers an event saying ENOTSUP. The KQueue backend thinks "Hey, the stdin is now readable!" so it invokes a callback associated with the fd. The thread which called "threadWaitRead" is now awakened and performs a supposedly non-blocking read on the fd, which in fact blocks but works anyway. However the situation has changed since the commit e5f5cfcd. The I/O manager now registers fds without passing an incoming event buffer, so kevent(2) no longer successfully delivers an error event instead it directly returns -1 with errno set to ENOTSUP, hence the "Operation not supported" exception. The Darwin's kqueue has started supporting tty since MacOS X 10.7 (Lion), but I heard it still doesn't support some important devices like /dev/random. FreeBSD's kqueue has some difficulties too. It's no doubt kqueue is a great mechanism for sockets, but IMHO it's not something to use for all kinds of file I/O. Should we then try kqueue(2) first and fallback to poll(2) if failed? Sadly no. Darwin's poll(2) is broken too, and select(2) is the only method reliable: http://pod.tst.eu/http://cvs.schmorp.de/libev/ev.pod#OS_X_AND_DARWIN_BUGS I wrote a small program to test if a given stdin is supported by kqueue, poll and select: https://gist.github.com/phonohawk/5169980#file-kqueue-poll-select-cpp MacOS X 10.5.8 is hopelessly broken. We can't use anything other than select(2) for tty and other devices: https://gist.github.com/phonohawk/5169980#file-powerpc-apple-darwin9-8-0-txt FreeBSD 8.0 does support tty but not /dev/random. I don't know what about the latest FreeBSD 9.1: https://gist.github.com/phonohawk/5169980#file-i386-unknown-freebsd8-0-txt NetBSD 6.99.17 works perfectly here: https://gist.github.com/phonohawk/5169980#file-x86_64-unknown-netbsd6-99-17… Just for reference, Linux 2.6.16 surely doesn't have kqueue(2) but it supports poll(2)ing on devices: https://gist.github.com/phonohawk/5169980#file-x86_64-unknown-linux2-6-16-t… I accordingly suggest that we should have some means to run two independent I/O managers on BSD-like systems: KQueue for sockets, pipes and regular file reads, and Select for any other devices and regular file writes. Note also that no implementations of kqueue support monitoring writability of regular file writes, which also endangers the current use of kqueue-based I/O manager namely "threadWaitWrite". Any ideas? Thanks, PHO _______________________________________________________ - PHO - http://cielonegro.org/ OpenPGP public key: 1024D/1A86EF72 Fpr: 5F3E 5B5F 535C CE27 8254 4D1A 14E7 9CA7 1A86 EF72

7 23

Account on abbot
by Simon Peyton-Jones 23 Mar '13

23 Mar '13

Dear Paul Could you please give Nicolas Frisby an account on abbot, so he can commit to GHC? He'll send you his ssh public key. (Ian, is that all we need to let Nick commit?) Many thanks Simon

2 1

Re: Advance notice that I'd like to make Cabal depend on parsec
by Mark Lentczner 22 Mar '13

22 Mar '13

This thread is raising all sorts of questions for me: Is it essential, or even sensical, that the serialization format GHC needs for storing package info bear any relation to the human authored form? If not, the split out of the package types could be accomplished in a way where GHC uses simple show/read(P) style serialization for storage of package info, where as cabal-lib would use a lovely parsec parser for humans. I'd like this approach. The issue of putting the yet one more HP package into GHC's core packages is increasing the exposure of the difficulty of the current GHC/HP relationship. See also threads in HP's mailing list for why can't we bump some packages in GHC's core set for the next HP release. The split arrangement is strange because we have two groups making up what is in the HP, but they have different processes and aims. The complex technical relationship between the moving parts only heightens the difficulty. Perhaps the major cause is that because GHC is shipped as a library itself, it exposes all it's package dependencies. And as it is a large, and growing, piece of software, the list only wants to grow. But I wonder how often GHC is used as a library itself? If not often, then perhaps GHC should be shipped as two parts: Just a compiler (plus the small number of packages that the compiler forces), and ghc-lib as an optional, even separate, package - perhaps one with even a traditional way of depending on other packages. In otherwords, users that wanted to incorporate the ghc-lib into their programs would depend, and download, and configure, and build, ghc-lib indpenendant of the GHC binaries installed on their system. Perhaps then, GHC, the compiler, built from ghc-lib, would be bootstrapped not from the past compiler, but from the past HP..... Okay, perhaps that is all just fantasy. But, no other programming system operates the way we do. They all fall into one of two camps: - The dominant implementation is maintained, built, and shipped along with a large collection of "common packages". Examples: Python, Ruby, PHP, Java. - The dominant implementation is shipped as a bare tool, and large common libraries are maintained and shipped independently. Examples: C++ (think g++ and boost), JavaScript (think browsers, and jQuery). We are in the middle and, I think, experiencing growing pains because of it. - Mark On Sat, Mar 16, 2013 at 3:42 PM, dag.odenhall(a)gmail.com < dag.odenhall(a)gmail.com> wrote: > I'd love to have a proper parser and source-location-aware AST for sake of > editor/IDE tools, so +1 from me. If you don't end up doing this after all, > I'd still like to see your parser in a separate package, although I > understand if you don't feel like maintaining two parsers especially given > the tedious process for verifying they work similarly. I guess it could > still be useful in the same way we find haskell-src-exts useful despite > some incompatibilities with GHC. > > > On Thu, Mar 14, 2013 at 3:53 PM, Duncan Coutts < > duncan.coutts(a)googlemail.com> wrote: > >> Hi folks, >> >> I want to give you advance notice that I would like to make Cabal depend >> on parsec. The implication is that GHC would therefore depend on parsec >> and thus it would become a core package, rather than just a HP package. >> So this would affect both GHC and the HP, though I hope not too much. >> >> The rationale is that Cabal needs to parse things, like .cabal files and >> currently we do not have a decent parser in the core libraries. By >> decent I mean one that can produce error messages with source locations >> and that doesn't have unpredictable memory use. The only parser in the >> core libraries at the moment is Text.ParserCombinators.ReadP from the >> base package and that fails my "decent" criteria on both counts. Its >> idea of an error message is (), and on some largish .cabal files we take >> 100s of MB to parse (I realise that the ReadP in the base package is a >> cutdown version so I don't mean to malign all ReadP-style libs out >> there). >> >> Partly due to the performance problem, the terrible .cabal file error >> messages, and partly because Doaitse Swierstra keeps asking me if .cabal >> files have a grammar, I've been writing a new .cabal parser. It uses an >> alex lexer and a parsec parser. It's fast and the error messages are >> pretty good. I have reverse engineered a grammar that closely matches >> the existing parser and .cabal files in the wild, though I'm not sure >> Doaitse will be satisfied with the approach I've taken to handling >> layout. >> >> Why did I choose parsec? Practicality dictates that I can only use >> things in the core libraries, and the nearest thing we have to that is >> the parser lib that is in the HP. I tried to use happy but I could not >> construct a grammar/lexer combo to handle the layout (also, happy is not >> exactly known for its great error messages). >> >> I've been doing regression testing against hackage and I'm satisfied >> that the new parser matches close enough. I've uncovered all kinds of >> horrors with .cabal files in the wild relying on quirks of the old >> parser. I've made adjustments for most of them but I will be breaking a >> half dozen old packages (most of those don't actually build correctly >> because though their syntax errors are not picked up by the parser, they >> do cause failure eventually). >> >> So far I've just done the outline parser, not the individual field >> parsers. I'll be doing those next and then integrate. So this change is >> still a bit of a ways off, but I thought it'd be useful to warn people >> now. >> >> Duncan >> >> >> _______________________________________________ >> cabal-devel mailing list >> cabal-devel(a)haskell.org >> http://www.haskell.org/mailman/listinfo/cabal-devel >> > >

10 17

constraint deduction bug?
by Gabor Greif 22 Mar '13

22 Mar '13

In context of a bigger source file I got this error from GHC (HEAD) Could not deduce (Intersect [KeySegment] (BuriedUnder sub k ('Empty [KeySegment])) inv ~ 'Empty [KeySegment]) from the context (Intersect [KeySegment] (BuriedUnder sub k ('Empty [KeySegment])) inv ~ 'Empty [KeySegment]) bound by a pattern with constructor Nil :: forall (k :: BOX). Sing (Inventory k) ('Empty k), in a pattern binding in 'do' block Can somebody confirm that this constraint should be trivially deducible from the context? Then I'll try to come up with a reduced repro case. Cheers, Gabor

4 4