- Glasgow-haskell-users

GHC Performance / Replacement for R?
by Dominic Steinitz 31 Aug '16

31 Aug '16

I am trying to use Haskell as a replacement for R but running into two problems which I describe below. Are there any plans to address the performance issues I have encountered? 1. I seem to have to jump through a lot of hoops just to be able to select the data I am interested in. {-# LANGUAGE ScopedTypeVariables #-} {-# OPTIONS_GHC -Wall #-} import Data.Csv hiding ( decodeByName ) import qualified Data.Vector as V import Data.ByteString ( ByteString ) import qualified Data.ByteString.Char8 as B import qualified Pipes.Prelude as P import qualified Pipes.ByteString as Bytes import Pipes import qualified Pipes.Csv as Csv import System.IO import qualified Control.Foldl as L main :: IO () main = withFile "examples/787338586_T_ONTIME.csv" ReadMode $ \h -> do let csvs :: Producer (V.Vector ByteString) IO () csvs = Csv.decode HasHeader (Bytes.fromHandle h) >-> P.concat uvectors :: Producer (V.Vector ByteString) IO () uvectors = csvs >-> P.map (V.foldr V.cons V.empty) vec_vec <- L.impurely P.foldM L.vector uvectors print $ (vec_vec :: V.Vector (V.Vector ByteString)) V.! 17 print $ V.length vec_vec let rockspring = V.filter (\x -> x V.! 8 == B.pack "RKS") vec_vec print $ V.length rockspring Here's the equivalent R: df <- read.csv("787338586_T_ONTIME.csv") rockspring <- df[df$ORIGIN == "RKS",] 2. Now I think I could improve the above to make an environment that is more similar to the one my colleagues are used to in R but more problematical is the memory usage. * 112.5M file * Just loading the source into ghci takes 142.7M * > foo <- readFile "examples/787338586_T_ONTIME.csv" > length foo takes me up to 4.75G. But we probably don't want to do this! * Let's try again. * > :set -XScopedTypeVariables * > h <- openFile "examples/787338586_T_ONTIME.csv" ReadMode * > let csvs :: Producer (V.Vector ByteString) IO () = Csv.decode HasHeader (Bytes.fromHandle h) >-> P.concat * > let uvectors :: Producer (V.Vector ByteString) IO () = csvs >-> P.map (V.map id) >-> P.map (V.foldr V.cons V.empty) * > vec_vec :: V.Vector (V.Vector ByteString) <- L.impurely P.foldM L.vector uvectors * Now I am up at 3.17G. In R I am under 221.3M. * > V.length rockspring takes a long time to return 155 and now I am at 3.5G!!! In R > rockspring <- df[df$ORIGIN == "RKS",] seems instantaneous and now uses only 379.5M. * > length(rockspring) 37 > length(df$ORIGIN) 471949 i.e. there are 37 columns and 471,949 rows. Running this as an executable gives ~/Dropbox/Private/labels $ ./examples/BugReport +RTS -s ["2014-01-01","EV","20366","N904EV","2512","10747","1074702","30747", "BRO","Brownsville, TX","Texas","11298","1129803","30194", "DFW","Dallas/Fort Worth, TX","Texas","0720","0718", "-2.00","8.00","0726","0837","7.00","0855","0844","-11.00","0.00", "","0.00","482.00","","","","","",""] 471949 155 14,179,764,240 bytes allocated in the heap 3,378,342,072 bytes copied during GC 786,333,512 bytes maximum residency (13 sample(s)) 36,933,976 bytes maximum slop 1434 MB total memory in use (0 MB lost due to fragmentation) Tot time (elapsed) Avg pause Max pause Gen 0 26989 colls, 0 par 1.423s 1.483s 0.0001s 0.0039s Gen 1 13 colls, 0 par 1.005s 1.499s 0.1153s 0.6730s INIT time 0.000s ( 0.003s elapsed) MUT time 3.195s ( 3.193s elapsed) GC time 2.428s ( 2.982s elapsed) EXIT time 0.016s ( 0.138s elapsed) Total time 5.642s ( 6.315s elapsed) %GC time 43.0% (47.2% elapsed) Alloc rate 4,437,740,019 bytes per MUT second Productivity 57.0% of total user, 50.9% of total elapsed

3 3

Using stringize and string concatenation in ghc preprocessing
by Harendra Kumar 22 Aug '16

22 Aug '16

Hi, To reduce boilerplate code in an FFI implementation file I am trying to use the stringizing and string concatenation features of the C preprocessor. Since ghc passes '-traditional' to the preprocessor which disables these features I thought I can pass my own flags to the preprocessor like this: {-# OPTIONS_GHC -optP -E -optP -undef #-} But "-optP" seems to only append to the flags that GHC already passes and gcc has no "-no-traditional" option to undo the effect of the "-traditional" that GHC has already passed. I think "-optP" should override the flags passed by ghc rather than appending to them. Is there a reason not to do that? Is there any other better way to achieve this? What is the standard way of doing this if any? -harendra

3 4

Behavior of GHC_PACKAGE_PATH
by Harendra Kumar 20 Aug '16

20 Aug '16

Hi, As per the GHC manual (https://downloads.haskell. org/~ghc/master/users-guide/packages.html#the-ghc-package- path-environment-variable), packages which come earlier in the GHC_PACKAGE_PATH supersede the ones which come later. But that does not seem to be the case always. I am dealing with a case where I have multiple versions of a package in different databases. I am passing the list of package dbs via GHC_PACKAGE_PATH and ghc is picking the one which comes later. I expect it to pick the one which comes earlier in the path. In one case the package being picked from a package db which comes later is a newer version which makes me wonder if GHC always prefers a newer version. In another case both the versions are same but still GHC is using the one which come later in the package path. Are there any undocumented factors into play here? None of the packages is broken (as reported by ghc-pkg check). See https://github.com/commercialhaskell/stack/issues/1957#issuecomment- 239655912 for more details on where it is coming from. Thanks, Harendra

2 2

ghci on pi zero
by Alex McLean 15 Aug '16

15 Aug '16

Hi all, I'm trying to get a working ghc on the pi zero, with its armv6l processor. I'm in need of ghci and/or the hint package, and raspbian only comes with a ghc without ghci and that is unable to compile hint. What's the current status of ghc (and in particular ghci) on the original pi and pi zero? Any tips on getting it running? Best wishes, alex

2 2

FINAL CALL FOR TALKS (Aug 8 deadline): Haskell Implementors Workshop 2016, Sep 24, Nara
by Edward Z. Yang 01 Aug '16

01 Aug '16

Deadline is in a week! Submit your talks! Call for Contributions ACM SIGPLAN Haskell Implementors' Workshop http://haskell.org/haskellwiki/HaskellImplementorsWorkshop/2016 Nara, Japan, 24 September, 2016 Co-located with ICFP 2016 http://www.icfpconference.org/icfp2016/ Important dates --------------- Proposal Deadline: Monday, 8 August, 2016 Notification: Monday, 22 August, 2016 Workshop: Saturday, 24 September, 2016 The 8th Haskell Implementors' Workshop is to be held alongside ICFP 2016 this year in Nara. It is a forum for people involved in the design and development of Haskell implementations, tools, libraries, and supporting infrastructure, to share their work and discuss future directions and collaborations with others. Talks and/or demos are proposed by submitting an abstract, and selected by a small program committee. There will be no published proceedings; the workshop will be informal and interactive, with a flexible timetable and plenty of room for ad-hoc discussion, demos, and impromptu short talks. Scope and target audience ------------------------- It is important to distinguish the Haskell Implementors' Workshop from the Haskell Symposium which is also co-located with ICFP 2016. The Haskell Symposium is for the publication of Haskell-related research. In contrast, the Haskell Implementors' Workshop will have no proceedings -- although we will aim to make talk videos, slides and presented data available with the consent of the speakers. In the Haskell Implementors' Workshop, we hope to study the underlying technology. We want to bring together anyone interested in the nitty-gritty details behind turning plain-text source code into a deployed product. Having said that, members of the wider Haskell community are more than welcome to attend the workshop -- we need your feedback to keep the Haskell ecosystem thriving. The scope covers any of the following topics. There may be some topics that people feel we've missed, so by all means submit a proposal even if it doesn't fit exactly into one of these buckets: * Compilation techniques * Language features and extensions * Type system implementation * Concurrency and parallelism: language design and implementation * Performance, optimisation and benchmarking * Virtual machines and run-time systems * Libraries and tools for development or deployment Talks ----- At this stage we would like to invite proposals from potential speakers for talks and demonstrations. We are aiming for 20 minute talks with 10 minutes for questions and changeovers. We want to hear from people writing compilers, tools, or libraries, people with cool ideas for directions in which we should take the platform, proposals for new features to be implemented, and half-baked crazy ideas. Please submit a talk title and abstract of no more than 300 words. Submissions should be made via HotCRP. The website is: https://icfp-hiw16.hotcrp.com/ We will also have a lightning talks session which will be organised on the day. These talks will be 5-10 minutes, depending on available time. Suggested topics for lightning talks are to present a single idea, a work-in-progress project, a problem to intrigue and perplex Haskell implementors, or simply to ask for feedback and collaborators. Organisers --- End forwarded message ---

1 0

Proposal process status
by Ben Gamari 21 Jul '16

21 Jul '16

Hello everyone, As you hopefully know, a few weeks ago we proposed a new process [1] for collecting, discussing, and deciding upon changes to GHC and its Haskell superset. While we have been happy to see a small contingent of contributors join the discussion, the number is significantly smaller than the set who took part in the earlier Reddit discussions. In light of this, we are left a bit uncertain of how to proceed. So, we would like to ask you to let us know your feelings regarding the proposed process: * Do you feel the proposed process is an improvement over the status quo? * Why? (this needn't be long, just a sentence hitting the major points) * What would you like to see changed in the proposed process, if anything? That's all. Again, feel free to reply either on the GitHub pull request [1] or this thread if you would prefer. Your response needn't be long; we just want to get a sense of how much of the community feels that 1) this effort is worth undertaking, and 2) that the proposal before us is in fact an improvement over the current state of affairs. Thanks for your help! Cheers, - Ben [1] https://github.com/ghc-proposals/ghc-proposals/pull/1

14 28

ArgumentDo proposal updated
by Akio Takano 13 Jul '16

13 Jul '16

Hi glasgow-haskell-users, Thank you for all the feedback to the ArgumentDo proposal. Following the discussion, I made changes to the proposal and updated the wiki page [0]. Now the proposed grammar is greatly simplified: it doesn't add a new non-terminal anymore, indeed it removes one instead. The proposed set of accepted programs remains unchanged. I hope the this update addresses one major concern that was raised in the previous discussion. Any feedback is appreciated. Regards, Takano Akio [0]: https://ghc.haskell.org/trac/ghc/wiki/ArgumentDo

6 5

Rethinking GHC's approach to managing proposals
by Ben Gamari 11 Jul '16

11 Jul '16

Hello everyone, Recently there has been a fair bit of discussion[1,2] around the mechanisms by which proposed changes to GHC are evaluated. While we have something of a formal proposal protocol [3], it is not clearly documented, inconsistently applied, and may be failing to serve a significant fraction of GHC's potential contributor pool. Over the last few weeks, I have been doing a fair amount of reading, thinking, and discussing to try to piece together a proposal scheme which better serves our community. The resulting proposal [4] is strongly inspired by the RFC process in place in the Rust community [5], the leaders of which have thought quite hard about fostering community growth and participation. While no process is perfect, I feel like the Rust process is a good starting point for discussion, offering enough structure to guide new contributors through the process while requiring only a modest investment of developer time. To get a sense for how well this will work in our community, I propose that we attempt to self-host the proposed process. To this end I have setup a ghc-proposals repository [6] and opened a pull request for discussion of the process proposal [4]. Let's see how this goes. Cheers, - Ben [1] https://www.reddit.com/r/haskell/comments/4oyxo2/blog_contributing_to_ghc/ [2] https://www.reddit.com/r/haskell/comments/4isua9/ghc_development_outsidein/ [3] https://ghc.haskell.org/trac/ghc/wiki/WorkingConventions/AddingFeatures [4] https://github.com/ghc-proposals/ghc-proposals/pull/1/files?short_path=14d6… [5] https://github.com/rust-lang/rfcs [6] https://github.com/ghc-proposals/ghc-proposals

5 4

Proposal: ArgumentDo
by Akio Takano 11 Jul '16

11 Jul '16

Hi glasgow-haskell-users, I have written a wiki page about a proposed extension called ArgumentDo. It's a small syntactic extension that allows "do" expressions, lambdas and a few other kinds of expressions to be used as function arguments, without parentheses. https://ghc.haskell.org/trac/ghc/wiki/ArgumentDo Any feedback is appreciated. In particular, since the idea has received mixed support (see the "Discussion" section on the wiki page), I'd like to make sure that there is enough support for this feature to justify an implementation in GHC. Regards, Takano Akio

15 36

Re: Glasgow-haskell-users Digest, Vol 155, Issue 9
by Ralf Hutchison 09 Jul '16

09 Jul '16

Well Iavor likes it. (see 2 below) Anyway, you know I'm really puzzled with the question of how to spend time. Maybe you can help me. In any case, it's such a lucky thing to have some to spend. Toshia and I are watching season 4 of Game of Thrones. I'm so addicted. I'm also addicted to speed chess in the same way. (I am ralf_ben_h at chess.com) I think that we have some structure in our minds, in our brains or our spines or our hearts say, and any story or game illuminates and awakens this structure. The addictive feeling or attachment is a fascination with this part of ourselves and a willingness to experience it. You wrote me a note about music recently. How are you feeling about music today? Best, Ralph On Sat, Jul 9, 2016 at 2:36 AM, <glasgow-haskell-users-request(a)haskell.org> wrote: > Send Glasgow-haskell-users mailing list submissions to > glasgow-haskell-users(a)haskell.org > > To subscribe or unsubscribe via the World Wide Web, visit > > http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users > > or, via email, send a message with subject or body 'help' to > glasgow-haskell-users-request(a)haskell.org > > You can reach the person managing the list at > glasgow-haskell-users-owner(a)haskell.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Glasgow-haskell-users digest..." > > > Today's Topics: > > 1. layout was Re: Proposal: ArgumentDo (C Maeder) > 2. Re: Proposal: ArgumentDo (Iavor Diatchki) > 3. Re: Proposal: ArgumentDo (C Maeder) > 4. Re: Proposal: ArgumentDo (Bardur Arantsson) > 5. Re: Proposal: ArgumentDo (Henrik Nilsson) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 8 Jul 2016 14:37:04 +0200 > From: C Maeder <chr.maeder(a)web.de> > To: glasgow-haskell-users(a)haskell.org > Subject: layout was Re: Proposal: ArgumentDo > Message-ID: <577F9E70.5090300(a)web.de> > Content-Type: text/plain; charset=utf-8 > > Hi, > > the layout language options are hard to find (at least in the user > guide). Therefore I try to give an overview here. The relevant options > I've found by using ghc-7.10.3 with option --supported-languages are: > > NondecreasingIndentation > DoAndIfThenElse > > RelaxedLayout > AlternativeLayoutRule > AlternativeLayoutRuleTransitional > > I ignore the last 3 options since these seem to be candidates for > removal: https://ghc.haskell.org/trac/ghc/ticket/11359. > > The default option is NondecreasingIndentation, that is > > do > x > do > y > > is legal and parsed as a single expression "do {x ; do y}". With > -XNoNondecreasingIndentation one gets an "Empty 'do' block" for the > second do (and this error would not change with ArgumentDo!). > > DoAndIfThenElse is not switched on by default and requires the keywords > "then" and "else" to be further indented for an "if" within a "do". > > So I believe these options do not interfere with ArgumentDo, but maybe > this should be discussed more and maybe mentioned on the wiki page. > > Surely a single space (inserted before x or removed before the second > "do") makes a difference between one or two argument expressions in the > example below. > > HTH Christian > > Am 08.07.2016 um 11:20 schrieb C Maeder: > > Surely layout can bite you: > > > > f > > do > > x > > do > > y > > > > and I'm having difficulties to find the documentation for the various > > layout options. > > > > But this is no argument against this proposal! > > > > Improper use of white spaces can always be used to obfuscate code! > > Style guides are important. Furthermore, a wrong/unintended parse tree > > (due to layout) will - in many cases - result in a typing error. > > > > Christian > > > > ------------------------------ > > Message: 2 > Date: Fri, 8 Jul 2016 09:42:24 -0700 > From: Iavor Diatchki <iavor.diatchki(a)gmail.com> > To: Aloïs Cochard <alois.cochard(a)gmail.com> > Cc: GHC users <glasgow-haskell-users(a)haskell.org>, Henrik Nilsson > <Henrik.Nilsson(a)nottingham.ac.uk> > Subject: Re: Proposal: ArgumentDo > Message-ID: > <CAGK9nuo93t=- > g9FktDG37iE7EeFwW6P5w5GOud3dmQkHA8oukg(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Hello, > > while we are voting here, I kind of like this proposal, so +1 for me. > > I understand that some of the examples look strange to Haskell old-timers > but, as Joachim points out, the behavior is very consistent. Besides, the > "Less Obvious Examples" were selected so that they are, well, less obvious. > The common use cases (as in ticket #10843) seem quite appealing, at least > to me, and not at all confusing. But, then, I also like the > records-with-no-parens notation :-) > > -Iavor > > > > On Fri, Jul 8, 2016 at 5:03 AM, Aloïs Cochard <alois.cochard(a)gmail.com> > wrote: > > > -1 for same reasons. > > > > On 8 July 2016 at 14:00, Henrik Nilsson <Henrik.Nilsson(a)nottingham.ac.uk > > > > wrote: > > > >> Hi all, > >> > >> Joachim Breitner wrote: > >> > >> > Am Freitag, den 08.07.2016, 13:09 +0200 schrieb Sven Panne: > >> > > I don't think so: https://ghc.haskell.org/trac/ghc > >> > /wiki/ArgumentDo#Bl > >> > [...] > >> > Where is the outer set of parenthesis coming from? > >> > > >> > This is all not related to the ArgumentDo notation. Note that [...] > >> > >> The very fact that that experts can't easily agree on how a small > >> Haskell fragment is parsed to me just confirms that Haskell already > >> is a syntactically very cumbersome language. > >> > >> The present proposal just makes matters worse. For that reason > >> alone, I don't find it compelling at all. (So -1 from me, then.) > >> > >> I will not repeat the many other strong arguments against that has been > >> made. But I must say I don't find the use cases as documented > >> on the associated web page compelling at all. Maybe there is a tacit > >> desire to be able to pretend functions are keywords for various > >> creative uses in supporting EDSLs and such. But for that to be truly > >> useful, one need to support groups of related keywords. Something > >> like Agda's mixfix syntax springs to mind. But this proposal does > >> not come close, so the benefits are minimal and the drawbacks large. > >> > >> As a final point, the inherent asymmetry of the proposal (the > >> last argument position is special as, for certain kinds of > >> expressions, parentheses may be omitted there but not elsewhere) > >> is also deeply unsettling. > >> > >> Best, > >> > >> /Henrik > >> > >> -- > >> Henrik Nilsson > >> School of Computer Science > >> The University of Nottingham > >> nhn(a)cs.nott.ac.uk > >> > >> > >> > >> > >> This message and any attachment are intended solely for the addressee > >> and may contain confidential information. If you have received this > >> message in error, please send it back to me, and immediately delete it. > >> Please do not use, copy or disclose the information contained in this > >> message or in any attachment. Any views or opinions expressed by the > >> author of this email do not necessarily reflect the views of the > >> University of Nottingham. > >> > >> This message has been checked for viruses but the contents of an > >> attachment may still contain software viruses which could damage your > >> computer system, you are advised to perform your own checks. Email > >> communications with the University of Nottingham may be monitored as > >> permitted by UK legislation. > >> > >> _______________________________________________ > >> Glasgow-haskell-users mailing list > >> Glasgow-haskell-users(a)haskell.org > >> http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users > >> > > > > > > > > -- > > *Λ\oïs* > > http://twitter.com/aloiscochard > > http://github.com/aloiscochard > > > > _______________________________________________ > > Glasgow-haskell-users mailing list > > Glasgow-haskell-users(a)haskell.org > > http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users > > > > >

1 1