- Haskell-Cafe - Haskell.org

Speeding up compilation of a happy generated parser
by Nathan Bloomfield 15 May '19

15 May '19

Hello all! I'm using Alex/Happy to generate a parser for PHP. This is my first go at using a parser generator instead of parser combinators and it's been going pretty smoothly so far, but I've hit a snag. Compiling the happy-generated parser is excruciatingly slow -- 20+ minutes with -O0 -- which messes up the typically very fast feedback loop haskell+ghci allows. Running the code is great; my quickcheck test suite is very fast. But compiling and loading into ghci are so slow that iterating on the project further is not practical until I can figure out how to speed it up. Are there any folklore techniques for dealing with this problem, or any resources about how to profile/debug slow compilation? I'm on macos using Stack with lts-13.21, which includes happy-1.19.10. I tried the ideas in this known issue: https://github.com/simonmar/happy/issues/109. I also tried downgrading to lts-6.35 (the most recent with GHC 7.*), all with similar results. I imagine some of this slowdown is inevitable because of the grammar's size; it has ~275 nonterminals and the generated `Parser.hs` module is 3.8MB. Usually for a problem like this I'd break the module up into smaller pieces that won't change as often, but in this case it's an opaque monolith generated by happy. I also see that the generated code doesn't have type signatures on all top level functions. Here's the relevant output from GHC with -v2 enabled: ``` *** Parser [PHP.Parse.Parser]: !!! Parser [PHP.Parse.Parser]: finished in 1980.24 milliseconds, allocated 3155.733 megabytes *** Renamer/typechecker [PHP.Parse.Parser]: !!! Renamer/typechecker [PHP.Parse.Parser]: finished in 243923.86 milliseconds, allocated 178189.005 megabytes *** Desugar [PHP.Parse.Parser]: Result size of Desugar (before optimization) = {terms: 1,346,664, types: 672,082,252, coercions: 789, joins: 0/9,651} Result size of Desugar (after optimization) = {terms: 834,353, types: 285,053,589, coercions: 246,053, joins: 235/735} !!! Desugar [PHP.Parse.Parser]: finished in 103983.68 milliseconds, allocated 17597.213 megabytes *** Simplifier [PHP.Parse.Parser]: Result size of Simplifier iteration=1 = {terms: 1,059,012, types: 69,942,882, coercions: 1,934,739, joins: 235/1,935} Result size of Simplifier iteration=2 = {terms: 815,760, types: 64,805,160, coercions: 1,368,873, joins: 233/1,929} Result size of Simplifier = {terms: 776,400, types: 64,474,214, coercions: 1,297,320, joins: 233/1,929} !!! Simplifier [PHP.Parse.Parser]: finished in 197842.50 milliseconds, allocated 116983.797 megabytes *** CoreTidy [PHP.Parse.Parser]: Result size of Tidy Core = {terms: 776,400, types: 64,474,214, coercions: 1,297,320, joins: 233/1,929} !!! CoreTidy [PHP.Parse.Parser]: finished in 97776.92 milliseconds, allocated 11254.968 megabytes *** CorePrep [PHP.Parse.Parser]: Result size of CorePrep = {terms: 795,714, types: 66,237,082, coercions: 1,297,320, joins: 233/4,234} !!! CorePrep [PHP.Parse.Parser]: finished in 7911.78 milliseconds, allocated 2564.067 megabytes *** Stg2Stg: *** CodeGen [PHP.Parse.Parser]: !!! CodeGen [PHP.Parse.Parser]: finished in 7048.73 milliseconds, allocated 6407.563 megabytes *** Assembler: *** CorePrep [PHP.Parse.Parser]: Result size of CorePrep = {terms: 795,714, types: 66,237,082, coercions: 1,297,320, joins: 233/4,234} !!! CorePrep [PHP.Parse.Parser]: finished in 3069.67 milliseconds, allocated 2564.018 megabytes *** Stg2Stg: *** CodeGen [PHP.Parse.Parser]: !!! CodeGen [PHP.Parse.Parser]: finished in 5970.62 milliseconds, allocated 6634.422 megabytes *** Assembler: *** Deleting temp files: Warning: deleting non-existent /var/folders/td/sxyy9wl919740vddr49g8nth0000gn/T/ghc1272_0/ghc_70.s Warning: deleting non-existent /var/folders/td/sxyy9wl919740vddr49g8nth0000gn/T/ghc1272_0/ghc_72.c Warning: deleting non-existent /var/folders/td/sxyy9wl919740vddr49g8nth0000gn/T/ghc1272_0/ghc_74.c ``` -- Nathan Bloomfield Systems Analyst, Team Neutron Automattic, Inc.

1 0

Metaprogramming Summer School (August 2019): call for applications
by Jeremy Yallop 15 May '19

15 May '19

==================================================================== Second International Summer School on Metaprogramming Schloss Dagstuhl, Leibniz Center for Informatics, Germany 11th-16th August 2019 (the week before ICFP'19) https://www.cl.cam.ac.uk/events/metaprog/2019/ ==================================================================== Metaprogramming is an approach to constructing programs by treating program fragments (such as expressions or types) as values that the program can manipulate. Metaprogramming comes in various forms --- for example, * in dependently-typed programming terms appear within types, supporting the construction of precise specifications of functions and data. * in multi-stage programming expressions are program values, making it possible to write safe program generation programs that can significantly improve performance. * in languages with macros programs execute partly during compilation and partly at run-time, eliminating the sharp distinction between built-in and user-defined constructs. * embedded domain-specific languages reuse host language features such as syntax and type-checking for convenient definition of little languages suited to a particular endeavour. Metaprogramming has many applications, including genericity, proof automation, language extensibility and user-defined optimization. The goal of the summer school is to explore the state-of-the art in metaprogramming and its applications, covering both theory and practice. -------------------------------------------------------------------- Lecturers and courses Oleg Kiselyov (Tohoku University) From the tagless-final cookbook: simple hardware description language and optimization-by-evaluation Matthew Flatt (University of Utah) Building Languages with Racket Conor McBride (University of Strathclyde) TBD Jonathan Protzenko (Microsoft Research Redmond) Meta-F*: efficient meta-programming of the F* compiler at every stage -------------------------------------------------------------------- Prerequisites The school is aimed at graduate students in programming languages and related areas, but is open to researchers, practitioners and strong masters students with the support of a supervisor. Some experience of typed functional programming in Haskell, OCaml, Scala, or a similar language will be assumed. -------------------------------------------------------------------- Costs Thanks to the Schloss Dagstuhl subsidies, accommodation costs are as follows, and the dates are immediately before ICFP'19 (also in Germany): Single-occupancy accommodation: €420 Double-occupancy accommodation: €330 Accommodation costs include full board (in a single- or double-occupancy room, including meals during stay) from Sunday 11 August (evening) to Friday 16 August (afternoon). -------------------------------------------------------------------- Application procedure You will need to complete the online registration form at: https://www.cl.cam.ac.uk/events/metaprog/2019/application.html and ensure your referees send your references to: metaprog-2019(a)cl.cam.ac.uk by the application deadline. TIMETABLE * 30 June: Application and reference letters deadline. * 10 July: Notification of acceptance. * 11 August: Summer school. -------------------------------------------------------------------- Further information For any questions relating to the school, please contact the organisers (Jeremy Yallop, Ohad Kammar, Yukiyoshi Kameyama) at metaprog-2019(a)cl.cam.ac.uk

1 0

[ANN] stdio-0.2.0.0
by Dong Han 15 May '19

15 May '19

Dear Haskellers, Today I'm very happy to announce a new beta version of stdio is released on hackage <http://hackage.haskell.org/package/stdio>, there're some large additions in this version: - Add UDP module. - Add JSON module. - Add ToText class to Std.Data.TextBuilder module. The largest addition is a new JSON module, features full RFC 8259 compliance <https://github.com/haskell-stdio/JSONTestSuite> and 2~3x faster performance <https://github.com/haskell-stdio/stdio/tree/master/bench/json>. Try to bring big improvements over a very mature library like aeson is not an easy task, parts of the improvements come from faster Parser and Builder infrastructures in stdio. But we also made a few other improvements: - To improve building lookup table during JSON parsing(and many other formats as well), An ordered map module based on sorted vectors and binary-search is also added, there're actually four modules providing similar functions to containers: - Std.Data.Vector.FlatMap - Std.Data.Vector.FlatIntMap - Std.Data.Vector.FlatSet - Std.Data.Vector.FlatIntSet These data structures have different time and memory characteristics from tree-based ones, and suits some situations better, e.g. intermediate lookup table. - To improve JSON string parsing, we made an UTF-8 validation state machine with JSON unescaping logic built-in written in C, combine with the ability to scan a whole chunk of input(provided by our new Parser, namely scanChunks), we now have faster JSON string parsing performance than some native code(JSON.parse from nodejs). - Numeric parsing is also improved, e.g. we changed Integer decoding loop to use machine words if range permits, then covert the result to Integer rather than use Integer all the time. There're also lots of improvements and simplifications contribute to faster performance. Overall this is good proof that our new Parser and Builder infrastructures work well. Besides the JSON module, another important addition is the ToText class in Std.Data.TextBuilder, which is also derivable via generics. The format is chosen to follow Show class as much as possible, this is a very handy class when you need to quickly turn something into a builder or text. A UDP module is added quite easily since the IO manager part is in place. Due to the message based nature, the API is quite different from TCP, design suggestions are welcomed! Happy hacking! ~Cheers Han Dong

1 0

Scrap your boilerplate
by The FLOSS Information 15 May '19

15 May '19

4 4

You're invited to the New York Haskell CoHack
by Chris Smith 14 May '19

14 May '19

Fellow Haskellers, In cooperation with the New York Haskell User Group, I'm now organizing a monthly co-hacking event for Haskell programmers in New York. We are meeting on June 1st, and the first Saturday of each month after that. (However, to avoid the U.S. holiday weekend, we're meeting on June 29 instead of July 6.) More details and RSVP form at https://www.meetup.com/NY-Haskell/events/261245898/ This is an informal event where you can meet and collaborate with the rest of the Haskell community in New York. I am taking the format from the Atlanta Haskathon <https://www.meetup.com/Atlambda-Haskathon/>, where it's worked really well. We'll have time for lightning talks, and then open hacking time. To stay focused on project work, we will ask everyone to state a goal for the day, and check in on progress when we're wrapping up. Your choice of goal, though, is entirely up to you. You can use the time to collaborate on a project, get help learning Haskell or advice on using libraries or tools, help others, or just work by yourself. There will be refreshments provided courtesy of the Microsoft Reactor space.

1 0

Cabal revision online form silently converts file to Windows line endings, breaking patches
by Niklas Hambüchen 14 May '19

14 May '19

I noticed that if you use Hackage's online Cabal file editor, it changes all your \n line endings to \r\n. This breaks all patches written against upstream .cabal files. For example, if I write a patch against `cachix-0.2.0` git repo's .cabal file, I can no longer apply it to whatever revisions of it Hackage has, even if there's no real conflict. Here's a reproduction (I changed only 1 line in the editor, and it shows the whole file as changed): diff -u <(curl https://hackage.haskell.org/package/AesonBson-0.2.0/revision/0.cabal) <(curl https://hackage.haskell.org/package/AesonBson-0.2.0/revision/1.cabal) I found this especially problematic for nixpkgs, where patching packages is a common practice.

6 6

How to optimize a directory scanning?
by Magicloud Magiclouds 12 May '19

12 May '19

Hi, I have asked this in Stackoverflow without getting an answer. Wondering if people here could have some thoughts. I have a function reading the content of /proc every second. Surprisingly, its CPU usage in top is around 5%, peak at 8%. But same logic in C or Rust just takes like 1% or 2%. Wondering if this can be improved. /proc is virtual filesystem, so this is not related to HDD performance. And I noticed this difference because my CPU is too old (Core Gen2). On modern CPU, as tested by others, the difference is barely noticeable. import Control.Exception import Control.Concurrent import Control.Monad import Data.Char import Data.Maybe import System.Directory import System.FilePath import System.Posix.Files import System.Posix.Signals import System.Posix.Types import System.Posix.User import System.IO.Strict as Strict watch u limit0s limit0h = do listDirectory "/proc/" >>= mapM_ (\fp -> do isMyPid' <- maybe False id <$> wrap2Maybe (isMyPid fp u) wrap2Maybe (Strict.readFile ("/proc/" </> fp </> "stat"))) threadDelay 1000000 watch u limit0s limit0h where wrap2Maybe :: IO a -> IO (Maybe a) wrap2Maybe f = catch ((<$>) Just $! f) (\(_ :: IOException) -> return Nothing) isMyPid :: FilePath -> UserID -> IO Bool isMyPid fp me = do let areDigit = fp >= "0" && fp <= "9" isDir <- doesDirectoryExist $ "/proc/" </> fp owner <- fileOwner <$> getFileStatus ("/proc" </> fp) return $ areDigit && isDir && (owner == me) -- 竹密岂妨流水过山高哪阻野云飞

10 18

LFVM-STG Compilation
by james faure 10 May '19

10 May '19

Hello, I want to introduce the STG backend I am currently working on: https://github.com/jfaure/lfvm-stg I'd love to see how much interest there is for this, to receive any comments, and find out if I've missed anything ! The goal is to massively speed up compilation times and generate faster and clearer code, and in that respect I am already quite successful. Llvm is very well suited for encoding a (lazy) functional language, however the evaluation mechanism is very different to ghc's stg, there is notably no continuation passing nor result registers. In LFVM, all non constant let bindings are given an llvm function. I believe this approach is more natural, the llvm more accurately reflects the source code and we get proper stack traces in a debugger. Status: The project is barely a month old, and I'm working to support everything ghc's code generation supports. The STG is heavily based on the one found in ghc and, at this time and as far as I can tell, supports everything except multithreading, a gc and lazy evaluation. The current idea I think is very promising is to use an inferred pi calculus to elegantly implement all 3 missing parts at once. Pi calculus [1] (process calculus) In the STG * Multithreading occurs for arguments of functions with >1 arity * Perfect garbage collection is quite probably possible [2], where allocation creates a new pi name, and its uses are modeled by pi calculus communication * Perfect Lazy evaluation (in the sense that the wrapper and associated wrapped type overhead is coerced away after the first evaluation) is possible in the pi calculus and closely linked the gc model. I intend to make strict evaluation the default, If you want lazy (for MonadFix or infinite lists or whatever), you must request it. [1]: https://en.wikipedia.org/wiki/%CE%A0-calculus [2]: https://www.researchgate.net/publication/228514964_Resources_garbage-collec… James Faure (Discord: J4#0303)

6 7

Need help understanding Ralf Hinze's Lifting Lemma, and its connection to the Transfer Principle
by Brian Merchant 10 May '19

10 May '19

Hi all, I am a student in mathematics exploring alternative ways in which the transfer principle ( https://en.wikipedia.org/wiki/Transfer_principle ) can be proved. I've found a paper by Ralf Hinze which I think is relevant: https://www.cs.ox.ac.uk/ralf.hinze/Lifting.pdf. However, I'd benefit from being able to talk to someone who understands the notation starting section 3 and onwards, as I am very much a Haskell newbie (everything I have learned, has been in the last 4 days, as I try to decipher this paper). Would anyone be willing to have a conversation with me regarding this? Kind regards, Brian

2 1

FHPNC'19 Deadline Extension and 2nd Call for Papers
by Marco Zocca 09 May '19

09 May '19

We are happy to extend the submission deadline to the 2019 ACM SIGPLAN Functional High Performance and Numerical Computing Workshop until Saturday, May 25, 2019, 12 PM AoE ("anywhere on Earth", i.e. UTC-6). Marco Zocca and Dominic Steinitz FHPNC'19 Organizing Committee ================================= 2nd Call for Papers (Extended Abstracts and Full Papers) ACM SIGPLAN Workshop on Functional High-Performance and Numerical Computing (FHPNC), Berlin, August 18, 2019 Satellite event of the 24th ACM SIGPLAN International Conference on Functional Programming (ICFP 2019) 18 - 23 August, 2019 Homepage : https://icfp19.sigplan.org/home/FHPNC-2019#Call-for-Papers The ACM SIGPLAN International Workshop on Functional High-Performance and Numerical Computing aims to bring together researchers and practitioners exploring or employing the use of functional or declarative programming languages or techniques in scientific computing, and specifically in the domains of high-performance computing and numerical programming. The purpose of the meeting is to enable sharing of results, experiences, and novel ideas about how high-level, declarative techniques can help make high-performance, distributed/parallel, or numerically-intensive code dealing with computationally challenging problems easier to write, read, maintain, or portable to new hardware architectures. Areas of interest include, but are not limited to: * relevant compiler technologies * runtime systems (including fault tolerance mechanisms and those supporting distributed or parallel computation) * domain-specific languages (embedded or standalone) * type systems * formal methods * software libraries (e.g. for exact or interval arithmetic). ## Submission details Submissions should fall into one of two categories: * Regular research papers (up to 12 pages) * Extended abstracts (1 - 2 pages) The bibliography will not be counted against the page limits for either category. Regular research papers are expected to present novel and interesting research results, and will be included in the formal proceedings. Extended abstracts should report work in progress that the authors would like to present at the workshop; they will be evaluated primarily for relevance and interest. Extended abstracts will be distributed to workshop attendees but will not be published in the formal proceedings. We welcome submissions from PC members (with the exception of the PC Chair), but these submissions will be held to a higher standard. Submission is handled through the HotCRP site. All submissions should be in portable document format (PDF) and formatted using the ACM SIGPLAN style guidelines. Submissions written with LaTeX are required to use the ‘acmart’ format and the two-column ‘sigplan’ subformat (not to be confused with the one-column ‘acmlarge’ subformat!). Extended Abstracts must be submitted with the label ‘Extended abstract’ clearly in the title. ## Publication The proceedings of FHPNC 2019 will be published in the ACM Digital Library. ## Related links * Author Information and LaTeX templates : http://www.sigplan.org/Resources/Author/ * Attendee Code of Conduct: http://www.sigplan.org/Resources/Policies/CodeOfConduct/

1 0