- Haskell-Cafe - Haskell.org

Re: [Haskell-cafe] Monad transformer performance - Request to review benchmarking code + results
by Saurabh Nanda 29 Jan '17

29 Jan '17

Thank you for the PR. Does your research suggest something is wrong with HtmlT when combined with any MonadIO, not necessarily ActionT? Is this an mtl issue or a lucid issue in that case? Curiously, what's your machine config? I'm on a late 2011 macbook pro with 10G ram and some old i5. -- Saurabh. On 29 Jan 2017 3:05 pm, "David Turner" <dct25-561bs(a)mythic-beasts.com> wrote: The methodology does look reasonable, although I think you should wait for all the scotty threads to start before starting the benchmarks, as I see this interleaved output: Setting phasers to stun... (port 3002) (ctrl-c to quit) Setting phasers to stun... (port 3003) (ctrl-c to quit) Setting phasers to stun... (port 3001) (ctrl-c to quit) benchmarking bareScotty Setting phasers to stun... (port 3000) (ctrl-c to quit) Your numbers are wayyy slower than the ones I see on my dev machine: benchmarking bareScotty Setting phasers to stun... (port 3000) (ctrl-c to quit) time 10.94 ms (10.36 ms .. 11.52 ms) 0.979 R² (0.961 R² .. 0.989 R²) mean 12.53 ms (11.98 ms .. 13.28 ms) std dev 1.702 ms (1.187 ms .. 2.589 ms) variance introduced by outliers: 66% (severely inflated) benchmarking bareScottyBareLucid time 12.95 ms (12.28 ms .. 13.95 ms) 0.972 R² (0.951 R² .. 0.989 R²) mean 12.20 ms (11.75 ms .. 12.69 ms) std dev 1.236 ms (991.3 μs .. 1.601 ms) variance introduced by outliers: 50% (severely inflated) benchmarking transScottyBareLucid time 12.05 ms (11.70 ms .. 12.39 ms) 0.992 R² (0.982 R² .. 0.996 R²) mean 12.43 ms (12.06 ms .. 13.01 ms) std dev 1.320 ms (880.5 μs .. 2.071 ms) variance introduced by outliers: 54% (severely inflated) benchmarking transScottyTransLucid time 39.73 ms (32.16 ms .. 49.45 ms) 0.668 R² (0.303 R² .. 0.969 R²) mean 42.59 ms (36.69 ms .. 54.38 ms) std dev 16.52 ms (8.456 ms .. 25.96 ms) variance introduced by outliers: 92% (severely inflated) benchmarking bareScotty time 11.46 ms (10.89 ms .. 12.07 ms) 0.986 R² (0.975 R² .. 0.994 R²) mean 11.73 ms (11.45 ms .. 12.07 ms) std dev 800.6 μs (636.8 μs .. 975.3 μs) variance introduced by outliers: 34% (moderately inflated) but nonetheless I do also see the one using renderTextT to be substantially slower than the one without. I've sent you a PR [1] that isolates Lucid from Scotty and shows that renderTextT is twice as slow over IO than it is over Identity, and it's ~10% slower over Reader too: benchmarking renderText time 5.529 ms (5.328 ms .. 5.709 ms) 0.990 R² (0.983 R² .. 0.995 R²) mean 5.645 ms (5.472 ms .. 5.888 ms) std dev 593.0 μs (352.5 μs .. 908.2 μs) variance introduced by outliers: 63% (severely inflated) benchmarking renderTextT Id time 5.439 ms (5.243 ms .. 5.640 ms) 0.991 R² (0.985 R² .. 0.996 R²) mean 5.498 ms (5.367 ms .. 5.631 ms) std dev 408.8 μs (323.8 μs .. 552.9 μs) variance introduced by outliers: 45% (moderately inflated) benchmarking renderTextT Rd time 6.173 ms (5.983 ms .. 6.396 ms) 0.990 R² (0.983 R² .. 0.995 R²) mean 6.284 ms (6.127 ms .. 6.527 ms) std dev 581.6 μs (422.9 μs .. 773.0 μs) variance introduced by outliers: 55% (severely inflated) benchmarking renderTextT IO time 12.35 ms (11.84 ms .. 12.84 ms) 0.989 R² (0.982 R² .. 0.995 R²) mean 12.22 ms (11.85 ms .. 12.76 ms) std dev 1.159 ms (729.5 μs .. 1.683 ms) variance introduced by outliers: 50% (severely inflated) I tried replacing forM [1..10000] (\_ -> div_ "hello world!") with replicateM_ 10000 (div_ "hello world!") which discards the list of 10,000 () values that the forM thing generates, but this made very little difference. Hope this helps, David [1] https://github.com/vacationlabs/monad-transformer-benchmark/pull/2 On 29 January 2017 at 07:26, Saurabh Nanda <saurabhnanda(a)gmail.com> wrote: > Hi, > > I was noticing severe drop in performance when Lucid's HtmlT was being > combined with Scotty's ActionT. I've tried putting together a minimal repro > at https://github.com/vacationlabs/monad-transformer-benchmark Request > someone with better knowledge of benchmarking to check if the benchmarking > methodology is correct. > > Is my reading of 200ms performance penalty correct? > > -- Saurabh. > > > _______________________________________________ > Haskell-Cafe mailing list > To (un)subscribe, modify options or view archives go to: > http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe > Only members subscribed via the mailman list are allowed to post. >

3 2

Monad transformer performance - Request to review benchmarking code + results
by Saurabh Nanda 29 Jan '17

29 Jan '17

Hi, I was noticing severe drop in performance when Lucid's HtmlT was being combined with Scotty's ActionT. I've tried putting together a minimal repro at https://github.com/vacationlabs/monad-transformer-benchmark Request someone with better knowledge of benchmarking to check if the benchmarking methodology is correct. Is my reading of 200ms performance penalty correct? -- Saurabh.

2 1

Munich Haskell Meeting, 2017-01-30 @ 19:30
by Christian Neukirchen 29 Jan '17

29 Jan '17

Dear all, Next week, our monthly Munich Haskell Meeting will take place again on Monday, January 28 at Augustiner-Gaststätte Rumpler at 19h30. For details see here: http://muenchen.haskell.bayern/dates.html If you plan to join, please add yourself to this dudle so we can reserve enough seats! It is OK to add yourself to the dudle anonymously or pseudonymously. https://dudle.inf.tu-dresden.de/haskell-munich-jan-2017/ Everybody is welcome! cu, -- Christian Neukirchen <chneukirchen(a)gmail.com> http://chneukirchen.org

2 2

Desugaring ArrowLoop notation
by Francesco Ariis 28 Jan '17

28 Jan '17

Hello cafe, I was reading Stephen's Arrow Tutorial [1], where (among other things), a way to compute the running mean of a stream of integers using circular programming (ArrowLoop) is shown: mean5 :: Fractional a => Circuit a a mean5 = proc value -> do rec (lastTot, lastN) <- delay (0,0) -< (tot, n) let (tot, n) = (lastTot + value, lastN + 1) let mean = tot / n returnA -< mean To understand the example better I tried to convert the `rec do` notation to plain loop/(>>>) one (i.e. without syntactic sugar), but failed. So I ask: what's the sour version of `mean5`? -F [1] https://en.wikibooks.org/wiki/Haskell/Arrow_tutorial

2 2

Re: [Haskell-cafe] Instrumentation without requiring code-level changes?
by Saurabh Nanda 28 Jan '17

28 Jan '17

> > Yes, I think so, with the PackageImports extension. > > https://downloads.haskell.org/~ghc/7.8.2/docs/html/users_ > guide/syntax-extns.html#package-imports > Thank you for this. If this works, it makes using instrumentedscotty & instrumentedopaleye easier to use.. Btw, there's a note in the docs: [...] Note: you probably don't need to use this feature, it was added mainly so that we can build backwards-compatible versions of packages when APIs change. It can lead to fragile dependencies in the common case: modules occasionally move from one package to another, rendering any package-qualified imports broken. [...] I hope GHC doesn't plan to remove this extension any time soon. -- Saurabh.

2 4

hash tables
by Dennis Raddle 28 Jan '17

28 Jan '17

I want to build a hash table and then freeze it and use it for fast lookup later. Is there a way to build it in IO or ST but freeze it and use it in transparent code later? D

4 3

Re: [Haskell-cafe] Using stringize and string concatenation in ghc preprocessing
by Ben Franksen 27 Jan '17

27 Jan '17

Am 20.08.2016 um 20:33 schrieb Brandon Allbery: > You might want to look at cpphs as an alternative preprocessor. There are > some ancient K&R-era hacks that could be used if absolutely necessary, but > cpphs should be a much simpler and cleaner solution. Is there any reason /not/ to prefer cpphs over cpp? If not, why does anyone still use cpp for Haskell? Cheers Ben -- "Make it so they have to reboot after every typo." ― Scott Adams

12 36

Problem using external package from git with stack
by Juan Luis Martty 27 Jan '17

27 Jan '17

Hello, I am trying to use binary-serialise-cbor in my project. Since it's not on hackage I followed the docs and added the following to stack.yaml: packages: - '.' - location: git: https://github.com/well-typed/binary-serialise-cbor.git commit: 1e11ea5cc7810d241b2fe1bca0ed435baa4d4371 extra-dep: true flags: binary-serialise-cbor: newtime15: true And in my .cabal file I have: build-depends: base >= 4.7 && < 5, network-simple, time, binary-serialize-cbor, bytestring When I do stack build the package is downloaded and built but then I get the following error: Error: While constructing the build plan, the following exceptions were encountered: In the dependencies for hcopia-0.1.0.0: binary-serialize-cbor must match -any, but the stack configuration has no specified version I attached both files just in case. Regards, Juan

4 6

Re: [Haskell-cafe] Instrumentation without requiring code-level changes?
by David Turner 27 Jan '17

27 Jan '17

On 27 Jan 2017 07:16, "Saurabh Nanda" <saurabhnanda(a)gmail.com> wrote: (copying the mailing list -- I don't think your intention was to keep this off the mailing list) Apologies, yes, I failed to CC the mailing list. Thanks. Double checked that this one is addressed properly. I've looked at middleware for Scotty (or any WAI application), but it has limitations. I want deeper insights about how much time is being spent where, when serving a web request -- which is why the need to instrument SQL, Redis, and HTML rendering. One example that doesn't necessarily need to be in the IO monad is HTML rendering using Lucid, where the data is being passed separately to it (standard Controller <> View separation). Ok, this is definitely something that's hard to define, let alone measure: Lucid's rendering uses Bytestring.Builder to interleave the rendering and the transmission of traffic over the network, so a simple endTime-minus-startTime calculation doesn't work. You'd have to do something within ByteString.Builder itself to count the time spent filling each buffer vs the time spent sending it. I don't know of an existing implementation, but I can see how you'd do it (and it's worth noting that the relevant code is in IO, hidden behind a call to `unsafePerformIO` or similar): https://hackage.haskell.org/package/bytestring-0.10.8.1/docs/Data-ByteStrin… Also, if everything needs to be run in MonadIO, what does that do to fine-grained effect tracking? I wanted to have HasDb, HasRedis, HasEmail kind of monads (or type-classes) that give me some guarantees about the **type** of IO interactions a particular function call can do. If I instrument them and add an additional MonadIO constraint, does it mean any IO can be lifted into such monads? Your type classes can avoid mentioning IO, it's the instance definitions that would want to contain the instrumentation code. But they need IO anyway so that's no big deal. It's unfortunate to know that the RTS doesn't have these kind of instrumentation capabilities built-in. Is it absolutely not possible to do this at an RTS level? Ah, actually, there's the RTS event log: https://ghc.haskell.org/trac/ghc/wiki/EventLog and https://hackage.haskell.org/package/base-4.9.1.0/docs/Debug-Trace.html#g:2 But it emits a _lot_ of events from the RTS if you switch it on. Cheers, David -- Saurabh. On Fri, Jan 27, 2017 at 12:32 PM, David Turner < dct25-561bs(a)mythic-beasts.com> wrote: > Hi Saurabh, > > There's no direct equivalent to monkey-patching in Haskell (AFAIK (*)). > Why not instrument things always? The performance impact is usually quite > negligible, and the ability to switch instrumentation on at runtime in > production is sometimes invaluable. > > For instrumenting Scotty (in particular) or WAI-based apps (in general) > you can instrument things simply by inserting an appropriate `Middleware`, > a decision that can be deferred until service-startup time, and which > involves no change to the Scotty-side code. > > There's no equivalent to WAI for database access so I don't have a general > answer in that case, but typically database APIs have only a few core > `execute` or `query` functions that are quite straightforward to wrap with > instrumentation if needs be. > > This all kinda _has_ to be in IO anyway, as you're trying to measure a > real-world activity. Certainly there's no problem with either Scotty or > databases in this regard, as they're running in IO. Do you have an example > where you need instrumentation of a pure (i.e. non-IO) computation? In such > a case you could use the `unsafePerformIO` escape hatch (like `Debug.Trace` > does) but you'll have to be very careful to measure what you think you're > measuring: the language is quite free to reorder computations and avoid > repeating duplicated computations via referential transparency etc. > > (*) I mean, you could do something with dynamic libs and LD_PRELOAD (the > original monkey-patching) but please don't! > > Hope that helps, > > David > > On 27 Jan 2017 05:21, "Saurabh Nanda" <saurabhnanda(a)gmail.com> wrote: > >> (another cross post from Reddit - https://www.reddit.com/r/haske >> ll/comments/5qfd36/instrumentation_without_required_application/) >> >> Is it possible to collect instrumentation data from a Hakell application >> without requiring the core application to change code? Here are some >> examples of instrumentation data: >> >> * Time taken to serve an HTTP request (not average time, but individual >> times, for individual requests) along with the incoming request >> params+headers and outgoing response length. >> * Time taken to make an SQL query along with the actual query (again, not >> average times, but individual times, for individual queries) >> * Time taken to fetch data from Redis along with the Redis command >> * Time taken to render an HTML page along with path to the HTML file >> >> Most dynamic languages (like Ruby) allow monkey-patching OR "decorating" >> existing functions during runtime. Most instrumentation agents (like >> Skylight, Newrelic), simply monkey-patch the API surfaces of known >> libraries to collect this data. Therefore, using these instrumentation >> agents is a one line code change (just include the gem, or drop the JAR, or >> whatever). >> >> What is the equivalent of this in Haskell? >> >> Here's what I've tried so far [1] , but it has the following >> disadvantages: >> >> * Requires changes to the application's cabal file to replace the >> non-instrumented versions of the libraries with the instrumented version >> (replace scotty with instrumentedscotty, in my example) >> * Requires the application to explicitly import the instrumented versions >> of the libraries (replace import Scotty with import InstrumentedScotty, in >> my example) >> * Forces all the code interacting with instrumented APIs to be run in a >> MonadIO environment - because of the underlying calls to getCurrentTime >> >> I'm sure there's a better way, but I couldn't get my hands on it! >> >> [1] https://gist.github.com/saurabhnanda/27592da0269bc35569ec6239e1a91b75 >> >> -- Saurabh. >> >> _______________________________________________ >> Haskell-Cafe mailing list >> To (un)subscribe, modify options or view archives go to: >> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe >> Only members subscribed via the mailman list are allowed to post. >> > -- http://www.saurabhnanda.com

1 0

Re: [Haskell-cafe] Instrumentation without requiring code-level changes?
by Saurabh Nanda 27 Jan '17

27 Jan '17

(copying the mailing list -- I don't think your intention was to keep this off the mailing list) I've looked at middleware for Scotty (or any WAI application), but it has limitations. I want deeper insights about how much time is being spent where, when serving a web request -- which is why the need to instrument SQL, Redis, and HTML rendering. One example that doesn't necessarily need to be in the IO monad is HTML rendering using Lucid, where the data is being passed separately to it (standard Controller <> View separation). Also, if everything needs to be run in MonadIO, what does that do to fine-grained effect tracking? I wanted to have HasDb, HasRedis, HasEmail kind of monads (or type-classes) that give me some guarantees about the **type** of IO interactions a particular function call can do. If I instrument them and add an additional MonadIO constraint, does it mean any IO can be lifted into such monads? It's unfortunate to know that the RTS doesn't have these kind of instrumentation capabilities built-in. Is it absolutely not possible to do this at an RTS level? -- Saurabh. On Fri, Jan 27, 2017 at 12:32 PM, David Turner < dct25-561bs(a)mythic-beasts.com> wrote: > Hi Saurabh, > > There's no direct equivalent to monkey-patching in Haskell (AFAIK (*)). > Why not instrument things always? The performance impact is usually quite > negligible, and the ability to switch instrumentation on at runtime in > production is sometimes invaluable. > > For instrumenting Scotty (in particular) or WAI-based apps (in general) > you can instrument things simply by inserting an appropriate `Middleware`, > a decision that can be deferred until service-startup time, and which > involves no change to the Scotty-side code. > > There's no equivalent to WAI for database access so I don't have a general > answer in that case, but typically database APIs have only a few core > `execute` or `query` functions that are quite straightforward to wrap with > instrumentation if needs be. > > This all kinda _has_ to be in IO anyway, as you're trying to measure a > real-world activity. Certainly there's no problem with either Scotty or > databases in this regard, as they're running in IO. Do you have an example > where you need instrumentation of a pure (i.e. non-IO) computation? In such > a case you could use the `unsafePerformIO` escape hatch (like `Debug.Trace` > does) but you'll have to be very careful to measure what you think you're > measuring: the language is quite free to reorder computations and avoid > repeating duplicated computations via referential transparency etc. > > (*) I mean, you could do something with dynamic libs and LD_PRELOAD (the > original monkey-patching) but please don't! > > Hope that helps, > > David > > On 27 Jan 2017 05:21, "Saurabh Nanda" <saurabhnanda(a)gmail.com> wrote: > >> (another cross post from Reddit - https://www.reddit.com/r/haske >> ll/comments/5qfd36/instrumentation_without_required_application/) >> >> Is it possible to collect instrumentation data from a Hakell application >> without requiring the core application to change code? Here are some >> examples of instrumentation data: >> >> * Time taken to serve an HTTP request (not average time, but individual >> times, for individual requests) along with the incoming request >> params+headers and outgoing response length. >> * Time taken to make an SQL query along with the actual query (again, not >> average times, but individual times, for individual queries) >> * Time taken to fetch data from Redis along with the Redis command >> * Time taken to render an HTML page along with path to the HTML file >> >> Most dynamic languages (like Ruby) allow monkey-patching OR "decorating" >> existing functions during runtime. Most instrumentation agents (like >> Skylight, Newrelic), simply monkey-patch the API surfaces of known >> libraries to collect this data. Therefore, using these instrumentation >> agents is a one line code change (just include the gem, or drop the JAR, or >> whatever). >> >> What is the equivalent of this in Haskell? >> >> Here's what I've tried so far [1] , but it has the following >> disadvantages: >> >> * Requires changes to the application's cabal file to replace the >> non-instrumented versions of the libraries with the instrumented version >> (replace scotty with instrumentedscotty, in my example) >> * Requires the application to explicitly import the instrumented versions >> of the libraries (replace import Scotty with import InstrumentedScotty, in >> my example) >> * Forces all the code interacting with instrumented APIs to be run in a >> MonadIO environment - because of the underlying calls to getCurrentTime >> >> I'm sure there's a better way, but I couldn't get my hands on it! >> >> [1] https://gist.github.com/saurabhnanda/27592da0269bc35569ec6239e1a91b75 >> >> -- Saurabh. >> >> _______________________________________________ >> Haskell-Cafe mailing list >> To (un)subscribe, modify options or view archives go to: >> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe >> Only members subscribed via the mailman list are allowed to post. >> > -- http://www.saurabhnanda.com

1 1