Looks like my initial guess was completely wrong: on my system, almost no time is spent on the database query. Instead, it looks like there's some kind of performance issue with Hamlet. For example, changing line 20 of Handler/Quote.hs from "addWidget" to "addHtml" increases performance dramatically (25 sec to 4.5 sec). I'll need to spend some more time to properly diagnose the problem. Again, thanks for bringing this to my attention. Michael On Mon, Mar 21, 2011 at 12:40 PM, Michael Snoyman wrote:

Firstly, thank you for running this benchmark: it's the only way we can find out where Yesod needs to be fixed.

I haven't had a chance to look into the code properly, but it looks like this is more a test of the database backend than of the web framework. I'll try to get to this myself later, but it would be very interesting to see the difference in response times between PostgreSQL and sqlite. Currently, persistent-sqlite binds directly to the sqlite C API, while persistent-postgresql uses HDBC. It's entirely possible that HDBC is adding some overhead.

One of the items on our wish list[1] is to migrate away from HDBC. Having hard numbers like you are providing will help us figure out our priorities a bit better.

Thanks, Michael

[1] http://wiki.yesodweb.com/Wishlist

On Mon, Mar 21, 2011 at 12:10 PM, Sven Koschnicke (GFXpro) wrote:

...

Hello,

I'm coming from a Rails background and was very impressed by the features of the Yesod Web Framework. Working on an trading app in Rails I wanted to port some features of this app to a Yesod app just to learn the framework and because we have some performance issues with Rails.

The first thing I noticed was that the Yesod app was significantly slower than the Rails app at trivial tasks. I am very astonished about that. I thought Rails should be slower because of its interpreted nature. I made some benchmarking and measured that a simple Rails app is fourteen times faster than a Yesod app that did the same thing (loading some records from the database and displaying them). Did I made a mistake or is my understanding, that the compiled app should be faster, just wrong?

I documented the results on github:

https://github.com/SKoschnicke/performance-test

Greetings  Sven Koschnicke

_______________________________________________ web-devel mailing list web-devel@haskell.org http://www.haskell.org/mailman/listinfo/web-devel

This was a great example of a performance bug being exactly where you don't expect it. I'm planning on writing up a blog post on this adventure later, but for the quick summary: the Criterion benchmarks have gone from 23.7ms (== 23,700us) to 141us. Meaning this is a 200 fold speedup for some operations. This came from switching a large monad transformer stack to a single RWS monad. But more on that in the blog post. I've run your test a few more times: it still seems like there's a slowdown from the database code, but I think the widget performance bug was the big issue. After some more testing on my end, I'll put the code on Hackage. Would you give the test another shot after that? Thanks again for bringing this up! Michael On Mon, Mar 21, 2011 at 5:41 PM, Sven Koschnicke (GFXpro) wrote:

Hello,

I think your guess wasn't completely wrong. I tested with Sqlite now, and with that rails and yesod are equally slow (rails 11.81 secs vs yesod 11.36 secs). But its also interesting that your code change speeds it up.

(I put the sqlite benchmark at https://github.com/SKoschnicke/performance-test/tree/sqlite)

My first guess was actually that its the marshalling which slows yesod so much and that rails uses some clever optimizations there, but I haven't got enough insight to validate that.

Greetings  Sven

On 03/21/2011 01:49 PM, Michael Snoyman wrote:

...

Looks like my initial guess was completely wrong: on my system, almost no time is spent on the database query. Instead, it looks like there's some kind of performance issue with Hamlet. For example, changing line 20 of Handler/Quote.hs from "addWidget" to "addHtml" increases performance dramatically (25 sec to 4.5 sec). I'll need to spend some more time to properly diagnose the problem. Again, thanks for bringing this to my attention.

Michael

On Mon, Mar 21, 2011 at 12:40 PM, Michael Snoyman  wrote:

...

Firstly, thank you for running this benchmark: it's the only way we can find out where Yesod needs to be fixed.

I haven't had a chance to look into the code properly, but it looks like this is more a test of the database backend than of the web framework. I'll try to get to this myself later, but it would be very interesting to see the difference in response times between PostgreSQL and sqlite. Currently, persistent-sqlite binds directly to the sqlite C API, while persistent-postgresql uses HDBC. It's entirely possible that HDBC is adding some overhead.

One of the items on our wish list[1] is to migrate away from HDBC. Having hard numbers like you are providing will help us figure out our priorities a bit better.

Thanks, Michael

[1] http://wiki.yesodweb.com/Wishlist

On Mon, Mar 21, 2011 at 12:10 PM, Sven Koschnicke (GFXpro)  wrote:

...

Hello,

I'm coming from a Rails background and was very impressed by the features of the Yesod Web Framework. Working on an trading app in Rails I wanted to port some features of this app to a Yesod app just to learn the framework and because we have some performance issues with Rails.

The first thing I noticed was that the Yesod app was significantly slower than the Rails app at trivial tasks. I am very astonished about that. I thought Rails should be slower because of its interpreted nature. I made some benchmarking and measured that a simple Rails app is fourteen times faster than a Yesod app that did the same thing (loading some records from the database and displaying them). Did I made a mistake or is my understanding, that the compiled app should be faster, just wrong?

I documented the results on github:

https://github.com/SKoschnicke/performance-test

Greetings  Sven Koschnicke

_______________________________________________ web-devel mailing list web-devel@haskell.org http://www.haskell.org/mailman/listinfo/web-devel

I would definitely appreciate some help with the PostgreSQL backend. Here's the basic game plan I have in mind right now: * Split out the TH code into a separate package. This will be done for a 0.5 release. * Review Aur's join code. Most likely that will get included immediately. * Switch over the PostgreSQL backend from HDBC to the PostgreSQL C API. It's the last step that I would really appreciate help with. It looks like we might use the libpq package[1]. The only downside I see is that it has a unix package dependency, meaning it won't work on Windows. I see a few possibilities: * Don't worry about it. Windows users can't use PostgreSQL. (Bad option IMO.) * Get libpq to convert over to unix-compat if possible. * Maintain both the HDBC backend and a libpq backend. Frankly, given how Persistent is structured internally, it shouldn't be very complicated to make a migration to a different backend. If you'd like to take a crack at it, email me off-list about how to get started (or just fork the repo on Github and get going). Thanks, Michael [1] http://hackage.haskell.org/package/libpq On Wed, Mar 23, 2011 at 9:23 AM, Sven Koschnicke (GFXpro) wrote:

Great that you found a way to speed things up! I'm looking forward to the blog post. I will run some more tests with the improved code, just let me know when its available. Is someone working on improving the postgres database backend? Maybe I could help there, the bad performance with postgres is really a showstopper for me.

Sven

On 03/23/2011 08:13 AM, Michael Snoyman wrote:

...

This was a great example of a performance bug being exactly where you don't expect it. I'm planning on writing up a blog post on this adventure later, but for the quick summary: the Criterion benchmarks have gone from 23.7ms (== 23,700us) to 141us. Meaning this is a 200 fold speedup for some operations. This came from switching a large monad transformer stack to a single RWS monad. But more on that in the blog post.

I've run your test a few more times: it still seems like there's a slowdown from the database code, but I think the widget performance bug was the big issue. After some more testing on my end, I'll put the code on Hackage. Would you give the test another shot after that?

Thanks again for bringing this up!

Michael

On Mon, Mar 21, 2011 at 5:41 PM, Sven Koschnicke (GFXpro)  wrote:

...

Hello,

I think your guess wasn't completely wrong. I tested with Sqlite now, and with that rails and yesod are equally slow (rails 11.81 secs vs yesod 11.36 secs). But its also interesting that your code change speeds it up.

(I put the sqlite benchmark at https://github.com/SKoschnicke/performance-test/tree/sqlite)

My first guess was actually that its the marshalling which slows yesod so much and that rails uses some clever optimizations there, but I haven't got enough insight to validate that.

Greetings  Sven

On 03/21/2011 01:49 PM, Michael Snoyman wrote:

...

Looks like my initial guess was completely wrong: on my system, almost no time is spent on the database query. Instead, it looks like there's some kind of performance issue with Hamlet. For example, changing line 20 of Handler/Quote.hs from "addWidget" to "addHtml" increases performance dramatically (25 sec to 4.5 sec). I'll need to spend some more time to properly diagnose the problem. Again, thanks for bringing this to my attention.

Michael

On Mon, Mar 21, 2011 at 12:40 PM, Michael Snoyman  wrote:

...

Firstly, thank you for running this benchmark: it's the only way we can find out where Yesod needs to be fixed.

I haven't had a chance to look into the code properly, but it looks like this is more a test of the database backend than of the web framework. I'll try to get to this myself later, but it would be very interesting to see the difference in response times between PostgreSQL and sqlite. Currently, persistent-sqlite binds directly to the sqlite C API, while persistent-postgresql uses HDBC. It's entirely possible that HDBC is adding some overhead.

One of the items on our wish list[1] is to migrate away from HDBC. Having hard numbers like you are providing will help us figure out our priorities a bit better.

Thanks, Michael

[1] http://wiki.yesodweb.com/Wishlist

On Mon, Mar 21, 2011 at 12:10 PM, Sven Koschnicke (GFXpro)    wrote:

...

Hello,

I'm coming from a Rails background and was very impressed by the features of the Yesod Web Framework. Working on an trading app in Rails I wanted to port some features of this app to a Yesod app just to learn the framework and because we have some performance issues with Rails.

The first thing I noticed was that the Yesod app was significantly slower than the Rails app at trivial tasks. I am very astonished about that. I thought Rails should be slower because of its interpreted nature. I made some benchmarking and measured that a simple Rails app is fourteen times faster than a Yesod app that did the same thing (loading some records from the database and displaying them). Did I made a mistake or is my understanding, that the compiled app should be faster, just wrong?

I documented the results on github:

https://github.com/SKoschnicke/performance-test

Greetings  Sven Koschnicke

_______________________________________________ web-devel mailing list web-devel@haskell.org http://www.haskell.org/mailman/listinfo/web-devel

Hi Sven, Can I ask you to try things out one more time, this time with persistent-postgresql 0.4.0.1? I think I fixed some more of the issue[1]. Thanks, Michael [1] http://docs.yesodweb.com/blog/improving-persistent-performance On Wed, Mar 23, 2011 at 9:23 AM, Sven Koschnicke (GFXpro) wrote:

Great that you found a way to speed things up! I'm looking forward to the blog post. I will run some more tests with the improved code, just let me know when its available. Is someone working on improving the postgres database backend? Maybe I could help there, the bad performance with postgres is really a showstopper for me.

Sven

On 03/23/2011 08:13 AM, Michael Snoyman wrote:

...

This was a great example of a performance bug being exactly where you don't expect it. I'm planning on writing up a blog post on this adventure later, but for the quick summary: the Criterion benchmarks have gone from 23.7ms (== 23,700us) to 141us. Meaning this is a 200 fold speedup for some operations. This came from switching a large monad transformer stack to a single RWS monad. But more on that in the blog post.

I've run your test a few more times: it still seems like there's a slowdown from the database code, but I think the widget performance bug was the big issue. After some more testing on my end, I'll put the code on Hackage. Would you give the test another shot after that?

Thanks again for bringing this up!

Michael

On Mon, Mar 21, 2011 at 5:41 PM, Sven Koschnicke (GFXpro)  wrote:

...

Hello,

I think your guess wasn't completely wrong. I tested with Sqlite now, and with that rails and yesod are equally slow (rails 11.81 secs vs yesod 11.36 secs). But its also interesting that your code change speeds it up.

(I put the sqlite benchmark at https://github.com/SKoschnicke/performance-test/tree/sqlite)

My first guess was actually that its the marshalling which slows yesod so much and that rails uses some clever optimizations there, but I haven't got enough insight to validate that.

Greetings  Sven

On 03/21/2011 01:49 PM, Michael Snoyman wrote:

...

Looks like my initial guess was completely wrong: on my system, almost no time is spent on the database query. Instead, it looks like there's some kind of performance issue with Hamlet. For example, changing line 20 of Handler/Quote.hs from "addWidget" to "addHtml" increases performance dramatically (25 sec to 4.5 sec). I'll need to spend some more time to properly diagnose the problem. Again, thanks for bringing this to my attention.

Michael

On Mon, Mar 21, 2011 at 12:40 PM, Michael Snoyman  wrote:

...

Firstly, thank you for running this benchmark: it's the only way we can find out where Yesod needs to be fixed.

I haven't had a chance to look into the code properly, but it looks like this is more a test of the database backend than of the web framework. I'll try to get to this myself later, but it would be very interesting to see the difference in response times between PostgreSQL and sqlite. Currently, persistent-sqlite binds directly to the sqlite C API, while persistent-postgresql uses HDBC. It's entirely possible that HDBC is adding some overhead.

One of the items on our wish list[1] is to migrate away from HDBC. Having hard numbers like you are providing will help us figure out our priorities a bit better.

Thanks, Michael

[1] http://wiki.yesodweb.com/Wishlist

On Mon, Mar 21, 2011 at 12:10 PM, Sven Koschnicke (GFXpro)    wrote:

...

Hello,

I'm coming from a Rails background and was very impressed by the features of the Yesod Web Framework. Working on an trading app in Rails I wanted to port some features of this app to a Yesod app just to learn the framework and because we have some performance issues with Rails.

The first thing I noticed was that the Yesod app was significantly slower than the Rails app at trivial tasks. I am very astonished about that. I thought Rails should be slower because of its interpreted nature. I made some benchmarking and measured that a simple Rails app is fourteen times faster than a Yesod app that did the same thing (loading some records from the database and displaying them). Did I made a mistake or is my understanding, that the compiled app should be faster, just wrong?

I documented the results on github:

https://github.com/SKoschnicke/performance-test

Greetings  Sven Koschnicke

_______________________________________________ web-devel mailing list web-devel@haskell.org http://www.haskell.org/mailman/listinfo/web-devel

Let's say that we have a template [$hamlet|foo #{bar} baz|]. With addHamlet, we essentially end up with the code: addHamlet $ string "foo " `mappend` toHtml bar `mappend` string " baz" With addWidget, we end up with: addWidget $ do addHtml $ string "foo " addHtml $ toHtml bar addHtml $ string " baz" As some of the recent benchmarking blog posts should point out, operations in monad stacks (like addHamlet and addHtml) are more expensive. Since addWidget incurs three of those instead of just one, I'm not surprised it's slower. It would be nice to try and optimize away some of that overhead, but I'm not sure if it's possible. Overall, as funny as this sounds, I think allowing high-level Yesod code to be about on a par with high-level Rails is a good place to be right now. Yesod was written with the idea of *allowing* high performance by making correct architectural choices, such as relying on ByteString and Builder appropriately. However, it hasn't actually undergone significant performance tuning. We're now far enough along towards a stable API that we *have* started that optimization process: starting at the bottom of the software stack, just look at the Warp benchmark numbers. And even Yesod-the-framework beats out non-framework Ruby code. It seems that some of our higher level abstractions, like Persistent and Widgets, could stand to be optimized. The next release of Yesod will hopefully begin this process by replacing String with Text throughout the codebase. But to continue this properly, I'd really like help from the community. The more people who submit cases like Sven's where Yesod is simply performing slower than expected, the faster we'll track down inefficiencies. But just to drive home the point more: Rails is a framework that (to the best of my knowledge) has had many eyes staring at it and optimizing it. Yesod is relatively young and has not undergone much performance tuning. If we're already gaining a slight edge on Rails now, just imagine what will happen when our focus shifts to performance tuning between the 0.9 and 1.0 releases. Michael On Sat, Mar 26, 2011 at 8:38 PM, Greg Weber wrote:

using RAILS_ENV=production ./script/server ree 1.8.7 ruby-pg driver 21s ruby 1.9 pg driver 16-17s given there are a few time columns I tried "gem install home_run" and RAILS_ENV=production home_run ./script/server It didn't seem to make a difference- I think time object creation is lazy in Rails. Yesod with persistent-postgresql-0.4.0.1 gets 17s But then I looked at the code. For the template it is using: addWidget $(widgetFile "quote-index") I changed it too: addHamlet $(hamletFile "quote-index") and the time dropped down to 13-14s So finally, we are beating Rails- if we can't beat Rails on a benchmark that should be considered a horrible defect :) Here is a patch to fix widgetFile, although we should probably investigate why addWidget is slower. https://github.com/gregwebs/yesod/commit/32d11707f3d5bec73a5c3002f3924c58d82... On Fri, Mar 25, 2011 at 12:41 AM, Michael Snoyman wrote:

...

Hi Sven,

Can I ask you to try things out one more time, this time with persistent-postgresql 0.4.0.1? I think I fixed some more of the issue[1].

Thanks, Michael

[1] http://docs.yesodweb.com/blog/improving-persistent-performance

On Wed, Mar 23, 2011 at 9:23 AM, Sven Koschnicke (GFXpro) wrote:

...

Great that you found a way to speed things up! I'm looking forward to the blog post. I will run some more tests with the improved code, just let me know when its available. Is someone working on improving the postgres database backend? Maybe I could help there, the bad performance with postgres is really a showstopper for me.

Sven

On 03/23/2011 08:13 AM, Michael Snoyman wrote:

...

This was a great example of a performance bug being exactly where you don't expect it. I'm planning on writing up a blog post on this adventure later, but for the quick summary: the Criterion benchmarks have gone from 23.7ms (== 23,700us) to 141us. Meaning this is a 200 fold speedup for some operations. This came from switching a large monad transformer stack to a single RWS monad. But more on that in the blog post.

I've run your test a few more times: it still seems like there's a slowdown from the database code, but I think the widget performance bug was the big issue. After some more testing on my end, I'll put the code on Hackage. Would you give the test another shot after that?

Thanks again for bringing this up!

Michael

On Mon, Mar 21, 2011 at 5:41 PM, Sven Koschnicke (GFXpro)  wrote:

...

Hello,

I think your guess wasn't completely wrong. I tested with Sqlite now, and with that rails and yesod are equally slow (rails 11.81 secs vs yesod 11.36 secs). But its also interesting that your code change speeds it up.

(I put the sqlite benchmark at https://github.com/SKoschnicke/performance-test/tree/sqlite)

My first guess was actually that its the marshalling which slows yesod so much and that rails uses some clever optimizations there, but I haven't got enough insight to validate that.

Greetings  Sven

On 03/21/2011 01:49 PM, Michael Snoyman wrote:

...

Looks like my initial guess was completely wrong: on my system, almost no time is spent on the database query. Instead, it looks like there's some kind of performance issue with Hamlet. For example, changing line 20 of Handler/Quote.hs from "addWidget" to "addHtml" increases performance dramatically (25 sec to 4.5 sec). I'll need to spend some more time to properly diagnose the problem. Again, thanks for bringing this to my attention.

Michael

On Mon, Mar 21, 2011 at 12:40 PM, Michael Snoyman  wrote: > > Firstly, thank you for running this benchmark: it's the only way we > can find out where Yesod needs to be fixed. > > I haven't had a chance to look into the code properly, but it looks > like this is more a test of the database backend than of the web > framework. I'll try to get to this myself later, but it would be > very > interesting to see the difference in response times between > PostgreSQL > and sqlite. Currently, persistent-sqlite binds directly to the > sqlite > C API, while persistent-postgresql uses HDBC. It's entirely possible > that HDBC is adding some overhead. > > One of the items on our wish list[1] is to migrate away from HDBC. > Having hard numbers like you are providing will help us figure out > our > priorities a bit better. > > Thanks, > Michael > > [1] http://wiki.yesodweb.com/Wishlist > > On Mon, Mar 21, 2011 at 12:10 PM, Sven Koschnicke (GFXpro) >    wrote: >> >> Hello, >> >> I'm coming from a Rails background and was very impressed by the >> features of >> the Yesod Web Framework. Working on an trading app in Rails I >> wanted >> to >> port >> some features of this app to a Yesod app just to learn the >> framework >> and >> because we have some performance issues with Rails. >> >> The first thing I noticed was that the Yesod app was significantly >> slower >> than the Rails app at trivial tasks. I am very astonished about >> that. >> I >> thought Rails should be slower because of its interpreted nature. I >> made >> some benchmarking and measured that a simple Rails app is fourteen >> times >> faster than a Yesod app that did the same thing (loading some >> records >> from >> the database and displaying them). Did I made a mistake or is my >> understanding, that the compiled app should be faster, just wrong? >> >> I documented the results on github: >> >> https://github.com/SKoschnicke/performance-test >> >> Greetings >>  Sven Koschnicke >> >> _______________________________________________ >> web-devel mailing list >> web-devel@haskell.org >> http://www.haskell.org/mailman/listinfo/web-devel >> >

_______________________________________________ web-devel mailing list web-devel@haskell.org http://www.haskell.org/mailman/listinfo/web-devel

Thanks for the thorough benchmarking job, it's very nice to see that history. Is there a chance you could add Rails production values there as well? Michael On Mon, Mar 28, 2011 at 2:31 PM, Sven Koschnicke (GFXpro) wrote:

Hello,

I tested again with persistent-postgresql 0.4.0.1 and now the results are what you would expect :) rails in dev mode is now 79 times slower[1]. Of course, thats not a fair compare, but Greg did some testing with rails production already. Thanks for improving that so quickly!

Greetings  Sven

[1] https://github.com/SKoschnicke/performance-test

On 03/25/2011 08:41 AM, Michael Snoyman wrote:

...

Hi Sven,

Can I ask you to try things out one more time, this time with persistent-postgresql 0.4.0.1? I think I fixed some more of the issue[1].

Thanks, Michael

[1] http://docs.yesodweb.com/blog/improving-persistent-performance

On Wed, Mar 23, 2011 at 9:23 AM, Sven Koschnicke (GFXpro)  wrote:

...

Great that you found a way to speed things up! I'm looking forward to the blog post. I will run some more tests with the improved code, just let me know when its available. Is someone working on improving the postgres database backend? Maybe I could help there, the bad performance with postgres is really a showstopper for me.

Sven

On 03/23/2011 08:13 AM, Michael Snoyman wrote:

...

This was a great example of a performance bug being exactly where you don't expect it. I'm planning on writing up a blog post on this adventure later, but for the quick summary: the Criterion benchmarks have gone from 23.7ms (== 23,700us) to 141us. Meaning this is a 200 fold speedup for some operations. This came from switching a large monad transformer stack to a single RWS monad. But more on that in the blog post.

I've run your test a few more times: it still seems like there's a slowdown from the database code, but I think the widget performance bug was the big issue. After some more testing on my end, I'll put the code on Hackage. Would you give the test another shot after that?

Thanks again for bringing this up!

Michael

On Mon, Mar 21, 2011 at 5:41 PM, Sven Koschnicke (GFXpro)    wrote:

...

Hello,

I think your guess wasn't completely wrong. I tested with Sqlite now, and with that rails and yesod are equally slow (rails 11.81 secs vs yesod 11.36 secs). But its also interesting that your code change speeds it up.

(I put the sqlite benchmark at https://github.com/SKoschnicke/performance-test/tree/sqlite)

My first guess was actually that its the marshalling which slows yesod so much and that rails uses some clever optimizations there, but I haven't got enough insight to validate that.

Greetings  Sven

On 03/21/2011 01:49 PM, Michael Snoyman wrote:

...

Looks like my initial guess was completely wrong: on my system, almost no time is spent on the database query. Instead, it looks like there's some kind of performance issue with Hamlet. For example, changing line 20 of Handler/Quote.hs from "addWidget" to "addHtml" increases performance dramatically (25 sec to 4.5 sec). I'll need to spend some more time to properly diagnose the problem. Again, thanks for bringing this to my attention.

Michael

On Mon, Mar 21, 2011 at 12:40 PM, Michael Snoyman  wrote: > > Firstly, thank you for running this benchmark: it's the only way we > can find out where Yesod needs to be fixed. > > I haven't had a chance to look into the code properly, but it looks > like this is more a test of the database backend than of the web > framework. I'll try to get to this myself later, but it would be very > interesting to see the difference in response times between > PostgreSQL > and sqlite. Currently, persistent-sqlite binds directly to the sqlite > C API, while persistent-postgresql uses HDBC. It's entirely possible > that HDBC is adding some overhead. > > One of the items on our wish list[1] is to migrate away from HDBC. > Having hard numbers like you are providing will help us figure out > our > priorities a bit better. > > Thanks, > Michael > > [1] http://wiki.yesodweb.com/Wishlist > > On Mon, Mar 21, 2011 at 12:10 PM, Sven Koschnicke (GFXpro) >      wrote: >> >> Hello, >> >> I'm coming from a Rails background and was very impressed by the >> features of >> the Yesod Web Framework. Working on an trading app in Rails I wanted >> to >> port >> some features of this app to a Yesod app just to learn the framework >> and >> because we have some performance issues with Rails. >> >> The first thing I noticed was that the Yesod app was significantly >> slower >> than the Rails app at trivial tasks. I am very astonished about >> that. >> I >> thought Rails should be slower because of its interpreted nature. I >> made >> some benchmarking and measured that a simple Rails app is fourteen >> times >> faster than a Yesod app that did the same thing (loading some >> records >> from >> the database and displaying them). Did I made a mistake or is my >> understanding, that the compiled app should be faster, just wrong? >> >> I documented the results on github: >> >> https://github.com/SKoschnicke/performance-test >> >> Greetings >>  Sven Koschnicke >> >> _______________________________________________ >> web-devel mailing list >> web-devel@haskell.org >> http://www.haskell.org/mailman/listinfo/web-devel >> >

Hi, Just curies, What does "79 times slower" mean? Does it mean that when Rails-dev has processed 1 transaction Yesod did 79? Best regards. BTW: Saying that RoR is 79 times slower would indicate that Yesod is slow, but RoR is (79 times) slower. On 28 March 2011 14:31, Sven Koschnicke (GFXpro) wrote:

Hello,

I tested again with persistent-postgresql 0.4.0.1 and now the results are what you would expect :) rails in dev mode is now 79 times slower[1]. Of course, thats not a fair compare, but Greg did some testing with rails production already. Thanks for improving that so quickly!

Greetings Sven

[1] https://github.com/SKoschnicke/performance-test

On 03/25/2011 08:41 AM, Michael Snoyman wrote:

...

Hi Sven,

Can I ask you to try things out one more time, this time with persistent-postgresql 0.4.0.1? I think I fixed some more of the issue[1].

Thanks, Michael

[1] http://docs.yesodweb.com/blog/improving-persistent-performance

On Wed, Mar 23, 2011 at 9:23 AM, Sven Koschnicke (GFXpro) wrote:

...

Great that you found a way to speed things up! I'm looking forward to the blog post. I will run some more tests with the improved code, just let me know when its available. Is someone working on improving the postgres database backend? Maybe I could help there, the bad performance with postgres is really a showstopper for me.

Sven

On 03/23/2011 08:13 AM, Michael Snoyman wrote:

...

This was a great example of a performance bug being exactly where you don't expect it. I'm planning on writing up a blog post on this adventure later, but for the quick summary: the Criterion benchmarks have gone from 23.7ms (== 23,700us) to 141us. Meaning this is a 200 fold speedup for some operations. This came from switching a large monad transformer stack to a single RWS monad. But more on that in the blog post.

I've run your test a few more times: it still seems like there's a slowdown from the database code, but I think the widget performance bug was the big issue. After some more testing on my end, I'll put the code on Hackage. Would you give the test another shot after that?

Thanks again for bringing this up!

Michael

On Mon, Mar 21, 2011 at 5:41 PM, Sven Koschnicke (GFXpro) wrote:

...

Hello,

I think your guess wasn't completely wrong. I tested with Sqlite now, and with that rails and yesod are equally slow (rails 11.81 secs vs yesod 11.36 secs). But its also interesting that your code change speeds it up.

(I put the sqlite benchmark at https://github.com/SKoschnicke/performance-test/tree/sqlite)

My first guess was actually that its the marshalling which slows yesod so much and that rails uses some clever optimizations there, but I haven't got enough insight to validate that.

Greetings Sven

On 03/21/2011 01:49 PM, Michael Snoyman wrote:

...

Looks like my initial guess was completely wrong: on my system, almost no time is spent on the database query. Instead, it looks like there's some kind of performance issue with Hamlet. For example, changing line 20 of Handler/Quote.hs from "addWidget" to "addHtml" increases performance dramatically (25 sec to 4.5 sec). I'll need to spend some more time to properly diagnose the problem. Again, thanks for bringing this to my attention.

Michael

On Mon, Mar 21, 2011 at 12:40 PM, Michael Snoyman wrote:

> > Firstly, thank you for running this benchmark: it's the only way we > can find out where Yesod needs to be fixed. > > I haven't had a chance to look into the code properly, but it looks > like this is more a test of the database backend than of the web > framework. I'll try to get to this myself later, but it would be very > interesting to see the difference in response times between > PostgreSQL > and sqlite. Currently, persistent-sqlite binds directly to the sqlite > C API, while persistent-postgresql uses HDBC. It's entirely possible > that HDBC is adding some overhead. > > One of the items on our wish list[1] is to migrate away from HDBC. > Having hard numbers like you are providing will help us figure out > our > priorities a bit better. > > Thanks, > Michael > > [1] http://wiki.yesodweb.com/Wishlist > > On Mon, Mar 21, 2011 at 12:10 PM, Sven Koschnicke (GFXpro) > wrote: > >> >> Hello, >> >> I'm coming from a Rails background and was very impressed by the >> features of >> the Yesod Web Framework. Working on an trading app in Rails I wanted >> to >> port >> some features of this app to a Yesod app just to learn the framework >> and >> because we have some performance issues with Rails. >> >> The first thing I noticed was that the Yesod app was significantly >> slower >> than the Rails app at trivial tasks. I am very astonished about >> that. >> I >> thought Rails should be slower because of its interpreted nature. I >> made >> some benchmarking and measured that a simple Rails app is fourteen >> times >> faster than a Yesod app that did the same thing (loading some >> records >> from >> the database and displaying them). Did I made a mistake or is my >> understanding, that the compiled app should be faster, just wrong? >> >> I documented the results on github: >> >> https://github.com/SKoschnicke/performance-test >> >> Greetings >> Sven Koschnicke >> >> _______________________________________________ >> web-devel mailing list >> web-devel@haskell.org >> http://www.haskell.org/mailman/listinfo/web-devel >> >> >

_______________________________________________ web-devel mailing list web-devel@haskell.org http://www.haskell.org/mailman/listinfo/web-devel

-- Knut Olav Bøhmer