[Haskell-cafe] scotty + aeson + ffi: optimising performance

4 Apr 2014

      Hi,
I have a C++ program that is doing some IO with one shared library (dll). I
made a Haskell program that uses ffi for communicating with external dll.
With scotty and aeson I made a web-service that communicates with the dll
via REST-API (json). I was expecting that the performance would be a bit
slower than the c++ internal dll calls, but I'm getting approx 3x slower
performance in my haskell program.

I tried to profile it but so far I can see that the bottleneck is Aeson
(please correct me if I'm wrong). Is there a way how to optimise it? The
data I'm sending via JSON are very small, so that can not be the problem.

I have compiled the program with:

ghc -O2 --make Service.hs -o eDS_FM -prof -auto-all -L. -lextlib

Cheers,
Miro

profile:

Fri Apr 04 23:57 2014 Time and Allocation Profiling Report  (Final)

   eDS_FM.exe +RTS -p -RTS

total time  =        0.81 secs   (810 ticks @ 1000 us, 1 processor)
total alloc = 638,446,652 bytes  (excludes profiling overheads)

COST CENTRE                 MODULE                     %time %alloc

MAIN                        MAIN                        65.3   63.1
go/Number                   Data.Aeson.Encode           22.5   26.2
array_                      Data.Aeson.Parser.Internal   7.0    6.6
tableArrayInDataToJSON.json FM.Json                      1.5    0.9
go/Array                    Data.Aeson.Encode            1.4    1.1

                            individual     inherited
COST CENTRE                                 MODULE
           no.     entries  %time %alloc   %time %alloc

MAIN                                        MAIN
           172           0   65.3   63.1   100.0  100.0
 unstream/resize                            Data.Text.Fusion
           419           7    0.0    0.0     0.0    0.0
 main.arrayDouble                           Main
           415        2158    0.0    0.2     0.0    0.2
 main.retCodeInt                            Main
           414        2158    0.0    0.0     0.0    0.0
 main.dataOut                               Main
           413        2158    0.0    0.0     0.0    0.0
 getarray.paramLength                       FM.ModelIO
           408        2158    0.0    0.0     0.0    0.0
 getarray                                   FM.ModelIO
           407        2158    0.1    0.7     0.1    0.7
  getarray.rowsReturnedInt                  FM.ModelIO
           412        2158    0.0    0.0     0.0    0.0
  getarray.arrayPtr                         FM.ModelIO
           411        2158    0.0    0.0     0.0    0.0
 main.resultInt                             Main
           396           1    0.0    0.0     0.0    0.0
 main.resultInt                             Main
           395           1    0.0    0.0     0.0    0.0
 setmodulestring                            FM.ModelIO
           391           1    0.0    0.0     0.0    0.0
  setmodulestring.setVarInArray             FM.ModelIO
           394           1    0.0    0.0     0.0    0.0
  setmodulestring.cValueLength              FM.ModelIO
           393           1    0.0    0.0     0.0    0.0
  setmodulestring.cParamLength              FM.ModelIO
           392           1    0.0    0.0     0.0    0.0
 main.queryString                           Main
           385           1    0.0    0.0     0.0    0.0
  stringDataToJSON                          FM.Json
          386           1    0.0    0.0     0.0    0.0
   stringDataToJSON.json                    FM.Json
          388           1    0.0    0.0     0.0    0.0
    object_                                 Data.Aeson.Parser.Internal
           389           0    0.0    0.0     0.0    0.0
     jstring_                               Data.Aeson.Parser.Internal
           390           4    0.0    0.0     0.0    0.0
   stringDataToJSON.retValue                FM.Json
          387           1    0.0    0.0     0.0    0.0
 main.resultInt                             Main
           382         290    0.0    0.0     0.0    0.0
 go/Object                                  Data.Aeson.Encode
          379        2553    0.0    0.1    24.1   27.5
  go/Array                                  Data.Aeson.Encode
          416        2158    1.2    0.8    21.7   26.2
   go/Number                                Data.Aeson.Encode
          418       63801   20.5   25.4    20.5   25.4
  go/Number                                 Data.Aeson.Encode
          383        2656    0.1    0.1     0.1    0.1
  string                                    Data.Aeson.Encode
          380        4814    0.1    0.1     2.2    1.1
   go/Number                                Data.Aeson.Encode
          384           0    1.9    0.8     2.0    1.0
    go/Array                                Data.Aeson.Encode
          417           0    0.1    0.2     0.1    0.2
   break                                    Data.Aeson.Encode
          381        4814    0.1    0.0     0.1    0.0
 setmoduletable                             FM.ModelIO
           372         290    0.2    0.0     0.4    0.1
  setmoduletable.cRows                      FM.ModelIO
           376         290    0.0    0.0     0.0    0.0
  setmoduletable.cColumns                   FM.ModelIO
           375         290    0.0    0.0     0.0    0.0
  setmoduletable.cParamLength               FM.ModelIO
           374         290    0.1    0.0     0.1    0.0
  setmoduletable.cTable                     FM.ModelIO
           373         290    0.0    0.0     0.0    0.0
 main.queryString                           Main
           363         290    0.0    0.0     9.6    7.7
  tableArrayInDataToJSON                    FM.Json
          364         290    0.0    0.0     9.6    7.7
   tableArrayInDataToJSON.json              FM.Json
          366         290    1.5    0.9     9.6    7.7
    object_                                 Data.Aeson.Parser.Internal
           368           0    0.1    0.0     8.1    6.8
     jstring_                               Data.Aeson.Parser.Internal
           369        1450    1.0    0.2     8.0    6.8
      array_                                Data.Aeson.Parser.Internal
           371           0    7.0    6.6     7.0    6.6
   tableArrayInDataToJSON.retValue          FM.Json
          365         290    0.0    0.0     0.0    0.0
 main                                       Main
           345           0    0.2    0.7     0.2    0.7
  run                                       FM.ModelIO
           398           0    0.0    0.0     0.0    0.0
   run.result                               FM.ModelIO
           405         103    0.0    0.0     0.0    0.0
   run.timeTotal                            FM.ModelIO
           404         103    0.0    0.0     0.0    0.0
   run.retData                              FM.ModelIO
           403         103    0.0    0.0     0.0    0.0
 CAF
 GHC.Integer.Logarithms.Internals          343           0    0.0    0.0
  0.0    0.0
 CAF                                        GHC.Float.ConversionUtils
          340           0    0.0    0.0     0.0    0.0
 CAF                                        GHC.IO.Encoding.CodePage
           327           0    0.0    0.0     0.0    0.0
 CAF                                        GHC.Conc.Windows
           317           0    0.0    0.0     0.0    0.0
 CAF                                        GHC.TopHandler
           315           0    0.0    0.0     0.0    0.0
 CAF                                        GHC.Float
          312           0    0.0    0.0     0.0    0.0
 CAF                                        Data.Fixed
           301           0    0.0    0.0     0.0    0.0
 CAF                                        GHC.IO.Handle.FD
           299           0    0.0    0.0     0.0    0.0
 CAF                                        GHC.IO.Encoding
          295           0    0.0    0.0     0.0    0.0
 CAF                                        GHC.IO.Exception
           294           0    0.1    0.0     0.1    0.0
 CAF
 Data.Text.Lazy.Builder.Int.Digits         282           0    0.0    0.0
  0.0    0.0
 CAF                                        Data.Text.Lazy.Builder.Int
           281           0    0.0    0.0     0.0    0.0
 CAF                                        Data.Text.Array
          275           0    0.0    0.0     0.0    0.0
 CAF                                        Data.Text.Internal
           274           0    0.0    0.0     0.0    0.0
 CAF                                        System.Locale
          269           0    0.0    0.0     0.0    0.0
 CAF                                        Data.Scientific
          268           0    0.0    0.0     0.0    0.0
 CAF                                        Data.Time.LocalTime.TimeOfDay
          265           0    0.0    0.0     0.0    0.0
 CAF                                        Data.Time.Clock.POSIX
          263           0    0.0    0.0     0.0    0.0
 CAF                                        Data.Aeson.Parser.Internal
           256           0    0.0    0.0     0.0    0.0
  array_                                    Data.Aeson.Parser.Internal
           370           1    0.0    0.0     0.0    0.0
  object_                                   Data.Aeson.Parser.Internal
           367           1    0.0    0.0     0.0    0.0
 CAF                                        Blaze.ByteString.Builder.HTTP
          249           0    0.0    0.0     0.0    0.0
 CAF                                        Network.HTTP.Types.Header
          227           0    0.0    0.0     0.0    0.0
 CAF                                        Network.HTTP.Types.URI
           226           0    0.0    0.0     0.0    0.0
 CAF                                        Network.HTTP.Types.Status
          225           0    0.0    0.0     0.0    0.0
 CAF                                        Network.HTTP.Types.Method
          224           0    0.0    0.0     0.0    0.0
 CAF                                        Network.Socket
           220           0    0.0    0.0     0.0    0.0
 CAF                                        Network.Wai.Parse
          211           0    0.0    0.0     0.0    0.0
 CAF
 Network.Wai.Handler.Warp.RequestHeader    203           0    0.0    0.0
  0.0    0.0
 CAF
 Network.Wai.Handler.Warp.ResponseHeader   202           0    0.0    0.0
  0.0    0.0
 CAF                                        Network.Wai.Handler.Warp.Date
          199           0    0.0    0.0     0.0    0.0
 CAF
 Network.Wai.Handler.Warp.Request          197           0    0.0    0.0
  0.0    0.0
 CAF
 Network.Wai.Handler.Warp.Response         196           0    0.0    0.0
  0.0    0.0
 CAF                                        Network.Wai.Handler.Warp.Recv
          194           0    0.0    0.0     0.0    0.0
 CAF                                        Network.Wai.Handler.Warp.Header
          193           0    0.0    0.0     0.0    0.0
 CAF                                        Network.Wai.Handler.Warp.Types
           191           0    0.0    0.0     0.0    0.0
 CAF
 Network.Wai.Handler.Warp.Settings         189           0    0.0    0.0
  0.0    0.0
 CAF                                        Web.Scotty.Trans
           187           0    0.0    0.0     0.0    0.0
 CAF                                        Web.Scotty.Route
           186           0    0.0    0.0     0.0    0.0
 CAF                                        Web.Scotty.Types
           185           0    0.0    0.0     0.0    0.0
  unstreamChunks/inner                      Data.Text.Lazy.Fusion
          349           0    0.0    0.0     0.0    0.0
   unstreamChunks/outer                     Data.Text.Lazy.Fusion
          350           1    0.0    0.0     0.0    0.0
 CAF                                        Web.Scotty.Action
          184           0    0.0    0.0     0.0    0.0
  unstreamChunks/outer                      Data.Text.Lazy.Fusion
          353           4    0.0    0.0     0.0    0.0
   unstreamChunks/inner                     Data.Text.Lazy.Fusion
          354          28    0.0    0.0     0.0    0.0
    unstreamChunks/resize                   Data.Text.Lazy.Fusion
          355           5    0.0    0.0     0.0    0.0
 CAF                                        Main
           182           0    0.0    0.0     0.0    0.0
  unstreamChunks/outer                      Data.Text.Lazy.Fusion
          406           1    0.0    0.0     0.0    0.0
  main.resultInt                            Main
           360           1    0.0    0.0     0.0    0.0
  main                                      Main
           344           1    0.0    0.0     0.0    0.0
   go/Object                                Data.Aeson.Encode
          357           1    0.0    0.0     0.0    0.0
    go/Number                               Data.Aeson.Encode
          361           1    0.0    0.0     0.0    0.0
    string                                  Data.Aeson.Encode
          358           1    0.0    0.0     0.0    0.0
     go/Number                              Data.Aeson.Encode
          362           0    0.0    0.0     0.0    0.0
     break                                  Data.Aeson.Encode
          359           1    0.0    0.0     0.0    0.0
   unstreamChunks/outer                     Data.Text.Lazy.Fusion
          346          10    0.0    0.0     0.0    0.0
    unstreamChunks/inner                    Data.Text.Lazy.Fusion
          347         104    0.0    0.0     0.0    0.0
     unstreamChunks/resize                  Data.Text.Lazy.Fusion
          348          20    0.0    0.0     0.0    0.0
 CAF                                        FM.Json
          180           0    0.1    0.0     0.1    0.0
  unstream/resize                           Data.Text.Fusion
           356          14    0.0    0.0     0.0    0.0
 CAF                                        FM.ModelIO
           179           0    0.0    0.0     0.0    0.0
  getarray                                  FM.ModelIO
           409           0    0.0    0.0     0.0    0.0
   getarray.rows                            FM.ModelIO
           410           1    0.0    0.0     0.0    0.0
  run.timeTot                               FM.ModelIO
           399           1    0.0    0.0     0.0    0.0
  run                                       FM.ModelIO
           397           1    0.0    0.0     0.0    0.0
   run.runType                              FM.ModelIO
           402           1    0.0    0.0     0.0    0.0
   run.timeNow                              FM.ModelIO
           401           1    0.0    0.0     0.0    0.0
   run.timeTot                              FM.ModelIO
           400           0    0.0    0.0     0.0    0.0
  setmoduletable                            FM.ModelIO
           377           0    0.0    0.0     0.0    0.0
   setmoduletable.firstTimeStepAfterRestart FM.ModelIO
           378           1    0.0    0.0     0.0    0.0
  initfm                                    FM.ModelIO
           351           1    0.0    0.0     0.0    0.0

[Haskell-cafe] scotty + aeson + ffi: optimising performance

Miro Karpis