Iustin Pop
Let's say we have a simple JSON message: an array of 5 million numbers. I would like to parse this in constant space, such that if I only need the last element, overall memory usage is low (yes, unrealistic use, but please bear with me for a moment).
Using aeson, I thought the following program will be nicely-behaved:
part of the problem is that aeson builds an intermediate JSON parse-tree which has quite an overhead for representing a list of numbers on the heap as each numbers requires multiple heap objects (see also [1]). This is an area where e.g. Python has a significantly smaller footprint (mostly due to a more efficient heap representation). [...]
It seems that the Array constructor holds a vector, and this results in too much strictness?
btw, using a list on the other hand would add an overhead of 2 words (due to the ':' constructor) for representing each JSON array element in the parse-tree, that's probably why aeson uses vectors instead of lists. [...] cheers, hvr [1]: https://github.com/bos/aeson/issues/22