Jefferson Heard wrote:
I thought this was fairly straightforward, but where the marked line finishes in 0.31 seconds on my machine, the actual transpose takes more than 5 minutes. I know it must be possible to read data in haskell faster than this.
I took a look into this, writing a similar, but simpler, program. Half
of the runtime, and 2/3 of the allocation, is spent in ByteString's
split function. The remaining portions are spent in transposing the list.
COST CENTRE %time %alloc ticks bytes
split 66.7 65.1 56 120130000
xpose 31.0 32.8 26 60618031
read 1.2 2.0 1 3640229
lines 1.2 0.1 1 260002
I've attached two programs to demonstrate the problem. One creates a
sample speadsheet; the other transposes it.
I spent a little time trying to find a faster replacement for
ByteString.split, but no luck before I had to return to other things.