Is the #functional programming paradigm antithetical to efficient strings? #Haskell
I can certainly see how it might seem so. Our main string representation uses about 10 times more memory than seems necessary, and is relatively fast, though probably not compared to C. Recently I spent some time looking into how to reduce this memory overhead, and I modified the pretty library (https://github.com/ddssff/pretty/tree/textdetails) so it could use any type that was an instance of ListLike and StringLike (https://github.com/JohnLato/listlike). Then I tried the UnitLargeDoc test with several different data types. This just concatenates a list of ten million "Hello" strings. Using String this happens in about 5 seconds. Using strict or lazy Text it immediately blew the stack. When I increased the stack size to over 1GB it took minutes to complete. When I used the Data.Text.Lazy.Builder type instead it took about 30 seconds to complete. Results with utf8 encoded ByteStrings were siimilar.
When I mentioned some of this, I was told "different data structures for different purposes", which sort of makes sense, but honestly if Haskell is a language where you have to sit down and choose a representation every time you want to build some text, I get a bit discouraged.
So is functional programming antithetical to efficient strings? In theory maybe not, but I would love to see some hard evidence.