I'd like to first say that I'm very impressed with Ian's thoroughness of review.
On the API differences between Data.Text and Data.ByteString.Char8, I agree with Duncan that the Data.Text API is more natural for text-oriented work, although I'm slightly uncomfortable with the similarities between Data.Text and Data.List. Everything works the same, until it doesn't because of a minor API change you didn't notice.
Would it be useful to list the API incompatibilities in the docs, either as a list or at each relevant function? Or would that just be extra noise?
John
> I compared the API of Data.Text and Data.ByteString.Char8 and found a
> number of differences:
Many of these are deliberate and sensible. The thing with text as
opposed to lists/arrays is that almost all operations you want to do
are substring based and not element based. A Unicode code point (a
Char) is sadly only roughly related to the human concept of a
character. In particular there are combining characters. So even if
you want to search or split on a particular "character" that may mean
searching for a short sequence of Chars / code points.
So where the ByteString API followed the List api by being byte
oriented, the Text API is substring oriented.
> BS: Â break :: (Char -> Bool) -> ByteString -> (ByteString, ByteString)
> Â Â Â breakEnd :: (Char -> Bool) -> ByteString -> (ByteString, ByteString)
> Â Â Â breakSubstring :: ByteString -> ByteString -> (ByteString, ByteString)
> Text: break :: Text -> Text -> (Text, Text)
> Â Â Â breakEnd :: Text -> Text -> (Text, Text)
> Â Â Â breakBy :: (Char -> Bool) -> Text -> (Text, Text)
>
> BS: Â count :: Char -> ByteString -> Int
> Text: count :: Text -> Text -> Int
>
> BS: Â find :: (Char -> Bool) -> ByteString -> Maybe Char
> Text: find :: Text -> Text -> [(Text, Text)]
> Â Â Â findBy :: (Char -> Bool) -> Text -> Maybe Char
>
> BS: Â replicate :: Int -> Char -> ByteString
> Text: replicate :: Int -> Text -> Text
>
> BS: Â split :: Char -> ByteString -> [ByteString]
> Text: split :: Text -> Text -> [Text]
>
> BS: Â span :: (Char -> Bool) -> ByteString -> (ByteString, ByteString)
> Â Â Â spanEnd :: (Char -> Bool) -> ByteString -> (ByteString, ByteString)
> Text: spanBy :: (Char -> Bool) -> Text -> (Text, Text)
>
> BS: Â splitBy :: (Char -> Bool) -> Text -> [Text]
> Text: splitWith :: (Char -> Bool) -> ByteString -> [ByteString]
>
> BS: Â unfoldrN :: Int -> (a -> Maybe (Char, a)) -> a -> (ByteString, Maybe a)
> Text: unfoldrN :: Int -> (a -> Maybe (Char, a)) -> a -> Text
>
> BS: Â zipWith :: (Char -> Char -> a) -> ByteString -> ByteString -> [a]
> Text: zipWith :: (Char -> Char -> Char) -> Text -> Text -> Text
>
> I think the two APIs ought to be brought into agreement.
Perhaps. If so, then it is the ByteString.Char8 that ought to be
brought into agreement with Text, not the other way around. I think
Text is right in this area. On the other hand, perhaps it makes sense
for ByteString.Char8 to remain like the ByteString byte interface
which is byte oriented (and probably rightly so). I hope the
significance and use of ByteString.Char8 will decrease as Text becomes
more popular. ByteString.Char8 is really just for the cases where
you're handling ASCII-like protocols.