I actually was not able to successfully google for Text vs. String benchmarks. If someone can point one out that would be very helpful. On Sat, Mar 17, 2012 at 1:52 AM, Christopher Done wrote:

On 17 March 2012 05:30, Tony Morris wrote:

...

Do you know if there is a good write-up of the benefits of Data.Text over String? I'm aware of the advantages just by my own usage; hoping someone has documented it rather than in our heads.

Good point, it would be good to collate the experience and wisdom of this decision with some benchmark results on the HaskellWiki as The Place to link to when justifying it.

_______________________________________________ Haskell-prime mailing list Haskell-prime@haskell.org http://www.haskell.org/mailman/listinfo/haskell-prime

This is the best I can do with Bryan's blog posts, but none of the graphs (which contain all the information) show up: http://web.archive.org/web/20100222031602/http://www.serpentine.com/blog/200... If someone has some benchmarks that can be ran that would be helpful. On Mon, Mar 19, 2012 at 7:51 AM, Johan Tibell wrote:

Hi Greg,

There are a few blog posts on Bryan's blog. Here are two of them:

   http://www.serpentine.com/blog/2009/10/09/announcing-a-major-revision-of-the...    http://www.serpentine.com/blog/2009/12/10/the-performance-of-data-text/

Unfortunately the blog seems partly broken. Images are missing and some articles are missing altogether (i.e. the article is there but the actualy body text is gone.)

-- Johan

On 18 March 2012 19:29, ARJANEN Loïc Jean David wrote:

Good point, but rather than specifying in the standard that the new string type should be the Text datatype, maybe the new definition should be that String is a newtype with suitable operations defined on it, and perhaps a typeclass to convert to and from this newtype. The reason of my remark is although most implementations compile to native code, an implementation compiling to, for example, JavaScript might wish to use JavaScript's string type rather than forcing its users to have a native library installed.

I agree that the language standard should not prescribe the implementation of a Text datatype. It should instead require an abstract data type (which may just be a newtype wrapper for [Char] in some implementations) and a (minimal) set of operations on it. Regarding the type class for converting to and from that type, there is a perhaps more complicated question: The current fromString method uses String as the source type which causes unnecessary overhead. This is unfortunate since GHC's built-in mechanism actually uses unpackCString[Utf8]# which constructs the inefficient String representation from a compact memory representation. I think it would be best if the new fromString/fromText class allowed an efficient mechanism like that. unpackCString# has type Addr# -> [Char] which is obviously GHC-specific. -- Push the envelope. Watch it bend.

On 03/19/2012 04:53 PM, Johan Tibell wrote:

I've been thinking about this question as well. How about

class IsString s where unpackCString :: Ptr Word8 -> CSize -> s

What's the Ptr Word8 supposed to contain? A UTF-8 encoded string? Best regards Christian -- |------- Dr. Christian Siefkes ------- christian@siefkes.net ------- | Homepage: http://www.siefkes.net/ | Blog: http://www.keimform.de/ | Peer Production Everywhere: http://peerconomy.org/wiki/ |---------------------------------- OpenPGP Key ID: 0x346452D8 -- A choice of masters is not freedom. -- Bradley M. Kuhn and Richard M. Stallman, Freedom Or Power?

If the input is specified to be UTF-8, wouldn't it be better to call the method unpackUTF8 or something like that? On Mon, Mar 19, 2012 at 12:59 PM, Johan Tibell wrote:

On Mon, Mar 19, 2012 at 9:02 AM, Christian Siefkes wrote:

...

On 03/19/2012 04:53 PM, Johan Tibell wrote:

...

I've been thinking about this question as well. How about

class IsString s where unpackCString :: Ptr Word8 -> CSize -> s

What's the Ptr Word8 supposed to contain? A UTF-8 encoded string?

Yes.

We could make a distinction between byte and Unicode literals and have:

class IsBytes a where unpackBytes :: Ptr Word8 -> Int -> a

class IsText a where unpackText :: Ptr Word8 -> Int -> a

In the latter the caller guarantees that the passed in pointer points to wellformed UTF-8 data.

-- Johan

_______________________________________________ Haskell-prime mailing list Haskell-prime@haskell.org http://www.haskell.org/mailman/listinfo/haskell-prime

On Mon, Mar 19, 2012 at 9:02 AM, Christian Siefkes wrote:

...

On 03/19/2012 04:53 PM, Johan Tibell wrote:

...

I've been thinking about this question as well. How about

class IsString s where     unpackCString :: Ptr Word8 -> CSize -> s

What's the Ptr Word8 supposed to contain? A UTF-8 encoded string?

Yes.

We could make a distinction between byte and Unicode literals and have:

class IsBytes a where unpackBytes :: Ptr Word8 -> Int -> a

class IsText a where unpackText :: Ptr Word8 -> Int -> a

In the latter the caller guarantees that the passed in pointer points to wellformed UTF-8 data.

Is there a reason not to put all these methods in the IsString class, with appropriate default definitions? You would need a UTF-8 encoder (& decoder) of course, but it would reduce the burden on clients and improve backwards compatibility. Cheers, Simon

Don't forget that with -XOverloadedStrings we already have a IsString class. (That's not a Haskell Prime extension though.) class IsString a where fromString :: String -> a Simon | -----Original Message----- | From: haskell-prime-bounces@haskell.org [mailto:haskell-prime- | bounces@haskell.org] On Behalf Of Johan Tibell | Sent: 19 March 2012 15:54 | To: Thomas Schilling | Cc: haskell-prime@haskell.org | Subject: Re: String != [Char] | | On Mon, Mar 19, 2012 at 8:45 AM, Thomas Schilling | wrote: | > Regarding the type class for converting to and from that type, there | > is a perhaps more complicated question: The current fromString method | > uses String as the source type which causes unnecessary overhead. This | > is unfortunate since GHC's built-in mechanism actually uses | > unpackCString[Utf8]# which constructs the inefficient String | > representation from a compact memory representation.  I think it would | > be best if the new fromString/fromText class allowed an efficient | > mechanism like that.  unpackCString# has type Addr# -> [Char] which is | > obviously GHC-specific. | | I've been thinking about this question as well. How about | | class IsString s where | unpackCString :: Ptr Word8 -> CSize -> s | | It's morally equivalent of unpackCString#, but uses standard Haskell types. | | -- Johan | | _______________________________________________ | Haskell-prime mailing list | Haskell-prime@haskell.org | http://www.haskell.org/mailman/listinfo/haskell-prime

Hi, Thomas Schilling wrote:

I agree that the language standard should not prescribe the implementation of a Text datatype. It should instead require an abstract data type (which may just be a newtype wrapper for [Char] in some implementations) and a (minimal) set of operations on it.

Regarding the type class for converting to and from that type, there is a perhaps more complicated question: The current fromString method uses String as the source type which causes unnecessary overhead.

Is this still a problem if String would be replaced by an implementation-dependend newtype? Presumably, GHC would use a more efficient representation behind the newtype, so the following would be efficient in practice (or not?) newtype String = ... class IsString a where fromString :: String -> a The standard could even prescribe that an instance for [Char] exists: explode :: String -> [Char] explode = ... instance IsString [Char] where fromString = explode Tillmann

This proposal seems fairly uncontroversial at the moment. I would really like it if someone more familiar with the proposal process can take this proposal up and help make sure it gets in the next batch. I can't even figure out how to create a wiki page for the proposal right now :)

I am not trying to win an argument with anyone. Just trying to do what is best for the community. Many others here have a better grasp of the issue than me and can help answer questions and come up with a solution. I am also not saying this proposal is done. A lot of thought and work is needed to ensure it can be implemented as smoothly as possible. It does seem though that everyone thinks it is a good proposal. On Thu, Mar 22, 2012 at 2:06 PM, ARJANEN Loïc Jean David wrote:

Le 22/03/2012 04:29, Greg Weber a écrit :

...

This proposal seems fairly uncontroversial at the moment. I would really like it if someone more familiar with the proposal process can take this proposal up and help make sure it gets in the next batch. I can't even figure out how to create a wiki page for the proposal right now :)

Well, this proposal seems uncontroversial because we didn't arrive to the difficult part: what operations should we define on this String type for it to be useful. Because with only this proposal as it stands now (String defined as an implementation-defined newtype, a typeclass defined for conversion from/to String and [Char] as an instance of this typeclass), we're in a worse situation than before: not only String became useless given there is no operations defined on it, the only mean we have to portably work with it is to translate it to [Char] before doing anything. So now, the fun part begins...what operations should String support ? I propose obtaining the length of a String, taking a substring of a given size beginning at a given index, taking the character at index i in a String, concatenation, converting a string to upper/lower case and determining if a string is contained in/a prefix/a suffix of another. I am sure I am forgetting some useful operations and some operations I said are better placed in the typeclass or in a typeclass instance or are particular cases of general operations we should define rather than the particular cases. So, what are the operations we should define according to you ?

Regards, ARJANEN Loïc

_______________________________________________ Haskell-prime mailing list Haskell-prime@haskell.org http://www.haskell.org/mailman/listinfo/haskell-prime

2012/3/22 Greg Weber :

I am not trying to win an argument with anyone. Just trying to do what is best for the community. Many others here have a better grasp of the issue than me and can help answer questions and come up with a solution.

I am also not saying this proposal is done. A lot of thought and work is needed to ensure it can be implemented as smoothly as possible. It does seem though that everyone thinks it is a good proposal.

Sorry for the misunderstanding, but when you said that someone should take this proposal up and help make sure it gets in the next batch, I believed you thought we could take this proposal as is. Deeply sorry for my error. But now we have at least two tasks to do before we can put up the proposal: define what operations should be supported by String and should we apply this proposal in the next batch. Given that this proposal will break many codebases (we shouldn't hope to apply all of list's syntax to this string type) should we apply it alone or wait until we have more other codebase-breakers to apply ? -- ARJANEN Loïc Jean David http://luigiscorner.wordpress.com --- "Computer science is no more about computers than astronomy is about telescopes, biology is about microscopes, or chemistry is about beakers and test tubes. Science is not about tools. It is about how we use them, and what we find out when we do." Michael R. Fellows and Ian Parberry

On Fri, Mar 23, 2012 at 1:24 PM, Greg Weber wrote:

I would really just like for someone to show me how to create a wiki proposal page :)

This proposal doesn't have to break any codebases. One possibility is to add the Text type to the standard while keeping String and marking it as deprecated.

I'm in favour of this. In fact, I'm not sure if I would even deprecate String. I think just adding Text or something Text-like to the standard would be a good step towards encouraging libraries to use it as their first choice. It might, however, be wise to first adopt GHC's OverloadedStrings proposal so that the syntax for using string alternatives is more convenient. I'm inclined to start slow and small: OverloadedStrings is already in use, and makes sense with overloaded numeric literals that we already have, so I think it's reasonable to push for including that in the standard. I don't think that blessing any other string type is going to be very successful *without* OverloadedStrings, and I think that Duncan is right that we want to introduce a new type before removing the old one. With regards to performance of fromString, I feel like if it was a serious problem (and how many really big strings are going to be built that way?) then an effort to do some special-case inlining (after all, the parameters are constant and specified at compile time) might be beneficial. With regards to a general string API, I don't think a typeclass is the correct solution (except for string literals); my view is that things like ListLike may be practical but are awkward to use, and ambiguity problems only make things more upsetting. I think we should just take the abstract Text type and API, and leave implementors to do whatever they want behind that.

On 03/23/2012 02:13 PM, ARJANEN Loïc Jean David wrote:

2012/3/22 Greg Weber : But now we have at least two tasks to do before we can put up the proposal: define what operations should be supported by String and should we apply this proposal in the next batch. Given that this proposal will break many codebases (we shouldn't hope to apply all of list's syntax to this string type) should we apply it alone or wait until we have more other codebase-breakers to apply ?

I very much hope that the Haskell committee will never ever accept a proposal that "will break many codebases"! That's what ruined Perl 6 und Python 3, and quite unnecessarily so. Even if I a future Haskell standard defines String as something that doesn't have to be implemented as a list of Char, it still would have to behave as if it was [Char] for most practical purposes (except performance-wise, of course!). That's necessary for compatibility. Or String could just be complemented with an additional standardized Text type, as Greg suggested. Best regards Christian -- |------- Dr. Christian Siefkes ------- christian@siefkes.net ------- | Homepage: http://www.siefkes.net/ | Blog: http://www.keimform.de/ | Peer Production Everywhere: http://peerconomy.org/wiki/ |---------------------------------- OpenPGP Key ID: 0x346452D8 -- Just so that nobody takes his guess for the full truth, here's my standing on "keeping control", in 2 words (three?): I won't. -- Linus Torvalds, The Tanenbaum-Torvalds Debate

Hi, ARJANEN Loïc Jean David wrote:

But now we have at least two tasks to do before we can put up the proposal: define what operations should be supported by String and should we apply this proposal in the next batch. Given that this proposal will break many codebases (we shouldn't hope to apply all of list's syntax to this string type) should we apply it alone or wait until we have more other codebase-breakers to apply ?

I would expect the following steps: 1. Define what operations should be supported by String, that is, define a String API, possibly including thoughts on performance, formal specification, tests, benchmarks, ... 2. Convince all Haskell implementations to provide an implementation of the String API outside the Prelude, as an additional module (in the base package?). That implementation can be based on [Char] or something else. 3. Convince all string-like-packages on Hackage to provide exactly the String API in a separate module, so these packages are now drop-in replacements for the String implementations from step 2 above. At this point, we haven't touched the Prelude, but we have a blessed String API with multiple implementations. So applications can opt-in to use that String API instead of the Prelude-based [Char]. This allows us to: 4. Convince packages on Hackage to use the type String (from step 2) instead of Prelude-based [Char]; or to use the StringLike class instead of a concrete string type. 5. Refine the String API according to experience. And finally, we can: 6. Change Prelude.String to be the type from step 2 above. My hope is that because of steps 2 and 3, the investment from step 1 starts to pay off long before we reach step 6. Tillmann