Proposal: Deprecate decodeUtf8
I propose to add a deprecation pragma to the partial function Data.Text.decodeUtf8. I see checks or decoding failure accidentally omitted from production code every week. I claim decodeUtf8 to be *the* most abused partial function in the current Haskell ecosystem (with `head` and `tail` at least everybody knows they should feel bad when typing them down). decodeUtf8' with custom error message is a better solution in all cases I have seen and can come up with. I therefore propose: * Add deprecation pragma to decodeUtf8 * Do it with Text.Lazy and *16/*32 functions too * But keep them indefinitely (we don't want to break compiles) I believe that deprecating decodeUtf8 will result in lots of useful warnings across the world every day and less software crashing at runtime. Discussion period 2 months. Thanks!
+1 LGTM. On 18.08.2016 11:08, Niklas Hambüchen wrote:
I propose to add a deprecation pragma to the partial function Data.Text.decodeUtf8.
I see checks or decoding failure accidentally omitted from production code every week. I claim decodeUtf8 to be *the* most abused partial function in the current Haskell ecosystem (with `head` and `tail` at least everybody knows they should feel bad when typing them down).
decodeUtf8' with custom error message is a better solution in all cases I have seen and can come up with.
I therefore propose:
* Add deprecation pragma to decodeUtf8 * Do it with Text.Lazy and *16/*32 functions too * But keep them indefinitely (we don't want to break compiles)
I believe that deprecating decodeUtf8 will result in lots of useful warnings across the world every day and less software crashing at runtime.
Discussion period 2 months.
Thanks! _______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
-- Andreas Abel <>< Du bist der geliebte Mensch. Department of Computer Science and Engineering Chalmers and Gothenburg University, Sweden andreas.abel@gu.se http://www2.tcs.ifi.lmu.de/~abel/
Or should it be changed - decodeUtf8 = decodeUtf8With strictDecode + decodeUtf8 = decodeUtf8With lenientDecode
On 18 Aug 2016, at 12:08, Niklas Hambüchen <mail@nh2.me> wrote:
I propose to add a deprecation pragma to the partial function Data.Text.decodeUtf8.
I see checks or decoding failure accidentally omitted from production code every week. I claim decodeUtf8 to be *the* most abused partial function in the current Haskell ecosystem (with `head` and `tail` at least everybody knows they should feel bad when typing them down).
decodeUtf8' with custom error message is a better solution in all cases I have seen and can come up with.
I therefore propose:
* Add deprecation pragma to decodeUtf8 * Do it with Text.Lazy and *16/*32 functions too * But keep them indefinitely (we don't want to break compiles)
I believe that deprecating decodeUtf8 will result in lots of useful warnings across the world every day and less software crashing at runtime.
Discussion period 2 months.
Thanks! _______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Changing the combinator's semantics seems likely to be worse to me. I have actual code using that function that would become much more dangerous if it started accepting data through it that it can't round trip. Right now that partiality acts as a dumb but effective safety net. I'm neutral on the basic proposal, however. -Edward On Thu, Aug 18, 2016 at 6:11 AM, Oleg Grenrus <oleg.grenrus@iki.fi> wrote:
Or should it be changed
- decodeUtf8 = decodeUtf8With strictDecode + decodeUtf8 = decodeUtf8With lenientDecode
On 18 Aug 2016, at 12:08, Niklas Hambüchen <mail@nh2.me> wrote:
I propose to add a deprecation pragma to the partial function Data.Text.decodeUtf8.
I see checks or decoding failure accidentally omitted from production code every week. I claim decodeUtf8 to be *the* most abused partial function in the current Haskell ecosystem (with `head` and `tail` at least everybody knows they should feel bad when typing them down).
decodeUtf8' with custom error message is a better solution in all cases I have seen and can come up with.
I therefore propose:
* Add deprecation pragma to decodeUtf8 * Do it with Text.Lazy and *16/*32 functions too * But keep them indefinitely (we don't want to break compiles)
I believe that deprecating decodeUtf8 will result in lots of useful warnings across the world every day and less software crashing at runtime.
Discussion period 2 months.
Thanks! _______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
So (as also pointed by Edward): - Some people are aware that decodeUtf8 is partial and rely on that behaviour - Some people aren’t aware, and think that decodeUtf8 is like `decodeUtf8With lenientDecode` or `decodeUtf8With ignore`. Then if we add a {-# DEPRECATED #-} warning on `decodeUtf8`, we should add variants for both use cases lenientDecodeUtf8 which is total, and unsafeDecodeUtf8 or strictDecodeUtf8, which is what decodeUtf8 does today. I’d like the new names, for convenience, `OnDecodeError` values would require a new module import. - Oleg
On 18 Aug 2016, at 14:40, Niklas Hambüchen <mail@nh2.me> wrote:
On 18/08/16 18:11, Oleg Grenrus wrote:
Or should it be changed
- decodeUtf8 = decodeUtf8With strictDecode + decodeUtf8 = decodeUtf8With lenientDecode
I'd be very upset for that kind of a semantics change to sneak in under me.
If the intent is to keep this function indefinitely then why abuse DEPRECATED? Why not utilize WARNING? On Thu, Aug 18, 2016 at 7:50 AM, Oleg Grenrus <oleg.grenrus@iki.fi> wrote:
So (as also pointed by Edward):
- Some people are aware that decodeUtf8 is partial and rely on that behaviour - Some people aren’t aware, and think that decodeUtf8 is like `decodeUtf8With lenientDecode` or `decodeUtf8With ignore`.
Then if we add a {-# DEPRECATED #-} warning on `decodeUtf8`, we should add variants for both use cases
lenientDecodeUtf8 which is total, and unsafeDecodeUtf8 or strictDecodeUtf8, which is what decodeUtf8 does today.
I’d like the new names, for convenience, `OnDecodeError` values would require a new module import.
- Oleg
On 18 Aug 2016, at 14:40, Niklas Hambüchen <mail@nh2.me> wrote:
On 18/08/16 18:11, Oleg Grenrus wrote:
Or should it be changed
- decodeUtf8 = decodeUtf8With strictDecode + decodeUtf8 = decodeUtf8With lenientDecode
I'd be very upset for that kind of a semantics change to sneak in under me.
_______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
My idea was to use DEPRECATED to indicate that it comes from a time where partial functions were considered OK, but this is no longer common practice now. WARNING would work just as well though, I'd have no preference between the two. On 18/08/16 23:13, evan@evan-borden.com wrote:
If the intent is to keep this function indefinitely then why abuse DEPRECATED? Why not utilize WARNING?
On Thu, Aug 18, 2016 at 7:50 AM, Oleg Grenrus <oleg.grenrus@iki.fi <mailto:oleg.grenrus@iki.fi>> wrote:
So (as also pointed by Edward):
- Some people are aware that decodeUtf8 is partial and rely on that behaviour - Some people aren’t aware, and think that decodeUtf8 is like `decodeUtf8With lenientDecode` or `decodeUtf8With ignore`.
Then if we add a {-# DEPRECATED #-} warning on `decodeUtf8`, we should add variants for both use cases
lenientDecodeUtf8 which is total, and unsafeDecodeUtf8 or strictDecodeUtf8, which is what decodeUtf8 does today.
I’d like the new names, for convenience, `OnDecodeError` values would require a new module import.
- Oleg
> On 18 Aug 2016, at 14:40, Niklas Hambüchen <mail@nh2.me <mailto:mail@nh2.me>> wrote: > > On 18/08/16 18:11, Oleg Grenrus wrote: >> Or should it be changed >> >> - decodeUtf8 = decodeUtf8With strictDecode >> + decodeUtf8 = decodeUtf8With lenientDecode > > I'd be very upset for that kind of a semantics change to sneak in under me. >
_______________________________________________ Libraries mailing list Libraries@haskell.org <mailto:Libraries@haskell.org> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries <http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries>
Why not propose adding a variant that exposes the error handling. Deprecation is a heavy hammer when most users seem to know that it's partial or discover it rather quickly. On Thursday, August 18, 2016, Niklas Hambüchen <mail@nh2.me> wrote:
My idea was to use DEPRECATED to indicate that it comes from a time where partial functions were considered OK, but this is no longer common practice now.
WARNING would work just as well though, I'd have no preference between the two.
On 18/08/16 23:13, evan@evan-borden.com <javascript:;> wrote:
If the intent is to keep this function indefinitely then why abuse DEPRECATED? Why not utilize WARNING?
On Thu, Aug 18, 2016 at 7:50 AM, Oleg Grenrus <oleg.grenrus@iki.fi <javascript:;> <mailto:oleg.grenrus@iki.fi <javascript:;>>> wrote:
So (as also pointed by Edward):
- Some people are aware that decodeUtf8 is partial and rely on that behaviour - Some people aren’t aware, and think that decodeUtf8 is like `decodeUtf8With lenientDecode` or `decodeUtf8With ignore`.
Then if we add a {-# DEPRECATED #-} warning on `decodeUtf8`, we should add variants for both use cases
lenientDecodeUtf8 which is total, and unsafeDecodeUtf8 or strictDecodeUtf8, which is what decodeUtf8 does today.
I’d like the new names, for convenience, `OnDecodeError` values would require a new module import.
- Oleg
> On 18 Aug 2016, at 14:40, Niklas Hambüchen <mail@nh2.me <javascript:;> <mailto:mail@nh2.me <javascript:;>>> wrote: > > On 18/08/16 18:11, Oleg Grenrus wrote: >> Or should it be changed >> >> - decodeUtf8 = decodeUtf8With strictDecode >> + decodeUtf8 = decodeUtf8With lenientDecode > > I'd be very upset for that kind of a semantics change to sneak in under me. >
_______________________________________________ Libraries mailing list Libraries@haskell.org <javascript:;> <mailto:Libraries@haskell.org <javascript:;>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries <http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries>
_______________________________________________ Libraries mailing list Libraries@haskell.org <javascript:;> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
On Fri, 19 Aug 2016, Carter Schonwald wrote:
Why not propose adding a variant that exposes the error handling.
As previously mentioned, it is already there: decodeUtf8' :: ByteString -> Either UnicodeException Text
Deprecation is a heavy hammer when most users seem to know that it's partial or discover it rather quickly.
decodeUtf8 could be re-added with a more alerting name as proposed by Oleg Grenrus http://mail.haskell.org/libraries/2016-August/027265.html
Hi Niklas, AFAIK `text` is not maintained by CLC: https://wiki.haskell.org/Library_submissions#The_Core_Libraries (Please correct me if I'm wrong) You should probably work directly with the maintainer, e.g. open an issue. I think it is OK to discuss it on the list, but it should be clear that the maintainer is free to ignore any decision made here. Thanks, Yuras. On Thu, 2016-08-18 at 17:08 +0800, Niklas Hambüchen wrote:
I propose to add a deprecation pragma to the partial function Data.Text.decodeUtf8.
I see checks or decoding failure accidentally omitted from production code every week. I claim decodeUtf8 to be *the* most abused partial function in the current Haskell ecosystem (with `head` and `tail` at least everybody knows they should feel bad when typing them down).
decodeUtf8' with custom error message is a better solution in all cases I have seen and can come up with.
I therefore propose:
* Add deprecation pragma to decodeUtf8 * Do it with Text.Lazy and *16/*32 functions too * But keep them indefinitely (we don't want to break compiles)
I believe that deprecating decodeUtf8 will result in lots of useful warnings across the world every day and less software crashing at runtime.
Discussion period 2 months.
Thanks! _______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Ultimately any decision made would be made by the maintainer of the library, even if text were a core library. The library submissions process is to help inform a decision by the package maintainer by informing them of community preferences and to get the discussion out in the open where more people can talk about it and raise issues -- it isn't binding. While text isn't a "core library", it has been part of the platform for a long time, and this is a good place to at least start the discussion around any changes to it. Bryan remains of course free to accept of reject the proposals that come to him. -Edward On Thu, Aug 18, 2016 at 7:28 AM, Yuras Shumovich <shumovichy@gmail.com> wrote:
Hi Niklas,
AFAIK `text` is not maintained by CLC: https://wiki.haskell.org/Library_submissions#The_Core_Libraries (Please correct me if I'm wrong)
You should probably work directly with the maintainer, e.g. open an issue. I think it is OK to discuss it on the list, but it should be clear that the maintainer is free to ignore any decision made here.
Thanks, Yuras.
On Thu, 2016-08-18 at 17:08 +0800, Niklas Hambüchen wrote:
I propose to add a deprecation pragma to the partial function Data.Text.decodeUtf8.
I see checks or decoding failure accidentally omitted from production code every week. I claim decodeUtf8 to be *the* most abused partial function in the current Haskell ecosystem (with `head` and `tail` at least everybody knows they should feel bad when typing them down).
decodeUtf8' with custom error message is a better solution in all cases I have seen and can come up with.
I therefore propose:
* Add deprecation pragma to decodeUtf8 * Do it with Text.Lazy and *16/*32 functions too * But keep them indefinitely (we don't want to break compiles)
I believe that deprecating decodeUtf8 will result in lots of useful warnings across the world every day and less software crashing at runtime.
Discussion period 2 months.
Thanks! _______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
On Thu, Aug 18, 2016 at 2:08 AM, Niklas Hambüchen <mail@nh2.me> wrote:
I propose to add a deprecation pragma to the partial function Data.Text.decodeUtf8.
I'm not terribly open to adding a pragma for this, I'm afraid. Its documentation already clearly states that it will throw an exception. I understand that developers often fail to read documentation, but I don't want to train them to also ignore compiler output because they get permanent warnings that they can't suppress. I do of course agree that in hindsight, it would have been best to provide a safer function with this name.
Being the disruptive sort.... What if we DEPRECATE it for a cycle and then change its type? Almost all code that is no longer correct will fail to typecheck. On Aug 18, 2016 12:52 PM, "Bryan O'Sullivan" <bos@serpentine.com> wrote:
On Thu, Aug 18, 2016 at 2:08 AM, Niklas Hambüchen <mail@nh2.me> wrote:
I propose to add a deprecation pragma to the partial function Data.Text.decodeUtf8.
I'm not terribly open to adding a pragma for this, I'm afraid. Its documentation already clearly states that it will throw an exception. I understand that developers often fail to read documentation, but I don't want to train them to also ignore compiler output because they get permanent warnings that they can't suppress.
I do of course agree that in hindsight, it would have been best to provide a safer function with this name.
_______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
I like that idea. I've no right to argue with Bryan about his own library, but "read the docs" is the answer we get in Python, PHP, and Perl. Since it's demonstrably problematic in this case it seems the docs answer is not really working. On Thu, Aug 18, 2016 at 1:19 PM, David Feuer <david.feuer@gmail.com> wrote:
Being the disruptive sort.... What if we DEPRECATE it for a cycle and then change its type? Almost all code that is no longer correct will fail to typecheck.
On Aug 18, 2016 12:52 PM, "Bryan O'Sullivan" <bos@serpentine.com> wrote:
On Thu, Aug 18, 2016 at 2:08 AM, Niklas Hambüchen <mail@nh2.me> wrote:
I propose to add a deprecation pragma to the partial function Data.Text.decodeUtf8.
I'm not terribly open to adding a pragma for this, I'm afraid. Its documentation already clearly states that it will throw an exception. I understand that developers often fail to read documentation, but I don't want to train them to also ignore compiler output because they get permanent warnings that they can't suppress.
I do of course agree that in hindsight, it would have been best to provide a safer function with this name.
_______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
On Thu, Aug 18, 2016 at 10:33 AM, Elliot Cameron <eacameron@gmail.com> wrote:
I like that idea. I've no right to argue with Bryan about his own library, but "read the docs" is the answer we get in Python, PHP, and Perl.
I submit that you have every right to offer constructive feedback!
Since it's demonstrably problematic in this case it seems the docs answer is not really working.
Well ... I don't know about demonstrably problematic. One email from Niklas does not a convincing case make :-)
I submit that you have every right to offer constructive feedback!
Thank you. :)
One email from Niklas does not a convincing case make.
Fair point. I suppose we need to muster up a convincing case then. Niklas, would you send more emails please? One was not enough. ;) On Thu, Aug 18, 2016 at 1:56 PM, Bryan O'Sullivan <bos@serpentine.com> wrote:
On Thu, Aug 18, 2016 at 10:33 AM, Elliot Cameron <eacameron@gmail.com> wrote:
I like that idea. I've no right to argue with Bryan about his own library, but "read the docs" is the answer we get in Python, PHP, and Perl.
I submit that you have every right to offer constructive feedback!
Since it's demonstrably problematic in this case it seems the docs answer is not really working.
Well ... I don't know about demonstrably problematic. One email from Niklas does not a convincing case make :-)
On Thu, Aug 18, 2016 at 10:19 AM, David Feuer <david.feuer@gmail.com> wrote:
Being the disruptive sort.... What if we DEPRECATE it for a cycle and then change its type? Almost all code that is no longer correct will fail to typecheck.
The “almost” is the worrying part. ;) But I would second deprecation followed by a change in type or name, as I prefer not to use exceptions at all if I can help it. WARNING would be suitable if we could classify warnings, e.g.: {-# WARNING (partial) decodeUtf8 "…" #-}
participants (12)
-
Andreas Abel -
Bryan O'Sullivan -
Carter Schonwald -
David Feuer -
Edward Kmett -
Elliot Cameron -
evan＠evan-borden.com -
Henning Thielemann -
Jon Purdy -
Niklas Hambüchen -
Oleg Grenrus -
Yuras Shumovich