Announcing the release of network 3.0.0.0 http://hackage.haskell.org/package/network-3.0.0.0 Version 3.0.0.0 of network is less platform dependent, easier to build, easier to use correctly (yay no more String api), more extensible, and all around safer. Kazu Yamamoto and Tamar Christina deserve a special shout out, they put an exceptional amount of thought and work into this version and it really shows. Thanks to network's contributors for kicking the tires and continuing to keep quality high. Version 3.0.0.0 * Breaking change: the Network and Network.BSD are removed. Network.BSD is provided a new package: network-bsd. * Breaking change: the signatures are changed: ``` old fdSocket :: Socket -> CInt new fdSocket :: Socket -> IO CInt old mkSocket :: CInt -> Family -> SocketType -> ProtocolNumber -> SocketStatus -> IO Socket new mkSocket :: CInt Socket ``` * Breaking change: the deprecated APIs are removed: send, sendTo, recv, recvFrom, recvLen, htonl, ntohl, inet_addr, int_ntoa, bindSocket, sClose, SocketStatus, isConnected, isBound, isListening, isReadable, isWritable, sIsConnected, sIsBound, sIsListening, sIsReadable, sIsWritable, aNY_PORT, iNADDR_ANY, iN6ADDR_ANY, sOMAXCONN, sOL_SOCKET, sCM_RIGHTS, packSocketType, getPeerCred. * Breaking chage: SockAddrCan is removed from SockAddr. * Socket addresses are extendable with Network.Socket.Address. * "socket" is now asynchronous-exception-safe. [#336](https://github.com/haskell/network/pull/336) * "recvFrom" returns (0, addr) instead of throwing an error on EOF. [#360](https://github.com/haskell/network/pull/360) * All APIs are available on any platforms. * Build system is simplified. * Bug fixes. -- Evan Borden
Excellent! FYI Hackage isn't showing Haddocks for network yet, although I've noticed this for a few packages again recently. On Sat, 19 Jan 2019, 17.35 evan@evan-borden.com < evan@evanrutledgeborden.dreamhosters.com wrote:
Announcing the release of network 3.0.0.0 http://hackage.haskell.org/package/network-3.0.0.0
Version 3.0.0.0 of network is less platform dependent, easier to build, easier to use correctly (yay no more String api), more extensible, and all around safer. Kazu Yamamoto and Tamar Christina deserve a special shout out, they put an exceptional amount of thought and work into this version and it really shows. Thanks to network's contributors for kicking the tires and continuing to keep quality high.
Version 3.0.0.0
* Breaking change: the Network and Network.BSD are removed. Network.BSD is provided a new package: network-bsd. * Breaking change: the signatures are changed: ``` old fdSocket :: Socket -> CInt new fdSocket :: Socket -> IO CInt
old mkSocket :: CInt -> Family -> SocketType -> ProtocolNumber -> SocketStatus -> IO Socket new mkSocket :: CInt Socket ``` * Breaking change: the deprecated APIs are removed: send, sendTo, recv, recvFrom, recvLen, htonl, ntohl, inet_addr, int_ntoa, bindSocket, sClose, SocketStatus, isConnected, isBound, isListening, isReadable, isWritable, sIsConnected, sIsBound, sIsListening, sIsReadable, sIsWritable, aNY_PORT, iNADDR_ANY, iN6ADDR_ANY, sOMAXCONN, sOL_SOCKET, sCM_RIGHTS, packSocketType, getPeerCred. * Breaking chage: SockAddrCan is removed from SockAddr. * Socket addresses are extendable with Network.Socket.Address. * "socket" is now asynchronous-exception-safe. [#336](https://github.com/haskell/network/pull/336) * "recvFrom" returns (0, addr) instead of throwing an error on EOF. [#360](https://github.com/haskell/network/pull/360) * All APIs are available on any platforms. * Build system is simplified. * Bug fixes.
-- Evan Borden _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.
Hey Evan, it's great that work is being done on network, that is very appreciated. I do find one thing problematic though: Why do breaking type changes like `fdSocket` and `mkSocket` not go through a deprecation cycle? It's a best-practice to not break types of fundamental packages unless absolutely necessary. In this case it seems like these functions could have been deprecated, kept forever, and new functions with the improved types could have been added. This would avoid breakage for network's many users. Given that rest of the changelog in which deprecations are mentioned, it seems `network` contributers generall understand that and agree on that. Why was the deprecation approach not chosen for these functions? What in general is network's decision process for deprecation vs breaking change? Was any analysis performed that showed that only the fewest of network's 926 Hackage dependencies use these functions? Or should we expect major ecosystem breakages? Also interesting is that in https://github.com/haskell/network/commit/d69c3072071859a28a442ff713e90f8499... `fdSocket` was deprecated, but that deprecation was about a slightly different change and never made it into any Hackage release so users never saw an advance hint that `fdSocket` would soon be changed. Personally, it bugs me out that we can't get deprecations right in Haskell, when other ecosystems have understood and successfully executed the "deprecate and make better functions" approach for a very long time. But I'd be happy to hear why I'm wrong and this approach couldn't be used here. Niklas
Niklas,
I appreciate and empathize with your concerns. Care was taken to utilize a
deprecation cycle for as much of the API as possible, as we knew that these
changes would cause churn in the ecosystem. As you noted, some breaking
changes were not communicated early through these means. Instead we decided
to mark an epoch change in network to signal that there would be
significant breakage. This follows conventions established by Ed Kmett,
which are documented here https://pvp.haskell.org/faq/. I'm not sure a
specific convention exists for a situation where the type of a function
will change in a future version. I'd love to hear ideas.
Thank you very much for the feedback. Constructive criticism is necessary
to improve our handling of these issues and is always welcome with open
arms.
On Sat, Jan 19, 2019 at 9:56 AM Niklas Hambüchen
Hey Evan,
it's great that work is being done on network, that is very appreciated. I do find one thing problematic though:
Why do breaking type changes like `fdSocket` and `mkSocket` not go through a deprecation cycle?
It's a best-practice to not break types of fundamental packages unless absolutely necessary. In this case it seems like these functions could have been deprecated, kept forever, and new functions with the improved types could have been added. This would avoid breakage for network's many users.
Given that rest of the changelog in which deprecations are mentioned, it seems `network` contributers generall understand that and agree on that. Why was the deprecation approach not chosen for these functions?
What in general is network's decision process for deprecation vs breaking change? Was any analysis performed that showed that only the fewest of network's 926 Hackage dependencies use these functions? Or should we expect major ecosystem breakages?
Also interesting is that in https://github.com/haskell/network/commit/d69c3072071859a28a442ff713e90f8499... `fdSocket` was deprecated, but that deprecation was about a slightly different change and never made it into any Hackage release so users never saw an advance hint that `fdSocket` would soon be changed.
Personally, it bugs me out that we can't get deprecations right in Haskell, when other ecosystems have understood and successfully executed the "deprecate and make better functions" approach for a very long time. But I'd be happy to hear why I'm wrong and this approach couldn't be used here.
Niklas
-- -- Evan Borden
Thanks for the quick reply!
I'd love to hear ideas.
I recommend the following way: * Never change the type of an existing function unless absolutely necessary. * Do not change the the behaviour/semantics of a function unless it's an obvious fix/improvement. (In other words, if somebody may in some reasonable way rely on the old behaviour, don't change it). * Improve functions by introducing new functions with names, marking the old functions as deprecated. * GHC will show deprecation warnings to users. ** In the deprecation message, point out the intended replacement. ** If a deprecated function is planned to be removed in the next release, say it in the message. * Consider removing deprecated functions after some years. But only if e.g. maintainability demands it. ** In some cases, consider moving the functions to an `.Old` module or similar. Key point: * Use deprecations and new functions liberally to make progress. Almost never break existing functions. I like to call this approach "the Java way of deprecation", and it helps an ecosystem to move forward swiftly while keeping people happy a whole lot. This is because it allows incremental transitions instead of hard cutoff points that often don't align with people's schedules or test plans. How this relates to the PVP: This thought process happens *before* PVP considerations enter the stage. After you've made your changes (hopefully as few breaking ones as possible), you can use the PVP to determine what the new version should be. The reverse logic should not be applied: If some change made demands a major version bump (e.g. removal of an old deprecated function that nobody uses), that does not "allow" other functions to be removed more liberally. The rationale is that our goal is not to do as much as a given version jump allows, but to minimise breaking changes for users even when we know that some breaking changes must be made. (Also, while the PVP as mentioned doesn't tell you what to do with your package contents and applies afterwards, the diagram in https://pvp.haskell.org/pvp-decision-tree.svg mentions "Consider renaming the function instead" next to "Did the behaviour of any exported functions change", so I think what I recommend is in the spirit of those that came up with the process). Concrete example: This is how I imagine applying the above to network 3.0.0.0 would have worked: * Deprecate `send`, `sendTo`, `recv`, etc. (As correctly done in network-2.7) * Do NOT remove them. Leave at least 2 years time before touching them. * In this case, move them to Network.Socket.String, instead of removing them. For 2 reasons: ** These functions have been this ways since forever and lots of code will use them. Putting them into a legacy module will allow projects to trivially get into a compiling state again with one `import` change, vs having to fix every use site. This eases migration. ** While certainly not a good idea, the String based functions aren't so fatally flawed that they have no justification for further existence, so keeping them in a legacy module would do no harm. * Introduce `socketToFd :: Socket -> IO CInt` * Deprecate `fdSocket`, saying why it's bad and that `socketToFd` is the replacement * Keep `fdSocket` forever (Just to be clear, I think the `String` based network API is bug; a reasonable and modern programming ecosystem shouldn't have that, and this should be addressed. So I'm not recommending this approach because I think these functions are great, but because I think this is how to transition away from design bugs in general.) Then, looking at the above changes, if we follow PVP, we'd look at the PVP decision tree, and determine what the new version should be that way. If (after a long time period) we'd do the "move to Network.Socket.String", it would tell us that a major bump is necessary. Of course one can also introduce a major version bump when PVP says that a minor bump would be sufficient if one wants to "mark an epoch change" -- that is at the liberty of the maintainer. Discussion: I found this approach to work very well and am convinced of it. But I'm happy to hear other people's opinion about it to see if it's really as agreeable as I think. If yes, I'd be happy to write down these deprecation guidelines in some repo so that projects can refer to them and say "we follow that general approach" (similar to the PVP FAQ link). What do you think? Niklas
On Sat, Jan 19, 2019 at 06:38:09PM +0100, Niklas Hambüchen wrote:
I recommend the following way: [...] I'd be happy to write down these deprecation guidelines in some repo so that projects can refer to them
I think that's a great idea. Opaleye roughly follows your specification. I try to announce deprecations in major version N, implement the deprecation in N+1 and remove the deprecated entity no earlier than N+2. NB that network could still be changed to follow your plan if a new version were released and 3.0.0.0 deprecated (hah!). Tom
While those are good suggestions, the biggest problem is not whether a
given deprecation strategy is correct or not.
The biggest problem with the PVP in this case is that it does *not say
anything about how to release a "pre" release* so any deprecation strategy
could be tested by the community.
This makes the concept of *learning* painful, because mistakes are way too
expensive.
Can this be fixed first, then later we can figure out how to do
deprecations? Or am I wrong?
Alexander
On Sat, Jan 19, 2019 at 6:38 PM Niklas Hambüchen
Thanks for the quick reply!
I'd love to hear ideas.
I recommend the following way:
* Never change the type of an existing function unless absolutely necessary. * Do not change the the behaviour/semantics of a function unless it's an obvious fix/improvement. (In other words, if somebody may in some reasonable way rely on the old behaviour, don't change it). * Improve functions by introducing new functions with names, marking the old functions as deprecated. * GHC will show deprecation warnings to users. ** In the deprecation message, point out the intended replacement. ** If a deprecated function is planned to be removed in the next release, say it in the message. * Consider removing deprecated functions after some years. But only if e.g. maintainability demands it. ** In some cases, consider moving the functions to an `.Old` module or similar.
Key point: * Use deprecations and new functions liberally to make progress. Almost never break existing functions.
I like to call this approach "the Java way of deprecation", and it helps an ecosystem to move forward swiftly while keeping people happy a whole lot. This is because it allows incremental transitions instead of hard cutoff points that often don't align with people's schedules or test plans.
How this relates to the PVP: This thought process happens *before* PVP considerations enter the stage. After you've made your changes (hopefully as few breaking ones as possible), you can use the PVP to determine what the new version should be. The reverse logic should not be applied: If some change made demands a major version bump (e.g. removal of an old deprecated function that nobody uses), that does not "allow" other functions to be removed more liberally. The rationale is that our goal is not to do as much as a given version jump allows, but to minimise breaking changes for users even when we know that some breaking changes must be made.
(Also, while the PVP as mentioned doesn't tell you what to do with your package contents and applies afterwards, the diagram in https://pvp.haskell.org/pvp-decision-tree.svg mentions "Consider renaming the function instead" next to "Did the behaviour of any exported functions change", so I think what I recommend is in the spirit of those that came up with the process).
Concrete example:
This is how I imagine applying the above to network 3.0.0.0 would have worked:
* Deprecate `send`, `sendTo`, `recv`, etc. (As correctly done in network-2.7) * Do NOT remove them. Leave at least 2 years time before touching them. * In this case, move them to Network.Socket.String, instead of removing them. For 2 reasons: ** These functions have been this ways since forever and lots of code will use them. Putting them into a legacy module will allow projects to trivially get into a compiling state again with one `import` change, vs having to fix every use site. This eases migration. ** While certainly not a good idea, the String based functions aren't so fatally flawed that they have no justification for further existence, so keeping them in a legacy module would do no harm. * Introduce `socketToFd :: Socket -> IO CInt` * Deprecate `fdSocket`, saying why it's bad and that `socketToFd` is the replacement * Keep `fdSocket` forever
(Just to be clear, I think the `String` based network API is bug; a reasonable and modern programming ecosystem shouldn't have that, and this should be addressed. So I'm not recommending this approach because I think these functions are great, but because I think this is how to transition away from design bugs in general.)
Then, looking at the above changes, if we follow PVP, we'd look at the PVP decision tree, and determine what the new version should be that way. If (after a long time period) we'd do the "move to Network.Socket.String", it would tell us that a major bump is necessary. Of course one can also introduce a major version bump when PVP says that a minor bump would be sufficient if one wants to "mark an epoch change" -- that is at the liberty of the maintainer.
Discussion:
I found this approach to work very well and am convinced of it. But I'm happy to hear other people's opinion about it to see if it's really as agreeable as I think. If yes, I'd be happy to write down these deprecation guidelines in some repo so that projects can refer to them and say "we follow that general approach" (similar to the PVP FAQ link).
What do you think?
Niklas _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.
Hello,
* Never change the type of an existing function unless absolutely necessary.
The change of mkSocket and fdSocket is inevitable because the old definition of Socket is: data Socket = MkSocket CInt Family SocketType ProtocolNumber (MVar SocketStatus) while new one is: data Socket = Socket !(IORef CInt) CInt {- for Show -} Why this drastic change? Well, believing or not, old Socket cannot be GCed because of MVar! mkWeakMVar does not solve this issue. So, we decided to get rid of MVar. But without SocketStatus (ie Socket CInt), "close" becomes unsafe if "close" is called multiple time. To fix this, IORef was introduced. Now a closed Socket contains (-1). If you type "fdScoket" or "GC" to the github repo search, you can find a lot of discussions about this. --Kazu
participants (7)
-
Alexander Kjeldaas -
Bryan Richter -
Evan Borden -
evan＠evan-borden.com
-
Kazu Yamamoto -
Niklas Hambüchen -
Tom Ellis