Re: Splitting Network.URI from the network package

I asked Duncan for some feedback, so here's a summary for his benefit: First Michael's (and our) problem statement: - The network package is a pain to install on Windows for most users (especially new users), since it requires msys. - Most Windows users therefore install the Haskell Platform, avoiding the msys dependency. - The current release of HP installs text version 0.11.3.1. text is a dependency of parsec, and parsec is a dependency of network. Therefore, you can't build against a new version of text *and* use the network package without recompiling network, which as I mentioned is difficult. - A number of popular packages depend on newer versions of text. For example, since 0.8.0.0, aeson requires text version 1.1.0.0 or later. as does attoparsec since version 0.12.0.0. The two options: *Option 1: Users of Network.URI use a flag* - network-2.6 no longer exposes Network.URI. - network-uri-2.6 exposes Network.URI. User who want only network-uri writes: flag network-uri description: Get Network.URI from the network-uri package default: True library if flag(network-uri) build-depends: network-uri >= 2.6 else build-depends: network < 2.6 (If the user also wants network he/she adds it to the first build-depends line.) Cons: - More typing (an extra flag declaration, an if statement, and an extra build-depends line). - Could trigger possible unknown solver bugs (but there are no confirmed such bugs). *Options 2: Users of Network.URI depend on both network and a specially crafted network-uri* - network-2.5 exposes Network.URI. - network-2.6 no longer exposes Network.URI. - network-uri-2.5 Exposes no modules. Depends on < network-2.6. - network-uri-2.6 exposes Network.URI. Depends on network >= 2.6. User who want network-uri (only) writes: build-depends: network >= 2.5 && < 2.7, network-uri >= 2.5 && < 2.7 Cons: - network-uri has a false dependency on network (i.e. it doesn't actually need that package). - You can't build against a new version of text *and* use the network-uri package (this is the current problem we have with network in the problem description). - We need to teach users about the above special build-depends construction, just for use with the network and network-uri packages. Users who don't know of this special construction will still use a flag (and thus get a mix of cons from both approaches.) - network-uri-2.5 is an empty package, which might lead to slight confusion for some users. Cheers, Johan On Tue, Aug 12, 2014 at 2:59 PM, Michael Snoyman wrote:

Just to be clear from my side: even if we were 100% certain that all issues we've experienced in the past from the solver with flags would not exist with the newest versions of cabal-install, I would still want to follow the same proposal as I gave at the beginning of this thread, because forcing every user of Network.URI to add conditionals to their cabal file is IMO more painful than simply having two versions of the network-uri package released.

The only modification to the proposal that I think would make sense would be to have a single version of network-uri, and the exposing of the Network.URI module be conditional on the version of network. But I'd rather stick with the original proposal than encounter possibly unknown solver bugs.

On Tue, Aug 12, 2014 at 3:44 PM, Johan Tibell wrote:

...

Is this still the case with cabal-install 1.20.0.3/1.18.0.5? We recently bumped the max backtracking limit by about 10x. My current understanding of the state of the solver is that as of the cabal-install versions I gave

* there are no known bugs that cause the solver to find a plan if there is one (if the backtracking limit isn't reached) and * the backtracking limit is high enough for the packages we currently have on Hackage.

If this isn't true please let me know and we'll look into it.

If there's a problem I want to fix it at its source (i.e. in the solver) rather than trying too many workarounds in .cabal files. The latter leave residues that we have to live with for quite a long time. I'm happy to backport any solver fixes if it means we can have cleaner .cabal files.

On Sun, Aug 10, 2014 at 4:30 PM, Edward Kmett wrote:

...

The problem is more that there seem to be now packages that are within a factor of 2 of the total amount of backtracking they can do, so it seems pretty much every flag way down at the bottom of the package hierarchy pushes something out past the limit.

I hear something from someone every time I add a flag. Take that for what it's worth.

-Edward

On Sun, Aug 10, 2014 at 5:48 AM, Johan Tibell wrote:

...

I don't think (but Andres can say more) that a simple non-nested flag should give the backtracker problems (bugs or increase the search space so much that it won't find a solution.)

On Sun, Aug 10, 2014 at 3:22 AM, Edward Kmett wrote:

...

The problem with a flag based solution is it puts extra pressure on the backtracker in cabal, and when you pile enough of those up then you get things where cabal refuses to find a solution. Not for you, but for packages 20 dependencies later in the chain.

What I do with transformers-compat is I have multiple minor versions of the same package extant to work around bugs in the backtracker and then have separate point release that can work with transformers 0.2, one for transformers 0.3 and one for transformers 0.4, then users can depend on both transformers >= 0.2 and transformers-compat >= 0.3.3 and their code 'just works' regardless of which package is supplying the module.

On Wed, Aug 6, 2014 at 6:09 AM, Johan Tibell wrote:

...

On Mon, Aug 4, 2014 at 6:21 PM, Michael Snoyman wrote: > The idea is that users of Network.URI will be able to add a dependency along > the lines of: > > build-depends: network >= 2.5 && < 2.7, network-uri >= 2.5 && < 2.7 > > and work with both the pre- and post-split versions of the package, without > any conditional compilation or cabal flags. I'm sensitive to adding any > requirements of cabal flags, since (1) it clutters cabal files quite a > bit[1], and (2) we just went through some painful cabal-install dependency > solver issues around flags.

I agree that using flags to encode OR is a bit heavy weight. We should really support || in build-depends. For completeness, here's what a flag-based solution looks like. Assuming that network-uri-2.6 has the URI module and network-2.6 no longer has it we get this flag setup:

flag network-uri description: Get Network.URI from the network-uri package default: True

library if flag(network-uri) build-depends: network-uri >= 2.6 else build-depends: network < 2.6

If the user wants something else from network (e.g. Network.Socket) it would be

flag network-uri description: Get Network.URI from the network-uri package default: True

library if flag(network-uri) build-depends: network-uri >= 2.6, network >= 2.6 else build-depends: network < 2.6

Lets see if I understood your scheme correctly. We'd have

* network-2.5: Has URI module. * network-uri-2.5: Doesn't have URI module. Depends on < network-2.6 (i.e. network-2.5 in this example). * network-2.6: Doesn't have URI module. * network-uri-2.6: Has URI module. Depends on network >= 2.6 (i.e. network 2.6 in this example).

Given

build-depends: network >= 2.5 && < 2.7, network-uri >= 2.5 && < 2.7

legal combinations of the above packages are

* network-2.5 and network-uri-2.5, which gives URI through network, and * network-2.6 and network-uri-2.6, which gives URI through network-uri.

There are two implications of this, which are slightly strange:

* There's no point in anyone depending on only network-uri-2.5 (as it doesn't expose anything). * network-uri-2.6 and later will have to have a lower dependency on network forever, even though network-uri doesn't use anything from network (this is reflected when you discuss the upper bound in the context of the PVP later).

I don't know if there's any other implications. Can anyone think of any?

-- Johan

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries