On Tue, Dec 28, 2010 at 7:37 PM, Ross Paterson wrote:

That would mean adding deepseq as a GHC boot package.

If we agree that the proposal is the right thing to do. I think we should add deepseq as a boot package, unless it's a major pain for the GHC developers. Johan

On Tue, Dec 28, 2010 at 11:03 PM, Alexander Dunlap wrote:

It seems like a huge number of packages depend on containers, and almost everyone seems to have it installed anyway, so I don't see what we gain by removing the dependency on containers.

Problems with the current state of affairs: * The current NFData instances for the containers data types are inefficient, as deepseq don't have access to the internals of the containers package. For example, here's the Map instance instance (NFData k, NFData a) => NFData (Data.Map.Map k a) where rnf = rnf . Data.Map.toList This causes unnecessary allocation of list cells. In addition, the correctness of this definition assumes something about how the Map data type looks like internally. Forcing the keys and values might not be enough to fully evaluate a Map value. It happens to work in this case but could break if the Map data type ever added a non-strict field other than the one the holds the value. * Having containers below deepseq in the dependency hierarchy slightly increases the risk of diamond dependencies. Example: 1. A new major version of containers is released, version X.Y. 2. Package foo depends on containers-X.Y.* and the latest version of deepseq. 3. deepseq hasn't been updated yet so it depends on containers.X-1.Y.* foo is now unbuildable as the containers constraints can't be satisfied. * Why should containers be treated specially? Should deepseq depend on all libraries that want to provide NFData instances? * The dependency hierarchy gets weird. Why does text depend on containers? It's logically at the same or a lower level than containers. Johan

On Wed, Dec 29, 2010 at 2:41 PM, Christian Maeder wrote:

I don't think, that orphaned instances can be avoided in the long run. Therefore I would rather see a separate package for the NFData instances of container types.

Why not? The general recipe is to define type classes in separate packages so they can appear on the bottom of the dependency pyramid. That way any package that defines a data type can depend on the package that defines the type class and thus avoid creating orphan instances. Orphan instances should be avoided as they can cause hard to prevent and hard to fix breakages in large code bases. Johan

On Tue, Jan 4, 2011 at 11:51 AM, Christian Maeder wrote:

classes and data types are usually created independently and one cannot expect all data type maintainers to continuously adapt their packages to provide instances for new classes.

That is a downside of the approach I'm proposing. It's small in comparison to the downsides of the alternatives. Note that the opposite approach, having the package that defines the type class define the instances, would mean that the maintainers of that package would have to add a new instance for every data type (that makes sense) that gets added to Hackage. Since the number of data types is much larger than the number of type classes, my proposal distributes the work among many more maintainers. The problem with having the package defining the class depend on the packages defining the data types is that this approach doesn't scale. If we applied it consistently the deepseq package should depend on every package on Hackage that defines a data type (i.e. every package on Hackage), as NFData instances make sense for just about every data type. The current deepseq packages picks two at semi-random (i.e. due to legacy reasons.)

Will the next step be to move the Binary instances from the binary package to containers, too? (There are plenty of other serialization classes!)

Aside: I think the binary package should be split into two packages. It currently exports two completely separate pieces of functionality: * Low-level primitives for reading/writing machine native types: integers (unsigned or not) and binary blobs. * A specific and undocumented data format, via the Binary class. I don't think a type class approach is necessary the right thing. However, if we stick with one I suggest that the package provides instances for the data type in base, as the package (and indeed almost all packages) have to depend on base anyway. If the data format is to be extended to serialize containers the instances should be defined in terms of another type class and not in terms of a particular type. Tying a data format to a particular implementation of a container doesn't sound like a great idea. Here's a sketch of what this could look like: class IMap where empty :: ... insert :: ... instance IMap m => Binary m where get = do -- decode data and call insert repeatedly. The binary package would depend on the package that defines this data type.

...

Orphan instances should be avoided as they can cause hard to prevent and hard to fix breakages in large code bases.

The problem with large code bases are only duplicate orphaned instances that are added only as non-separated parts of other code.

If all code would be based on the same instances (provided by a central package on hackage!) I see no problem.

If I understand you correctly you're suggesting that the problem is solved by convention e.g. that all instances belonging to a particular type class is found in a particular package and no one should define an instance of this type class elsewhere? Is that a correct interpretation? I think there are at least three issues with this approach: * The package that defines the instance might have to depend on all of Hackage (see above). This is the same as when the package that defines the type class also defines the instances. * I don't think this is enforceable, even by convention. Where do you put instances for data types not on Hackage (e.g. in some company's source control repo)? * You need to tell everyone to not define instances for data types in the same package as they define the data type as all instances are supposed to be in the special instances package. If they do add the instance in the package that defines the data type, things will break when the *-instances package adds the same instance.

If NFData is such an important class it should go into the base package. Since other data types depend on base anyway, then there is no need to change the dependency from "base" to "base, deepseq" for many data packages.

Any packages dependency problem can be solved by having only one package (e.g. base). But all Haskell code (not even all type classes people might ever define) can live in base. We can try to avoid the question of where to declare what by say that some type classes are "important" and goes into base and pretend other type classes don't exist. This doesn't really answer the question of where to define type classes and their instances, in general, though.

Is it possible to find out where (if) NFData instances of container types are actually used (for hackage packages)?

They are used in a bunch of Criterion benchmarks for different packages at the moment. Johan

On Tue, Jan 4, 2011 at 1:02 PM, Christian Maeder wrote:

Why do you propose this for the container package only? Don't the same arguments apply to the array-package, too?

Yes, an oversight on my part. array should go too. Johan

On Wed, Jan 5, 2011 at 9:24 PM, Ian Lynagh wrote:

However, I think types in the GHC bootlibs should be an exception to this rule. Otherwise any classes that someone wants the bootlibs to have an instance for will need to be added to the bootlibs, and we'd like to keep the set of bootlibs as small as possible.

What would happen if GHC HQ find an interesting package on Hackage and put it in bootlib because they found it useful for GHC? Would that library author then not be allowed to add new dependencies to his/her package? I find that to be quite a weird model. Normally you don't get to dictate what the respective authors of packages you depend on do with their packages. (You can ask nicely of course.) The reason I want to get rid of the container dependency is that I don't want the data structure package I'm working depend on another data structure package that provides similar functionality. That seems, weird. I'd rather just skip the NFData instances. Cheers, Johan

On 05/01/11 22:10, Johan Tibell wrote:

On Wed, Jan 5, 2011 at 9:24 PM, Ian Lynagh wrote:

...

However, I think types in the GHC bootlibs should be an exception to this rule. Otherwise any classes that someone wants the bootlibs to have an instance for will need to be added to the bootlibs, and we'd like to keep the set of bootlibs as small as possible.

What would happen if GHC HQ find an interesting package on Hackage and put it in bootlib because they found it useful for GHC? Would that library author then not be allowed to add new dependencies to his/her package? I find that to be quite a weird model. Normally you don't get to dictate what the respective authors of packages you depend on do with their packages. (You can ask nicely of course.)

Of course we don't get to dictate. If the library grew some dependencies that we couldn't put in bootlibs we have two options: stop using the library in GHC, or carry on using the old version. We already have one library like this: binary, which is shipped as ghc-binary so that users can install and upgrade binary without breaking GHC. The renaming is only necessary due to limitations in Cabal and the package system, which don't know that binary is only used internally in GHC and could safely be mixed with another version of binary without problems.

The reason I want to get rid of the container dependency is that I don't want the data structure package I'm working depend on another data structure package that provides similar functionality. That seems, weird. I'd rather just skip the NFData instances.

It seems a little weird, but I don't think it has any undesirable consequences in practice, does it? Cheers, Simon

Same here. Is it really that big a deal to add deepseq to the boot libraries? It seems like GHC might eventually want to use some of its own parallelism constructs for compilation, and then deepseq could be required anyway? Dan On Thu, Jan 6, 2011 at 11:23 AM, Iavor Diatchki wrote:

Hello, I agree with Johan that the dependency seems backward. "containers" is a perfectly fine package but there is no reason to assume that everyone has it. Consider, for example, a situation where I want to write a replacement for "containers"---it would certainly be undesirable for my package to depend on "containers". -Iavor

On Wed, Jan 5, 2011 at 2:45 PM, Simon Marlow wrote:

...

On 05/01/11 22:10, Johan Tibell wrote:

...

On Wed, Jan 5, 2011 at 9:24 PM, Ian Lynagh wrote:

...

However, I think types in the GHC bootlibs should be an exception to this rule. Otherwise any classes that someone wants the bootlibs to have an instance for will need to be added to the bootlibs, and we'd like to keep the set of bootlibs as small as possible.

What would happen if GHC HQ find an interesting package on Hackage and put it in bootlib because they found it useful for GHC? Would that library author then not be allowed to add new dependencies to his/her package? I find that to be quite a weird model. Normally you don't get to dictate what the respective authors of packages you depend on do with their packages. (You can ask nicely of course.)

Of course we don't get to dictate. If the library grew some dependencies that we couldn't put in bootlibs we have two options: stop using the library in GHC, or carry on using the old version.

We already have one library like this: binary, which is shipped as ghc-binary so that users can install and upgrade binary without breaking GHC. The renaming is only necessary due to limitations in Cabal and the package system, which don't know that binary is only used internally in GHC and could safely be mixed with another version of binary without problems.

The reason I want to get rid of the container dependency is that I

...

don't want the data structure package I'm working depend on another data structure package that provides similar functionality. That seems, weird. I'd rather just skip the NFData instances.

It seems a little weird, but I don't think it has any undesirable consequences in practice, does it?

Cheers, Simon

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

On Thu, Jan 06, 2011 at 06:57:33PM +0100, Christian Maeder wrote:

Alternatively, add the module Control.Deepseq to base without the array and container instances (and nuke the deepseq package). Maybe it should even be renamed to Control.NFData.

<insert my standard objection to adding things to base here> Thanks Ian

I've summarized the discussion here: http://hackage.haskell.org/trac/ghc/ticket/4868#comment:4 Ian and Christian, I'm not sure what you're feeling on the proposal are. Could you please give a more explicit opinion either for or against? Thanks, Johan

On Mon, Jan 10, 2011 at 05:27:28PM +0100, Johan Tibell wrote:

I've summarized the discussion here: http://hackage.haskell.org/trac/ghc/ticket/4868#comment:4

Ian and Christian, I'm not sure what you're feeling on the proposal are. Could you please give a more explicit opinion either for or against?

Against (although I would prefer it to orphaning instances). Thanks Ian

On 06/01/11 17:06, Daniel Peebles wrote:

Same here. Is it really that big a deal to add deepseq to the boot libraries? It seems like GHC might eventually want to use some of its own parallelism constructs for compilation, and then deepseq could be required anyway?

You're right, it's not a big deal, and I'm not arguing against Johan's proposal. Cheers, Simon

Dan

On Thu, Jan 6, 2011 at 11:23 AM, Iavor Diatchki mailto:iavor.diatchki@gmail.com> wrote:

Hello, I agree with Johan that the dependency seems backward. "containers" is a perfectly fine package but there is no reason to assume that everyone has it. Consider, for example, a situation where I want to write a replacement for "containers"---it would certainly be undesirable for my package to depend on "containers". -Iavor

On Wed, Jan 5, 2011 at 2:45 PM, Simon Marlow mailto:marlowsd@gmail.com> wrote:

On 05/01/11 22:10, Johan Tibell wrote:

On Wed, Jan 5, 2011 at 9:24 PM, Ian Lynaghmailto:igloo@earth.li> wrote:

However, I think types in the GHC bootlibs should be an exception to this rule. Otherwise any classes that someone wants the bootlibs to have an instance for will need to be added to the bootlibs, and we'd like to keep the set of bootlibs as small as possible.

What would happen if GHC HQ find an interesting package on Hackage and put it in bootlib because they found it useful for GHC? Would that library author then not be allowed to add new dependencies to his/her package? I find that to be quite a weird model. Normally you don't get to dictate what the respective authors of packages you depend on do with their packages. (You can ask nicely of course.)

Of course we don't get to dictate. If the library grew some dependencies that we couldn't put in bootlibs we have two options: stop using the library in GHC, or carry on using the old version.

We already have one library like this: binary, which is shipped as ghc-binary so that users can install and upgrade binary without breaking GHC. The renaming is only necessary due to limitations in Cabal and the package system, which don't know that binary is only used internally in GHC and could safely be mixed with another version of binary without problems.

The reason I want to get rid of the container dependency is that I don't want the data structure package I'm working depend on another data structure package that provides similar functionality. That seems, weird. I'd rather just skip the NFData instances.

It seems a little weird, but I don't think it has any undesirable consequences in practice, does it?

Cheers, Simon

_______________________________________________ Libraries mailing list Libraries@haskell.org mailto:Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org mailto:Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries