Re: [Hs-Generics] Re: Owning SYB
Hello all, As Johan mentioned, here in Utrecht we are working on libraries for generic programming. We want to make it easier for people to use generic libraries, so we are packaging EMGM [1] and a library for generic programming for mutually recursive datatypes [2]. We intend to release these on Hackage soon (Summer vacations are delaying us a bit), along with useful generic applications (a zipper and a generic rewriting framework). Maintaining SYB fits well in this idea, and if no other natural maintainers volunteer, I (with some support from the other people at Utrecht) am happy to take it upon me. I probably won't do heavy development on the library, but including patches, and providing support is fine. We're also planning to maintain EMGM here in Utrecht, although we didn't develop that ourselves. Recently, (at least) Claus and Oleg have been posting interesting suggestions of improvements/modifications to SYB. Those should be further analyzed and discussed, and finally introduced (or not) in the library. The generic map for SYB, for instance, evolved from the "impossible to implement", through the "unsafe implementation", until the latest gmap2 as described by Oleg [3]. If further tests show this function behaves as expected, then it's clearly a good candidate for extending SYB. We should also rethink if other things previously deemed impossible remain so. Maintaining SYB, alongside with the other generic libraries, will require things such as: * Releasing packages in Hackage, properly documented with Haddock; * Updating such packages as necessary for new releases of GHC; * Writing examples of how to use the libraries (from a user perspective); * Writing testsuites, which are important for checking backwards compatibility of any changes; * Having an updated webpage linking to the library sources, documentation, possibly a bug tracker, etc. These are all things we plan to do for the libraries. Additionally, we could think of improving syb-with-class [4] in parallel with regular SYB. This is something to ask to its maintainer. Cheers, Pedro [1] http://books.google.com/books?id=OyY3ioMJRAsC&pg=PA199&sig=ACfU3U1nczeRAIjN9mc_vYnL1LnYAs70NA [2] http://www.cs.uu.nl/research/techreps/UU-CS-2008-019.html [3] http://www.haskell.org/pipermail/generics/2008-July/000362.html [4] http://hackage.haskell.org/cgi-bin/hackage-scripts/package/syb-with-class
Thank you! This is great news. Having a set of generic libraries with a single owner would be great. (Some comparative discussion would be useful too, to help people select the right package for their needs.) Simon From: generics-bounces@haskell.org [mailto:generics-bounces@haskell.org] On Behalf Of José Pedro Magalhães Sent: 01 August 2008 07:37 To: generics@haskell.org; libraries@haskell.org Subject: Re: [Hs-Generics] Re: Owning SYB Hello all, As Johan mentioned, here in Utrecht we are working on libraries for generic programming. We want to make it easier for people to use generic libraries, so we are packaging EMGM [1] and a library for generic programming for mutually recursive datatypes [2]. We intend to release these on Hackage soon (Summer vacations are delaying us a bit), along with useful generic applications (a zipper and a generic rewriting framework). Maintaining SYB fits well in this idea, and if no other natural maintainers volunteer, I (with some support from the other people at Utrecht) am happy to take it upon me. I probably won't do heavy development on the library, but including patches, and providing support is fine. We're also planning to maintain EMGM here in Utrecht, although we didn't develop that ourselves. Recently, (at least) Claus and Oleg have been posting interesting suggestions of improvements/modifications to SYB. Those should be further analyzed and discussed, and finally introduced (or not) in the library. The generic map for SYB, for instance, evolved from the "impossible to implement", through the "unsafe implementation", until the latest gmap2 as described by Oleg [3]. If further tests show this function behaves as expected, then it's clearly a good candidate for extending SYB. We should also rethink if other things previously deemed impossible remain so. Maintaining SYB, alongside with the other generic libraries, will require things such as: * Releasing packages in Hackage, properly documented with Haddock; * Updating such packages as necessary for new releases of GHC; * Writing examples of how to use the libraries (from a user perspective); * Writing testsuites, which are important for checking backwards compatibility of any changes; * Having an updated webpage linking to the library sources, documentation, possibly a bug tracker, etc. These are all things we plan to do for the libraries. Additionally, we could think of improving syb-with-class [4] in parallel with regular SYB. This is something to ask to its maintainer. Cheers, Pedro [1] http://books.google.com/books?id=OyY3ioMJRAsC&pg=PA199&sig=ACfU3U1nczeRAIjN9mc_vYnL1LnYAs70NA [2] http://www.cs.uu.nl/research/techreps/UU-CS-2008-019.html [3] http://www.haskell.org/pipermail/generics/2008-July/000362.html [4] http://hackage.haskell.org/cgi-bin/hackage-scripts/package/syb-with-class
Hi Pedro, and thanks for volunteering! I include a summary of where I'm at, for your information (and that of other interested readers;-)
Recently, (at least) Claus and Oleg have been posting interesting suggestions of improvements/modifications to SYB. Those should be further analyzed and discussed, and finally introduced (or not) in the library. The generic map for SYB, for instance, evolved from the "impossible to implement", through the "unsafe implementation", until the latest gmap2 as described by Oleg [3]. If further tests show this function behaves as expected, then it's clearly a good candidate for extending SYB. We should also rethink if other things previously deemed impossible remain so.
Further discussion welcome, of course!-) And it isn't just about getting fmap/traverse/.. from Data/Typeable, but also about reorganizing imports of Data instances, and providing alternatives of everywhere/everything that avoid traversals of irrelevant subterms. A current snapshot of my code can be found here: http://www.cs.kent.ac.uk/~cr3/tmp/syb/syb-utils-0.0.2008.7.31.tar.gz It currently contains: a) a suggested split for the Instances module, with alternatives to Data.Generics that either export only Standard instances, or no instances at all. On top of the existing Syb in base, these are somewhat tricky to use, because of the implicit re-exports of Data.Generics.Instances from other libraries http://www.haskell.org/pipermail/generics/2008-July/000371.html and the long-standing GHC "session accumulates instances" bug http://hackage.haskell.org/trac/ghc/ticket/2182 http://www.haskell.org/ghc/docs/latest/html/users_guide/bugs.html#bugs-ghc relevant modules: Data.Generics.Alt, Data.Generics.NoInstances, Data.Generics.Instances.Standard, Data.Generics.Instances.Dubious, examples\Examples.hs status: I think this change should go into base (perhaps renaming Data.Generics.Alt to Data.Generics.Standard, and deprecating Data.Generics and Data.Generics.Instances), for all the reasons discussed here recently, and because that GHC bug makes it near impossible to provide this change as an addon to base. Existing importers of Data.Generics or Data.Generics.Instances in core and extralibs should be redirected to one of the new modules. b) variants of everywhere/everything/mkQ/.. that retain the type domain of generically extended queries, build substructure type maps of types to be traversed, and use a slight generalisation of the Uniplate PlateData trick to avoid traversals of irrelevant substructures (usually but not always including, and not limited to, Strings) relevant modules: Data.Generics.GPS, examples\GPSbenchmark.hs, examples\CompanyDatatypes.hs status: This could be provided as an addon package. Performance of generic queries and transformation on the usual Paradise benchmark improve as expected; there is some overhead, which is visible in an alternative benchmark where there are no irrelevant substructures. The current code also tries to replace the linear search for applicable specific-type-queries with Map lookup, but here the overhead seems to outweigh the benefits, so I'll probably disable this code in future. One issue that I still need to address are nested types: just as Data.Generics.PlateData, Data.Generics.GPS currently fails for these (no finite representation of substructure types); current plan is to recognize nested types and to fall back to unoptimized traversals. c) generic default instance methods for Control.Monad.Functor and Data.Traversable relevant modules: Data.Generics.Utils, examples\Examples.hs status: under discusssion. could become part of that add-on package, or move into base. d) some Data/Typeable instances and utilities for GHC Api relevant modules: GHC.Syb.Instances, GHC.Syb.Instances0, GHC.Syb.Utils status: needs more testing, will probably be rendered obsolete when GHC Api starts providing those instances itself.
Maintaining SYB, alongside with the other generic libraries, will require things such as: * Releasing packages in Hackage, properly documented with Haddock; * Updating such packages as necessary for new releases of GHC; * Writing examples of how to use the libraries (from a user perspective); * Writing testsuites, which are important for checking backwards compatibility of any changes; * Having an updated webpage linking to the library sources, documentation, possibly a bug tracker, etc. These are all things we plan to do for the libraries.
Additionally, we could think of improving syb-with-class [4] in parallel with regular SYB. This is something to ask to its maintainer.
Cheers, Pedro
[1] http://books.google.com/books?id=OyY3ioMJRAsC&pg=PA199&sig=ACfU3U1nczeRAIjN9mc_vYnL1LnYAs70NA [2] http://www.cs.uu.nl/research/techreps/UU-CS-2008-019.html [3] http://www.haskell.org/pipermail/generics/2008-July/000362.html [4] http://hackage.haskell.org/cgi-bin/hackage-scripts/package/syb-with-class
--------------------------------------------------------------------------------
_______________________________________________ Generics mailing list Generics@haskell.org http://www.haskell.org/mailman/listinfo/generics
oops, keybinding malfunction - that message went out unfinished.
Seems to have most parts, though, so I leave it at that.
One other thing I meant to ask was about procedure,
given that Syb is currently in base and hence under the
library modification process. How is this going to combine
with an active maintainer and some parts on hackage?
Claus
----- Original Message -----
From: "Claus Reinke"
Hi Pedro,
and thanks for volunteering! I include a summary of where I'm at, for your information (and that of other interested readers;-)
Recently, (at least) Claus and Oleg have been posting interesting suggestions of improvements/modifications to SYB. Those should be further analyzed and discussed, and finally introduced (or not) in the library. The generic map for SYB, for instance, evolved from the "impossible to implement", through the "unsafe implementation", until the latest gmap2 as described by Oleg [3]. If further tests show this function behaves as expected, then it's clearly a good candidate for extending SYB. We should also rethink if other things previously deemed impossible remain so.
Further discussion welcome, of course!-) And it isn't just about getting fmap/traverse/.. from Data/Typeable, but also about reorganizing imports of Data instances, and providing alternatives of everywhere/everything that avoid traversals of irrelevant subterms.
A current snapshot of my code can be found here:
http://www.cs.kent.ac.uk/~cr3/tmp/syb/syb-utils-0.0.2008.7.31.tar.gz
It currently contains:
a) a suggested split for the Instances module, with alternatives to Data.Generics that either export only Standard instances, or no instances at all. On top of the existing Syb in base, these are somewhat tricky to use, because of the implicit re-exports of Data.Generics.Instances from other libraries
http://www.haskell.org/pipermail/generics/2008-July/000371.html
and the long-standing GHC "session accumulates instances" bug
http://hackage.haskell.org/trac/ghc/ticket/2182 http://www.haskell.org/ghc/docs/latest/html/users_guide/bugs.html#bugs-ghc
relevant modules: Data.Generics.Alt, Data.Generics.NoInstances, Data.Generics.Instances.Standard, Data.Generics.Instances.Dubious, examples\Examples.hs status: I think this change should go into base (perhaps renaming Data.Generics.Alt to Data.Generics.Standard, and deprecating Data.Generics and Data.Generics.Instances), for all the reasons discussed here recently, and because that GHC bug makes it near impossible to provide this change as an addon to base. Existing importers of Data.Generics or Data.Generics.Instances in core and extralibs should be redirected to one of the new modules.
b) variants of everywhere/everything/mkQ/.. that retain the type domain of generically extended queries, build substructure type maps of types to be traversed, and use a slight generalisation of the Uniplate PlateData trick to avoid traversals of irrelevant substructures (usually but not always including, and not limited to, Strings)
relevant modules: Data.Generics.GPS, examples\GPSbenchmark.hs, examples\CompanyDatatypes.hs
status: This could be provided as an addon package. Performance of generic queries and transformation on the usual Paradise benchmark improve as expected; there is some overhead, which is visible in an alternative benchmark where there are no irrelevant substructures.
The current code also tries to replace the linear search for applicable specific-type-queries with Map lookup, but here the overhead seems to outweigh the benefits, so I'll probably disable this code in future.
One issue that I still need to address are nested types: just as Data.Generics.PlateData, Data.Generics.GPS currently fails for these (no finite representation of substructure types); current plan is to recognize nested types and to fall back to unoptimized traversals. c) generic default instance methods for Control.Monad.Functor and Data.Traversable
relevant modules: Data.Generics.Utils, examples\Examples.hs
status: under discusssion. could become part of that add-on package, or move into base.
d) some Data/Typeable instances and utilities for GHC Api
relevant modules: GHC.Syb.Instances, GHC.Syb.Instances0, GHC.Syb.Utils
status: needs more testing, will probably be rendered obsolete when GHC Api starts providing those instances itself.
Maintaining SYB, alongside with the other generic libraries, will require things such as: * Releasing packages in Hackage, properly documented with Haddock; * Updating such packages as necessary for new releases of GHC; * Writing examples of how to use the libraries (from a user perspective); * Writing testsuites, which are important for checking backwards compatibility of any changes; * Having an updated webpage linking to the library sources, documentation, possibly a bug tracker, etc. These are all things we plan to do for the libraries.
Additionally, we could think of improving syb-with-class [4] in parallel with regular SYB. This is something to ask to its maintainer.
Cheers, Pedro
[1] http://books.google.com/books?id=OyY3ioMJRAsC&pg=PA199&sig=ACfU3U1nczeRAIjN9mc_vYnL1LnYAs70NA [2] http://www.cs.uu.nl/research/techreps/UU-CS-2008-019.html [3] http://www.haskell.org/pipermail/generics/2008-July/000362.html [4] http://hackage.haskell.org/cgi-bin/hackage-scripts/package/syb-with-class
--------------------------------------------------------------------------------
_______________________________________________ Generics mailing list Generics@haskell.org http://www.haskell.org/mailman/listinfo/generics
_______________________________________________ Generics mailing list Generics@haskell.org http://www.haskell.org/mailman/listinfo/generics
participants (3)
-
Claus Reinke -
José Pedro Magalhães -
Simon Peyton-Jones