ANNOUNCE: module-management-0.9.3 - clean import lists, split and merge modules

I am pleased to announce the first release of module-management, a package for cleaning import lists, and splitting and merging modules. You can see a description at the top of the documentation for Language.Haskell.Modules (once it appears) here: http://hackage.haskell.org/package/module-management It is worth noting that the split and merge operations cause global changes to the module tree, all the modules that import the split or merged modules get their import lists adjusted. For this reason, before these operations can run the caller needs to specify the "moduVerse", the set of modules to scan for import changes. A number of bugs in GHC's -ddump-minimal-imports feature cropped up while I was writing this, and when the fixes for these issues make it into the compiler things will work a bit more smoothly, particularly if you are using NoImplicitPrelude or type families. http://hackage.haskell.org/trac/ghc/ticket/7963 http://hackage.haskell.org/trac/ghc/ticket/7969 http://hackage.haskell.org/trac/ghc/ticket/8000 http://hackage.haskell.org/trac/ghc/ticket/8011 There are workarounds for 7963 and 7969. Not surprisingly, template haskell can cause problems, but frequently it does not. CPP directives do not work. I hope you find this useful - all feedback is welcome! -david

Dear David, would you mind adding a short intro about how to use your library? I mean editor plugin authors may want to know whether or how to integrate the features of your library ?
From looking at the cabal file I see there is a library and a commented tests (by the way you can make tests optional by using cabal flags).
(before library/ executables): flag build_test description: build the test executable default: Fales To the executable add: if !flag(build_test) buildable: False Marc Weber

I put an intro into the top module - hackage will generate it in a
little while, but until then you can look here:
http://doc.seereason.com/libghc-module-management-doc/html/Language-Haskell-...
I commented out the test section because the test cases use the debian
module, and I didn't want people to have to mess with that - I will
use cabal flags to control that, or better still replace those cases
with something more self contained.
On Thu, Jun 27, 2013 at 5:26 PM, Marc Weber
Dear David,
would you mind adding a short intro about how to use your library?
I mean editor plugin authors may want to know whether or how to integrate the features of your library ?
From looking at the cabal file I see there is a library and a commented tests (by the way you can make tests optional by using cabal flags).
(before library/ executables):
flag build_test description: build the test executable default: Fales
To the executable add:
if !flag(build_test) buildable: False
Marc Weber
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

let me give you an example: splitModule :: MonadClean m => ModuleName -> m () Split each of a module's declarations into a new module. Update the imports of all the modules in the moduVerse to reflect the split. Why do I want that? What does it mean? === start file == module Start where a = 1 b = 2 c = 3 will calling that function yield module foobar where a = 1 module baz.something.idont.know where b =2 module 3rd where c =3 I want to say: I have no idea - how the new modules will be named - which paths the new modules will have Function's return type is m (), so doesn't help much. Consider improving that documentation. More questions: input/output. findPaths "Language" >>= runMonadClean . mapM cleanImports . toList Whose task is it to write the "new cleaned up module" to disk again? does runMonadClean do it for me? Docs don't tell yet. Marc Weber

Thanks, great feedback, clearly I've been too close to this to see
what people need to know. Let me give some answers, and they I will
integrate them into the documentation.
On Thu, Jun 27, 2013 at 6:06 PM, Marc Weber
let me give you an example:
splitModule :: MonadClean m => ModuleName -> m ()
Split each of a module's declarations into a new module. Update the imports of all the modules in the moduVerse to reflect the split.
Why do I want that? What does it mean?
Split and merge were the simplest set of operations that would allow you build more complex operations like move a symbol from one module to another. Directly implementing more complex for reasons that I could discuss later. It may be desirable to provide compound operations like move symbol implemented in terms of split and merge.
=== start file == module Start where
a = 1 b = 2 c = 3
will calling that function yield
module foobar where a = 1
module baz.something.idont.know where b =2
module 3rd where c =3
I want to say: I have no idea - how the new modules will be named - which paths the new modules will have
Yes, you will get three modules each containing the declarations of a single symbol. The modules are placed in a sub-directory of the original module, and the names are constructed from the symbol names. So you will get modules Start.A, Start.B and Start.C. If there are non-alphanumeric characters in the symbol they are removed, and the first character is capitalized. If more than one symbol maps to the same module name a single module is created containing both. If no module name can be constructed the symbol goes into Start.OtherSymbols. Instances go into a module named Start.Instances. Symbols that were just imported and re-exported from Start go into Start.ReExported. Symbols that were not exported from Start go into the subdirectory Internal, so if 'a' was not exported it would go into Start.Internal.A.
Function's return type is m (), so doesn't help much.
Consider improving that documentation.
More questions: input/output.
findPaths "Language" >>= runMonadClean . mapM cleanImports . toList
Whose task is it to write the "new cleaned up module" to disk again? does runMonadClean do it for me? Docs don't tell yet.
Marc Weber
The cleanImports function is the one that computes the new file contents and does the IO. The runMonadClean function sets up the environment for cleanImports to run, if you wanted to chain a series of clean/split/merge operations together you would do it in a single runMonadClean operation so that the set of modules that need to be checked and updated can be tracked. runCleanT would probably be a better name.

Excerpts from David Fox's message of Fri Jun 28 04:04:59 +0200 2013:
So you will get modules Start.A, Start.B and Start.C. If there are
But that's very unlikly what the programmer wants. I mean I might want Types and Funs as module names, move A,B to Types, C to Funs. I agree that I could reach my goal using a "merge" afterwards ? Because some modules may have many symbols having that many files created feels strange. Useful for the programmer (me?) would be: up to line 200 move to A, from 200 till end move to module B Even if you think the way to go is creating 50 files (because you happen to have 50 symbols) you may want to consider telling the user that he knows what to expect - and that you welcome patches to make this even nicer and more useful. Marc Weber

On Thu, Jun 27, 2013 at 11:18 PM, Marc Weber
Excerpts from David Fox's message of Fri Jun 28 04:04:59 +0200 2013:
So you will get modules Start.A, Start.B and Start.C. If there are
But that's very unlikly what the programmer wants. I mean I might want Types and Funs as module names, move A,B to Types, C to Funs.
I agree that I could reach my goal using a "merge" afterwards ?
From what I'm reading, I don't actually agree that the goal may be reached by using a merge afterwards. I assume that split-then-merge isn't the same as identity since at very least the order of the symbols is lost.
-- Felipe.

Since you pass a list of modules to merge, you can (must) specify the
order that the symbols will appear in the new module. So it is almost
an identity operation, unless the symbols went into the "OtherSymbols"
module.
On Thu, Jun 27, 2013 at 7:24 PM, Felipe Almeida Lessa
On Thu, Jun 27, 2013 at 11:18 PM, Marc Weber
wrote: Excerpts from David Fox's message of Fri Jun 28 04:04:59 +0200 2013:
So you will get modules Start.A, Start.B and Start.C. If there are
But that's very unlikly what the programmer wants. I mean I might want Types and Funs as module names, move A,B to Types, C to Funs.
I agree that I could reach my goal using a "merge" afterwards ?
From what I'm reading, I don't actually agree that the goal may be reached by using a merge afterwards. I assume that split-then-merge isn't the same as identity since at very least the order of the symbols is lost.
-- Felipe.

On Thu, Jun 27, 2013 at 7:18 PM, Marc Weber
Excerpts from David Fox's message of Fri Jun 28 04:04:59 +0200 2013:
So you will get modules Start.A, Start.B and Start.C. If there are
But that's very unlikly what the programmer wants. I mean I might want Types and Funs as module names, move A,B to Types, C to Funs.
I agree that I could reach my goal using a "merge" afterwards ?
Because some modules may have many symbols having that many files created feels strange.
Useful for the programmer (me?) would be:
up to line 200 move to A, from 200 till end move to module B
Even if you think the way to go is creating 50 files (because you happen to have 50 symbols) you may want to consider telling the user that he knows what to expect - and that you welcome patches to make this even nicer and more useful.
Marc Weber
Yes, these are issues that should be addressed by layers on top of what I have so far. I could even see this being integrated into an IDE. The split-in-two operation sounds very useful. As does mega-split warning. Moving a single symbol, or a list of symbols, sounds like another great option. I realized that split was not going to be the final solution, but it had a very simple signature and led to the implementation of all the mechanisms these nicer operations will require.

There are several modes of operations that are controlled by settings
in Language.Haskell.Modules.Params:
modifyModuVerse - controls the initial set of modules whose imports
will be updated as modules are split and merged. This set is updated
as splits and merges are performed.
modifyRemoveEmptyImports - If this flag is set, imports that become
empty are removed. Sometimes this will lead to errors, as the
instances in the removed import will no longer be available. In that
case this flag should be set. Note that an import that is already
empty will never be removed, on the assumption that was placed there
to import instances.
modifyExtensions - This controls the language extensions that are used
by module parser, and by GHC when it runs -ddump-minimal-imports.
This is usually not necessary, because the module usually contains
pragmas saying what extensions it needs.
modifyHsFlags - lets you pass additional flags to GHC
modifySourceDirs - lets you specify a list of directories to search
for modules. This is similar to the Hs-Source-Dirs cabal field.
On Thu, Jun 27, 2013 at 7:04 PM, David Fox
Thanks, great feedback, clearly I've been too close to this to see what people need to know. Let me give some answers, and they I will integrate them into the documentation.
On Thu, Jun 27, 2013 at 6:06 PM, Marc Weber
wrote: let me give you an example:
splitModule :: MonadClean m => ModuleName -> m ()
Split each of a module's declarations into a new module. Update the imports of all the modules in the moduVerse to reflect the split.
Why do I want that? What does it mean?
Split and merge were the simplest set of operations that would allow you build more complex operations like move a symbol from one module to another. Directly implementing more complex for reasons that I could discuss later. It may be desirable to provide compound operations like move symbol implemented in terms of split and merge.
=== start file == module Start where
a = 1 b = 2 c = 3
will calling that function yield
module foobar where a = 1
module baz.something.idont.know where b =2
module 3rd where c =3
I want to say: I have no idea - how the new modules will be named - which paths the new modules will have
Yes, you will get three modules each containing the declarations of a single symbol. The modules are placed in a sub-directory of the original module, and the names are constructed from the symbol names. So you will get modules Start.A, Start.B and Start.C. If there are non-alphanumeric characters in the symbol they are removed, and the first character is capitalized. If more than one symbol maps to the same module name a single module is created containing both. If no module name can be constructed the symbol goes into Start.OtherSymbols.
Instances go into a module named Start.Instances. Symbols that were just imported and re-exported from Start go into Start.ReExported. Symbols that were not exported from Start go into the subdirectory Internal, so if 'a' was not exported it would go into Start.Internal.A.
Function's return type is m (), so doesn't help much.
Consider improving that documentation.
More questions: input/output.
findPaths "Language" >>= runMonadClean . mapM cleanImports . toList
Whose task is it to write the "new cleaned up module" to disk again? does runMonadClean do it for me? Docs don't tell yet.
Marc Weber
The cleanImports function is the one that computes the new file contents and does the IO. The runMonadClean function sets up the environment for cleanImports to run, if you wanted to chain a series of clean/split/merge operations together you would do it in a single runMonadClean operation so that the set of modules that need to be checked and updated can be tracked.
runCleanT would probably be a better name.

David Fox
writes:
I am pleased to announce the first release of module-management, a package for cleaning import lists, and splitting and merging modules. You can see a description at the top of the documentation for Language.Haskell.Modules (once it appears) here:
How about building an executable along with the library called "cleanImports", so that I can use it from the command-line. Otherwise, every who wants to use your library in this way will be writing pretty much the exact same code. cleanImports is something I've been wanting, just hadn't gotten around to writing it yet. Thanks! -- John Wiegley FP Complete Haskell tools, training and consulting http://fpcomplete.com johnw on #haskell/irc.freenode.net

On Thu, Jun 27, 2013 at 8:46 PM, John Wiegley
David Fox
writes: I am pleased to announce the first release of module-management, a package for cleaning import lists, and splitting and merging modules. You can see a description at the top of the documentation for Language.Haskell.Modules (once it appears) here:
How about building an executable along with the library called "cleanImports", so that I can use it from the command-line. Otherwise, every who wants to use your library in this way will be writing pretty much the exact same code.
cleanImports is something I've been wanting, just hadn't gotten around to writing it yet. Thanks!
Cliff Beshers wrote a CLI for this, I will add it as a cabal executable in the next version.

David Fox
writes:
Cliff Beshers wrote a CLI for this, I will add it as a cabal executable in the next version.
Oh, also, I was unable to build the library using GHC 7.4.2. It looks like it still depends on the old Exception stuff that used to be Prelude? -- John Wiegley FP Complete Haskell tools, training and consulting http://fpcomplete.com johnw on #haskell/irc.freenode.net

Excerpts from John Wiegley's message of Fri Jun 28 05:46:31 +0200 2013:
How about building an executable along with the library called "cleanImports", Does it require knowledge about the libraries to be used? If so eventually it should be a cabal option? Or it should be able to load dependencies from a cabal configure file (.dist/something).
Then there should also be an env var CABAL_CURRENT_DIST_DIR or such, because sometimes you want to work on multiple dist directories. cabal could honor this itself. Then you could export CABAL_CURRENT_DIST_DIR=dist-foo ./Setup configure && ./Setup build cleanImports FOO And everything would work without --dist-dir=dist-foo parameter ? Marc Weber

I've just uploaded version 0.10, which corrects some formatting bugs and incorporates most of the changes suggested in this thread. Please give it a try!

On Sat, Jun 29, 2013 at 12:09 PM, David Fox
I've just uploaded version 0.10, which corrects some formatting bugs and incorporates most of the changes suggested in this thread. Please give it a try!
Version 0.10.1 is now available - it should build with GHC 7.4.1, fixes more formatting bugs, includes the unit test input data and debian/changelog. The next release will address people's concerns about the signature of the splitModule function. It will take a function defining how to map symbols to module names, so people can define their own splits. It will also return a set of values describing what it did. Should I keep posting updates in this thread?

Ok, version 0.11.1 is probably my last upload for a while unless I get some specific requests, as I need to get back to real work. It adds the new splitModule function that lets you specify a function defining which symbols go to which modules, with the old function replaced by a call to splitModules defaultSymbolToModule.

I am pleased to announce that, after a terrible struggle, version 0.13
of the module-management package is now available on hackage. The
performance has been upped from dismal to adequate (about a 30-fold
improvement) and many bugs have been fixed. The most important for
import cleaning is that it no longer gets confused by tabs. The split
and merge operations are vastly more reliable, though it is likely
that bugs still lurk. Also, the API has been simplified a little. and
the examples in Language.Haskell.Modules have been updated. The
documentation should appear soon at
http://hackage.haskell.org/package/module-management.
On Tue, Jul 2, 2013 at 1:36 PM, David Fox
Ok, version 0.11.1 is probably my last upload for a while unless I get some specific requests, as I need to get back to real work. It adds the new splitModule function that lets you specify a function defining which symbols go to which modules, with the old function replaced by a call to splitModules defaultSymbolToModule.
participants (5)
-
David Fox
-
Felipe Almeida Lessa
-
John Wiegley
-
Marc Weber
-
Oliver Charles