check-pvp

newer
Haddock fails with "Module defined...

Henning Thielemann

28 Feb 2014 28 Feb '14

3:04 p.m.

Following Mikhail Glushenkov's advice I have prepared a separate tool that checks PVP compliance of imports as proposed in: https://github.com/haskell/cabal/issues/1703 There remain some problems: * How can I exclude automatically generated modules like Path_*.hs from the check? * How can I access modules that are preprocessed? HSC processed modules are stored in dist/build, but CPP processed modules are not stored anywhere. * How shall I cope with condition flags? Currently I use flattenPackageDescription. Alternatively I might only check the modules that are part of the current configuration. But I may miss many modules this way. I might perform checks on all possible configurations. * Can you give me advice on what I shall write to stdout and what to stderr? The purpose of the program is to generate a list of warnings. Shall they be written to stderr because they are warnings or shall they be written to stdout because they are the intended output of the program? * Regarding levels of verbosity: How silent shall silent be? Shall the test results be emitted in silent mode or only in normal verbosity mode?

Show replies by date

Johan Tibell

28 Feb 28 Feb

3:23 p.m.

On Fri, Feb 28, 2014 at 9:04 PM, Henning Thielemann < lemming@henning-thielemann.de> wrote:

...

Following Mikhail Glushenkov's advice I have prepared a separate tool that checks PVP compliance of imports as proposed in: https://github.com/haskell/cabal/issues/1703

I added some feedback in this bug, before I saw this email.

...

* How can I exclude automatically generated modules like Path_*.hs from the check?

Somewhere in the code there's a function for getting them. Unfortunately I don't remember where. Not very helpful, I know.

...

* How can I access modules that are preprocessed? HSC processed modules are stored in dist/build, but CPP processed modules are not stored anywhere.

Perhaps because GHC applies CPP "on the fly". I guess you prefer to use haskell-src-exts, but using the GHC API might be better in the long run, as your tool will work on all packages (more or less a requirement for cabal integration.)

...

* How shall I cope with condition flags? Currently I use flattenPackageDescription. Alternatively I might only check the modules that are part of the current configuration. But I may miss many modules this way. I might perform checks on all possible configurations.

* Can you give me advice on what I shall write to stdout and what to stderr? The purpose of the program is to generate a list of warnings. Shall they be written to stderr because they are warnings or shall they be written to stdout because they are the intended output of the program?

I think it depends on the context where this is run. If it was part of cabal check, stderr. If it's a standalone tool, perhaps stdout.

...

* Regarding levels of verbosity: How silent shall silent be? Shall the test results be emitted in silent mode or only in normal verbosity mode?

By test results you mean your warnings? I think we should consider making this part of cabal check. I think we might want to only run it on the library section for now and only optionally (i.e. using a flag) check test-suites etc. The PVP really only applies to libraries. -- Johan

Roman Cheplyaka

4:06 p.m.

* Johan Tibell [2014-02-28 21:23:23+0100]

...

...
* How can I access modules that are preprocessed? HSC processed modules are stored in dist/build, but CPP processed modules are not stored anywhere.

Perhaps because GHC applies CPP "on the fly". I guess you prefer to use haskell-src-exts, but using the GHC API might be better in the long run, as your tool will work on all packages (more or less a requirement for cabal integration.)

FWIW, hse-cpp[1] adds CPP preprocessing to haskell-src-exts, although everyone should be advised against using it since its author doesn't follow PVP. [1]: http://hackage.haskell.org/package/hse-cpp Roman

Henning Thielemann

4:21 p.m.

Am 28.02.2014 22:06, schrieb Roman Cheplyaka:

...

* Johan Tibell [2014-02-28 21:23:23+0100]

...
...
* How can I access modules that are preprocessed? HSC processed modules are stored in dist/build, but CPP processed modules are not stored anywhere.

Perhaps because GHC applies CPP "on the fly". I guess you prefer to use haskell-src-exts, but using the GHC API might be better in the long run, as your tool will work on all packages (more or less a requirement for cabal integration.)

FWIW, hse-cpp[1] adds CPP preprocessing to haskell-src-exts, although everyone should be advised against using it since its author doesn't follow PVP.

I see. :-)

...

[1]: http://hackage.haskell.org/package/hse-cpp

It would still be nicer to have a solution that also works with HSC, CHS and maybe others. I have also not checked LHS so far.

Roman Cheplyaka

4:31 p.m.

* Henning Thielemann [2014-02-28 22:21:39+0100]

...

...
[1]: http://hackage.haskell.org/package/hse-cpp

It would still be nicer to have a solution that also works with HSC, CHS and maybe others. I have also not checked LHS so far.

hse-cpp takes care of .lhs, too. As for .hsc and .chs, those are handled by cabal. The idea is that hse-cpp adds the same transformations to haskell-src-exts as ghc has. And ghc doesn't know/care about .hsc or .chs. Roman

Henning Thielemann

4:48 p.m.

Am 28.02.2014 22:31, schrieb Roman Cheplyaka:

...

* Henning Thielemann [2014-02-28 22:21:39+0100]

...
...
[1]: http://hackage.haskell.org/package/hse-cpp

It would still be nicer to have a solution that also works with HSC, CHS and maybe others. I have also not checked LHS so far.

hse-cpp takes care of .lhs, too. As for .hsc and .chs, those are handled by cabal. The idea is that hse-cpp adds the same transformations to haskell-src-exts as ghc has.

Nice! Is there a recommended way to transfer CPP options from the Cabal file to the CpphsOptions record?

Roman Cheplyaka

5 p.m.

* Henning Thielemann [2014-02-28 22:48:59+0100]

...

Am 28.02.2014 22:31, schrieb Roman Cheplyaka:

...
* Henning Thielemann [2014-02-28 22:21:39+0100]

...
...
[1]: http://hackage.haskell.org/package/hse-cpp

It would still be nicer to have a solution that also works with HSC, CHS and maybe others. I have also not checked LHS so far.

hse-cpp takes care of .lhs, too. As for .hsc and .chs, those are handled by cabal. The idea is that hse-cpp adds the same transformations to haskell-src-exts as ghc has.

Nice! Is there a recommended way to transfer CPP options from the Cabal file to the CpphsOptions record?

Yes, haskell-packages[2] lets you easily create a cabal-integrated "compiler". See the compile method[3] in particular. For an example of how this all glues together, see [4]. [2]: http://hackage.haskell.org/package/haskell-packages [3]: http://hackage.haskell.org/package/haskell-packages-0.2.3.4/docs/Distributio... [4]: https://github.com/haskell-suite/haskell-names/blob/master/hs-gen-iface/src/...

Henning Thielemann

1 Mar 1 Mar

10:57 a.m.

Am 28.02.2014 23:00, schrieb Roman Cheplyaka:

...

* Henning Thielemann [2014-02-28 22:48:59+0100]

...

...
Nice! Is there a recommended way to transfer CPP options from the Cabal file to the CpphsOptions record?

Yes, haskell-packages[2] lets you easily create a cabal-integrated "compiler". See the compile method[3] in particular.

For an example of how this all glues together, see [4].

[2]: http://hackage.haskell.org/package/haskell-packages [3]: http://hackage.haskell.org/package/haskell-packages-0.2.3.4/docs/Distributio... [4]: https://github.com/haskell-suite/haskell-names/blob/master/hs-gen-iface/src/...

Treating check-pvp as compiler driven by Cabal sounds reasonable, since this would do all the preprocessing stuff and would also work on tarballs etc. However the haskell-name framework seems to expect a binary executable as compiler. According to http://documentup.com/haskell-suite/haskell-names I would have to design the checker as a tool that only reads modules, not the package description, called maybe 'check-modules-pvp' and then run $ cabal install --haskell-suite -w check-modules-pvp mypkg My problem is: The checker needs to read the package description in order to classify the dependency ranges. The compiler has no access to the package description. If it tries to load the package description manually, that will fail on tarballs. I can also not see how I define my own command line options in the Compiler interface.

Henning Thielemann

1:02 p.m.

Am 01.03.2014 16:57, schrieb Henning Thielemann:

...

Treating check-pvp as compiler driven by Cabal sounds reasonable, since this would do all the preprocessing stuff and would also work on tarballs etc. However the haskell-name framework seems to expect a binary executable as compiler. According to

http://documentup.com/haskell-suite/haskell-names

I would have to design the checker as a tool that only reads modules, not the package description, called maybe 'check-modules-pvp' and then run

$ cabal install --haskell-suite -w check-modules-pvp mypkg

I have pushed a new version to the repo that contains two executables: The stand-alone executable check-pvp and the haskell-suite compiler check-pvp-compiler: http://code.haskell.org/~thielema/check-pvp/ You can run the compiler with $ cabal install --haskell-suite -w check-pvp-compiler I got some problems. Using NamesDB as in hs-gen-iface works, but I guess, I do not need NamesDB and thus haskell-names. I tried to define a StandardDB type with a custom name type: data CheckPVPName = CheckPVPName instance IsDBName CheckPVPName where getDBName = Tagged "check-pvp" theTool :: Compiler.Simple (StandardDB CheckPVPName) However with this definition the above 'cabal install' fails in configuration phase utility$ cabal install --haskell-suite -w check-pvp-compiler Resolving dependencies... cabal: Could not resolve dependencies: trying: utility-ht-0.0.10 (user goal) next goal: base (dependency of utility-ht-0.0.10) rejecting: base-4.6.0.1, 4.6.0.0, 4.5.1.0, 4.5.0.0, 4.4.1.0, 4.4.0.0, 4.3.1.0, 4.3.0.0, 4.2.0.2, 4.2.0.1, 4.2.0.0, 4.1.0.0, 4.0.0.0 (only already installed instances can be used) rejecting: base-3.0.3.2 (conflict: base => base>=4.0 && <4.3) rejecting: base-3.0.3.1 (conflict: base => base>=4.0 && <4.2) Dependency tree exhaustively searched. I guess I do not understand the purpose of the StandardDB type and its associated name type. There is another problem: With NamesDB it worked, but the final installation phase fails, because check-pvp-compiler does not generate files. What is the recommended way to cope with compilers that do not write something to files?

Roman Cheplyaka

4:29 p.m.

* Henning Thielemann [2014-03-01 19:02:35+0100]

...

Am 01.03.2014 16:57, schrieb Henning Thielemann:

...
Treating check-pvp as compiler driven by Cabal sounds reasonable, since this would do all the preprocessing stuff and would also work on tarballs etc. However the haskell-name framework seems to expect a binary executable as compiler. According to

http://documentup.com/haskell-suite/haskell-names

I would have to design the checker as a tool that only reads modules, not the package description, called maybe 'check-modules-pvp' and then run

$ cabal install --haskell-suite -w check-modules-pvp mypkg

I have pushed a new version to the repo that contains two executables: The stand-alone executable check-pvp and the haskell-suite compiler check-pvp-compiler: http://code.haskell.org/~thielema/check-pvp/

You can run the compiler with $ cabal install --haskell-suite -w check-pvp-compiler

I got some problems. Using NamesDB as in hs-gen-iface works, but I guess, I do not need NamesDB and thus haskell-names.

Certainly not. Perhaps the compiler abstraction is not such a good fit for check-pvp. For a usual compiler, you need your dependencies to be installed. For check-pvp, you don't. And, as you note, check-pvp doesn't produce any artifacts either. I guess a new abstraction (and a new command, similar to build) should be added to haskell-packages to support such use cases. I have to think about this. (TBH I'm too concerned with what's happening to my country right now to think clearly.) If you have any suggestions, let me know. Are there any similar use cases to this one, or is it too different from anything else? In other words, is it worth it to create an abstraction for this?

...

I tried to define a StandardDB type with a custom name type:

data CheckPVPName = CheckPVPName

instance IsDBName CheckPVPName where getDBName = Tagged "check-pvp"

theTool :: Compiler.Simple (StandardDB CheckPVPName)

However with this definition the above 'cabal install' fails in configuration phase

utility$ cabal install --haskell-suite -w check-pvp-compiler Resolving dependencies... cabal: Could not resolve dependencies: trying: utility-ht-0.0.10 (user goal) next goal: base (dependency of utility-ht-0.0.10) rejecting: base-4.6.0.1, 4.6.0.0, 4.5.1.0, 4.5.0.0, 4.4.1.0, 4.4.0.0, 4.3.1.0, 4.3.0.0, 4.2.0.2, 4.2.0.1, 4.2.0.0, 4.1.0.0, 4.0.0.0 (only already installed instances can be used) rejecting: base-3.0.3.2 (conflict: base => base>=4.0 && <4.3) rejecting: base-3.0.3.1 (conflict: base => base>=4.0 && <4.2) Dependency tree exhaustively searched.

I guess I do not understand the purpose of the StandardDB type and its associated name type.

There is another problem: With NamesDB it worked, but the final installation phase fails, because check-pvp-compiler does not generate files. What is the recommended way to cope with compilers that do not write something to files?

Henning Thielemann

4:40 p.m.

Am 01.03.2014 22:29, schrieb Roman Cheplyaka:

...

* Henning Thielemann [2014-03-01 19:02:35+0100]

...
I got some problems. Using NamesDB as in hs-gen-iface works, but I guess, I do not need NamesDB and thus haskell-names.

Certainly not.

Perhaps the compiler abstraction is not such a good fit for check-pvp.

For a usual compiler, you need your dependencies to be installed. For check-pvp, you don't.

I need the packages installed, since if I find the dependency containers >=0.5 && <0.6 I need to know the modules belonging to "containers" in order to check imports of them. However, there is no need to run "check-pvp" on the imported packages - this is what cabal-install with check-pvp-compiler currently tries to do.

...

And, as you note, check-pvp doesn't produce any artifacts either.

That's true. However, if haskell-package insists on a result per module I could write a test report for each module into a corresponding file.

...

I guess a new abstraction (and a new command, similar to build) should be added to haskell-packages to support such use cases. I have to think about this. (TBH I'm too concerned with what's happening to my country right now to think clearly.)

I see.

...

If you have any suggestions, let me know. Are there any similar use cases to this one, or is it too different from anything else? In other words, is it worth it to create an abstraction for this?

So far I find the idea appealing to use the infrastructure for doing all the preprocessing. There might be more analysis tools that work the same way, but I have no concrete plans.

Roman Cheplyaka

4:11 p.m.

* Henning Thielemann [2014-03-01 16:57:56+0100]

...

Am 28.02.2014 23:00, schrieb Roman Cheplyaka:

...
* Henning Thielemann [2014-02-28 22:48:59+0100]

...
...
Nice! Is there a recommended way to transfer CPP options from the Cabal file to the CpphsOptions record?

Yes, haskell-packages[2] lets you easily create a cabal-integrated "compiler". See the compile method[3] in particular.

For an example of how this all glues together, see [4].

[2]: http://hackage.haskell.org/package/haskell-packages [3]: http://hackage.haskell.org/package/haskell-packages-0.2.3.4/docs/Distributio... [4]: https://github.com/haskell-suite/haskell-names/blob/master/hs-gen-iface/src/...

Treating check-pvp as compiler driven by Cabal sounds reasonable, since this would do all the preprocessing stuff and would also work on tarballs etc. However the haskell-name framework seems to expect a binary executable as compiler. According to

http://documentup.com/haskell-suite/haskell-names

I would have to design the checker as a tool that only reads modules, not the package description, called maybe 'check-modules-pvp' and then run

$ cabal install --haskell-suite -w check-modules-pvp mypkg

My problem is: The checker needs to read the package description in order to classify the dependency ranges. The compiler has no access to the package description.

Right, — similar to how ghc doesn't access package description either. I guess there's no reason not to add such a feature, if you're willing to write a patch (to both haskell-packages and Cabal).

...

If it tries to load the package description manually, that will fail on tarballs.

Not sure what you mean here. What tarballs?

...

I can also not see how I define my own command line options in the Compiler interface.

Good point. I've added 'customMain' to git repo. Let me know if it works for you. Roman

Henning Thielemann

4:22 p.m.

Am 01.03.2014 22:11, schrieb Roman Cheplyaka:

...

* Henning Thielemann [2014-03-01 16:57:56+0100]

...
My problem is: The checker needs to read the package description in order to classify the dependency ranges. The compiler has no access to the package description.

Right, — similar to how ghc doesn't access package description either. I guess there's no reason not to add such a feature, if you're willing to write a patch (to both haskell-packages and Cabal).

...
If it tries to load the package description manually, that will fail on tarballs.

Not sure what you mean here. What tarballs?

If foobar-0.2.tar.gz contains a tarball created by "cabal sdist", then I can install it via $ cabal install foobar-0.2.tar.gz In this case, cabal-install unpacks the archive, configures, builds and installs the package. It would be neat to run the pvp check in the same way: $ cabal install --haskellsuite -w check-pvp-compiler foobar-0.2.tar.gz However, this requires that check-pvp-compiler gets the path to the package description as parameter and does not try to load it from the current directory.

...

...
I can also not see how I define my own command line options in the Compiler interface.

Good point. I've added 'customMain' to git repo. Let me know if it works for you.

I'll try that.

Henning Thielemann

4:31 p.m.

Am 01.03.2014 22:11, schrieb Roman Cheplyaka:

...

Good point. I've added 'customMain' to git repo. Let me know if it works for you.

I can't find Distribution.HaskellSuite.Compiler.customMain. :-( I pulled from master of git://github.com/haskell-suite/haskell-packages.git

Roman Cheplyaka

5:37 p.m.

* Henning Thielemann [2014-03-01 22:31:56+0100]

...

Am 01.03.2014 22:11, schrieb Roman Cheplyaka:

...
Good point. I've added 'customMain' to git repo. Let me know if it works for you.

I can't find Distribution.HaskellSuite.Compiler.customMain. :-( I pulled from master of git://github.com/haskell-suite/haskell-packages.git

D'oh, I forgot to export it. Should be fixed now.

Henning Thielemann

5:48 p.m.

Am 01.03.2014 23:37, schrieb Roman Cheplyaka:

...

* Henning Thielemann [2014-03-01 22:31:56+0100]

...
Am 01.03.2014 22:11, schrieb Roman Cheplyaka:

...
Good point. I've added 'customMain' to git repo. Let me know if it works for you.

I can't find Distribution.HaskellSuite.Compiler.customMain. :-( I pulled from master of git://github.com/haskell-suite/haskell-packages.git

D'oh, I forgot to export it. Should be fixed now.

[6 of 8] Compiling Distribution.HaskellSuite.Compiler ( src/Distribution/HaskellSuite/Compiler.hs, dist/build/Distribution/HaskellSuite/Compiler.o ) src/Distribution/HaskellSuite/Compiler.hs:21:5: Not in scope: `customMain' I guess this is because it is missing from Cabal. Seems that I have to provide options via the optparser interface. For check-pvp I have chosen the GetOpt module for portability reasons. Can I convert GetOpt stuff to optparser?

Henning Thielemann

5:52 p.m.

Am 01.03.2014 23:48, schrieb Henning Thielemann:

...

[6 of 8] Compiling Distribution.HaskellSuite.Compiler ( src/Distribution/HaskellSuite/Compiler.hs, dist/build/Distribution/HaskellSuite/Compiler.o )

src/Distribution/HaskellSuite/Compiler.hs:21:5: Not in scope: `customMain'

...

I guess this is because it is missing from Cabal.

haskell-packages$ git diff --unified src/Distribution/HaskellSuite/Cabal.hs-boot diff --git a/src/Distribution/HaskellSuite/Cabal.hs-boot b/src/Distribution/HaskellSuite/Cabal.hs-boot index bc9bdfb..b45cbe5 100644 --- a/src/Distribution/HaskellSuite/Cabal.hs-boot +++ b/src/Distribution/HaskellSuite/Cabal.hs-boot @@ -1,7 +1,13 @@ module Distribution.HaskellSuite.Cabal where import {-# SOURCE #-} qualified Distribution.HaskellSuite.Compiler as Compiler +import Options.Applicative main :: Compiler.Is c => c -> IO () + +customMain + :: Compiler.Is c + => Parser (IO ()) + -> c -> IO ()

Roman Cheplyaka

6:04 p.m.

* Henning Thielemann [2014-03-01 23:48:11+0100]

...

Am 01.03.2014 23:37, schrieb Roman Cheplyaka:

...
* Henning Thielemann [2014-03-01 22:31:56+0100]

...
Am 01.03.2014 22:11, schrieb Roman Cheplyaka:

...
Good point. I've added 'customMain' to git repo. Let me know if it works for you.

I can't find Distribution.HaskellSuite.Compiler.customMain. :-( I pulled from master of git://github.com/haskell-suite/haskell-packages.git

D'oh, I forgot to export it. Should be fixed now.

[6 of 8] Compiling Distribution.HaskellSuite.Compiler ( src/Distribution/HaskellSuite/Compiler.hs, dist/build/Distribution/HaskellSuite/Compiler.o )

src/Distribution/HaskellSuite/Compiler.hs:21:5: Not in scope: `customMain'

I guess this is because it is missing from Cabal.

I applied your patch, thanks.

...

Seems that I have to provide options via the optparser interface. For check-pvp I have chosen the GetOpt module for portability reasons. Can I convert GetOpt stuff to optparser?

I guess you could trick optparse-applicative into giving you the full argument list (see the argument' parser), and then parse that using GetOpt. If you ask Paolo maybe he'll show a more direct way to get hold of the argument list. Roman

4099

Age (days ago)

4100

Last active (days ago)

List overview

Download

17 comments

3 participants

participants (3)

Henning Thielemann
Johan Tibell
Roman Cheplyaka

check-pvp

Johan Tibell

Roman Cheplyaka

Roman Cheplyaka

Roman Cheplyaka

Roman Cheplyaka

Roman Cheplyaka

Roman Cheplyaka

Roman Cheplyaka

tags

participants (3)