Cabal Questions

Neil Mitchell

17 Aug 2006 17 Aug '06

4:34 p.m.

Hi, I want to use a preprocessor (haskell source extensions) on .xhs files in the Hoogle repo. This isn't a preprocessor that is known about by Cabal as far as I can tell, so I need to do "something". I remember that one of the main drives behind Cabal was so that additional preprocessors could be specified by the individual program. I remembered that lambdabot uses a preprocessor, but sadly that uses GHC specific preprocessor flags, and so isn't using the cabal architecture for doing this. How does Cabal do this? Any examples I can follow? Thanks Neil PS. I note that there is a new cabal-devel mailing list, but the cabal docs still suggest sending to this list for cabal questions. What should go on this list, and what should go down cabal-devel?

Show replies by date

Duncan Coutts

17 Aug 17 Aug

4:51 p.m.

On Thu, 2006-08-17 at 21:34 +0100, Neil Mitchell wrote:

...

Hi,

I want to use a preprocessor (haskell source extensions) on .xhs files in the Hoogle repo. This isn't a preprocessor that is known about by Cabal as far as I can tell, so I need to do "something". I remember that one of the main drives behind Cabal was so that additional preprocessors could be specified by the individual program.

I remembered that lambdabot uses a preprocessor, but sadly that uses GHC specific preprocessor flags, and so isn't using the cabal architecture for doing this. How does Cabal do this? Any examples I can follow?

We don't yet have a generic preprocessor support system. It is indeed part of the plan. I know I'll need it for Gtk2Hs.

...

PS. I note that there is a new cabal-devel mailing list, but the cabal docs still suggest sending to this list for cabal questions.

Yeah.

...

What should go on this list, and what should go down cabal-devel?

Well it's a bit hazy, but in particular discussion of patches and other development discussion goes on cabal-devel. So if you're interested in contributing code to cabal then do subscribe - and I think you should be! We need a windows champion to make sure we don't accidentally ignore things. Duncan

Esa Ilari Vuokko

5:05 p.m.

On 8/17/06, Duncan Coutts wrote:

...

We don't yet have a generic preprocessor support system. It is indeed part of the plan. I know I'll need it for Gtk2Hs.

What about UserHooks.hookedPreProcessors? The code seemd like it could work, but I haven't tested it. Best regards, --Esa

Duncan Coutts

5:24 p.m.

On Fri, 2006-08-18 at 00:05 +0300, Esa Ilari Vuokko wrote:

...

On 8/17/06, Duncan Coutts wrote:

...
We don't yet have a generic preprocessor support system. It is indeed part of the plan. I know I'll need it for Gtk2Hs.

What about UserHooks.hookedPreProcessors? The code seemd like it could work, but I haven't tested it.

Ok, I'm probably wrong then. Give it a go. I must admit I don't know enough about that bit. Duncan

Neil Mitchell

18 Aug 18 Aug

7:42 a.m.

Hi I have given it a go, attached is the Setup.hs file from Hoogle which I'm trying to get working. There are a few issues however: 1) Doesn't work at all, absolutely no signs of life. In fact, the trhxs function never seems to be called. 2) Even if it did work, now it just errors out when the user doesn't have the appropriate preprocessor half way through the build. Configure should really check for this (how?) and the build shouldn't just call error - it makes it impossible for tools to "figure out" what is missing and provide it. Thanks Neil On 8/17/06, Duncan Coutts wrote:

...

On Fri, 2006-08-18 at 00:05 +0300, Esa Ilari Vuokko wrote:

...
On 8/17/06, Duncan Coutts wrote:

...
We don't yet have a generic preprocessor support system. It is indeed part of the plan. I know I'll need it for Gtk2Hs.

What about UserHooks.hookedPreProcessors? The code seemd like it could work, but I haven't tested it.

Ok, I'm probably wrong then. Give it a go. I must admit I don't know enough about that bit.

Duncan

Niklas Broberg

7:58 a.m.

...

1) Doesn't work at all, absolutely no signs of life. In fact, the trhxs function never seems to be called.

By default the preprocessor is called 'trhsx', not 'trhxs', could be the reason why there's no sign of life. But then I would have thought you'd get an error. And I agree with you on point 2. /Niklas

Neil Mitchell

8:01 a.m.

...

By default the preprocessor is called 'trhsx', not 'trhxs', could be the reason why there's no sign of life. But then I would have thought you'd get an error. And I agree with you on point 2.

You are of course correct, I've fixed it, still no signs of life at all. Thanks Neil

Esa Ilari Vuokko

8:57 a.m.

Hi Neil, On 8/18/06, Neil Mitchell wrote:

...

1) Doesn't work at all, absolutely no signs of life. In fact, the trhxs function never seems to be called.

If I build a minimal project and use your Setup.hs, I get: Preprocessing library Foo-0.1... setup: trhxs not found, cannot process: Foo.xhs (But I am using freshly compiled Cabal-1.1.5 and ghc-6.5, maybe that matters)

...

2) Even if it did work, now it just errors out when the user doesn't have the appropriate preprocessor half way through the build. Configure should really check for this (how?) and the build shouldn't just call error - it makes it impossible for tools to "figure out" what is missing and provide it.

Yes. My suggestion was incomplete. I think the right way to do this is (when using Simple): * hook configure-step a bit: You should be able to simply append your program description into ConfigFlags.configPrograms and call the default hook. * that will append it into LocalBuildInfo.withPrograms * add the preprocessor to userhooks, but rather than finding executable at that point, take it from LocalBuildInfo.withPrograms I have not done this myself, so I'm not 100% sure HTH, --Esa Ilari Vuokko

Neil Mitchell

9:58 a.m.

...

Preprocessing library Foo-0.1... setup: trhxs not found, cannot process: Foo.xhs

Hmm, I built a minimal project (attached), and it didn't work. Am I doing something wrong? I'm using a freshly compiled Cabal and GHC 6.4.2

...

Yes. My suggestion was incomplete. I think the right way to do this is (when using Simple): * hook configure-step a bit: You should be able to simply append your program description into ConfigFlags.configPrograms and call the default hook. * that will append it into LocalBuildInfo.withPrograms * add the preprocessor to userhooks, but rather than finding executable at that point, take it from LocalBuildInfo.withPrograms

That seems much better, once I have this preprocessor working basically, I'll try and follow that pattern. Thanks Neil

Esa Ilari Vuokko

10:24 a.m.

Neil Mitchell wrote:

...

Hmm, I built a minimal project (attached), and it didn't work. Am I doing something wrong? I'm using a freshly compiled Cabal and GHC 6.4.2 You should add (to Hoogle.cabal)

Other-Modules: Test because unfortunately Cabal has no dependency analysis, and hence, in practice, it is better to keep all modules listed, under on of Expoted-Modules (libraries), Main-Is (executables) and Other-Modules (libs and executables). HTH, --Esa Ilari Vuokko

Neil Mitchell

11:14 a.m.

Hi,

...

because unfortunately Cabal has no dependency analysis, and hence, in practice, it is better to keep all modules listed, under on of Expoted-Modules (libraries), Main-Is (executables) and Other-Modules (libs and executables).

That's unfortunate. Is this considered as a bug that will be fixed at some point, or is this by design? Thanks Neil

Esa Ilari Vuokko

11:18 a.m.

Hi, On 8/18/06, Neil Mitchell wrote:

...

...
because unfortunately Cabal has no dependency analysis, and hence, in practice, it is better to keep all modules listed, under on of Expoted-Modules (libraries), Main-Is (executables) and Other-Modules (libs and executables).

That's unfortunate. Is this considered as a bug that will be fixed at some point, or is this by design?

It's a missing feature, or a bug, whichever you prefer. http://hackage.haskell.org/trac/hackage/ticket/15 Best regards, --Esa

Duncan Coutts

12:01 p.m.

On Fri, 2006-08-18 at 18:18 +0300, Esa Ilari Vuokko wrote:

...

Hi,

On 8/18/06, Neil Mitchell wrote:

...
...
because unfortunately Cabal has no dependency analysis, and hence, in practice, it is better to keep all modules listed, under on of Expoted-Modules (libraries), Main-Is (executables) and Other-Modules (libs and executables).

That's unfortunate. Is this considered as a bug that will be fixed at some point, or is this by design?

It's a missing feature, or a bug, whichever you prefer.

http://hackage.haskell.org/trac/hackage/ticket/15

Yeah, we will need this for proper support other preprocessors too. For example .chs files can have dependencies on each other. Currently Cabal totally ignores that. Perhaps we can pinch the .hs dep chasing code from either GHC or hmake and make it extensible for other pre-processors. But yes the license is important. Duncan

Neil Mitchell

12:20 p.m.

Hi,

...

Perhaps we can pinch the .hs dep chasing code from either GHC or hmake and make it extensible for other pre-processors. But yes the license is important.

There is always Yhc. Its also GPL licensed, but me/Tom wrote all the dependancy/make stuff, so license things can probably be solved relatively easy. In fact, I even have it on my todo list to make a Yhc.Make API, so other programs can do Haskell dependancy chasing automatically (I need it for my phd). If there was a nice simple dependancy chasing API, would Cabal want to use that? If so, what would the interface be? At the moment the design goes along the lines of: getDependancies :: HaskellFile -> IO [HaskellModule] resolveDependancy :: State -> HaskellModule -> IO HaskellFile orderDependancies :: State -> HaskellFile -> IO [HaskellFile] State is some kind of "whats the current directory", "what packages are in scope" - some global immutable set of flags and options. HaskellFile is a concrete .hs source file, i.e. System/Directory.hs HaskellModule is a Haskell module, i.e. System.Directory Any use to Cabal? If not, what tweaks would be needed? Thanks Neil

John Meacham

8:18 p.m.

On Fri, Aug 18, 2006 at 05:01:42PM +0100, Duncan Coutts wrote:

...

Yeah, we will need this for proper support other preprocessors too. For example .chs files can have dependencies on each other. Currently Cabal totally ignores that. Perhaps we can pinch the .hs dep chasing code from either GHC or hmake and make it extensible for other pre-processors. But yes the license is important.

If you wanted to take anything from jhc, I would happily dual licence it 3bsd. the code for following dependencies is in Ho.Build (search for 'scc modules' as that prints out the results). it can be fed the HsSyn output from Language.Haskell.Parser directly in the standard libs, but you probably want to prune out everything but the import list parsing to not be messed up by extensions the full parser can't handle. John -- John Meacham - ⑆repetae.net⑆john⑈

Neil Mitchell

8:22 p.m.

...

it can be fed the HsSyn output from Language.Haskell.Parser directly in the standard libs, but you probably want to prune out everything but the import list parsing to not be messed up by extensions the full parser can't handle.

I believe that for Yhc, as long as the extensions are after the last import, they won't be read. The parser is sufficiently lazy that it can get the import declarations out of a module without parsing the whole thing. This gives both a performance boost, and happily ignores extensions. Although I suspect a custom parser for import files would be best, to get maximum flexibility. Thanks Neil

John Meacham

8:32 p.m.

On Sat, Aug 19, 2006 at 01:22:57AM +0100, Neil Mitchell wrote:

...

...
it can be fed the HsSyn output from Language.Haskell.Parser directly in the standard libs, but you probably want to prune out everything but the import list parsing to not be messed up by extensions the full parser can't handle.

I believe that for Yhc, as long as the extensions are after the last import, they won't be read. The parser is sufficiently lazy that it can get the import declarations out of a module without parsing the whole thing.

Really? fully lazy parsers are notoriously tricky, which parsing library do you use? I keep wanting to switch jhc to packrat parsing http://pdos.csail.mit.edu/~baford/packrat/ actually, any monadic formulation would be better than 'happy'for me because with pre-generated parsers you can't change your parse rules on the fly easily, like depending on what extensions are enabled. John -- John Meacham - ⑆repetae.net⑆john⑈

Neil Mitchell

8:48 p.m.

...

Really? fully lazy parsers are notoriously tricky, which parsing library do you use? I have been told that its lazy in that respect, I've never actually checked. As for the parsing library, its an entirely arbitrary one :) Its not monadic, its really really old, and its in the Yhc repo under src/compiler98/Parse. I don't have a clue how it works...

...

I keep wanting to switch jhc to packrat parsing http://pdos.csail.mit.edu/~baford/packrat/ I've seen that, but I want to switch Yhc to: http://www-users.cs.york.ac.uk/~ndm/projects/parsing.php (O(n), super-duper fast, no shift/reduce nonsense, easy to specify)

I prefer pre-generated parsers - you can always parse the additional syntax and then tell the user what flag that have to enable to get it to actually work. Thanks Neil

shelarcy

17 Aug 17 Aug

7:52 p.m.

New subject: Cabal with Generating files (Was: Cabal Questions

On Fri, 18 Aug 2006 06:05:36 +0900, Esa Ilari Vuokko wrote:

...

On 8/17/06, Duncan Coutts wrote:

...
We don't yet have a generic preprocessor support system. It is indeed part of the plan. I know I'll need it for Gtk2Hs.

What about UserHooks.hookedPreProcessors? The code seemd like it could work, but I haven't tested it.

I think changing Setup.(l)hs definition with UserHook is very strong, but this approach is too complex to specify just generating file from other resources by only one tool. I think cabal must support to write using self preproccesser definition in *.cabal files like this. name: hdirect version: 0.1 (snip) othermodules: ComPrim, AutoPrim generatable: ComPrim g-sources: ComPrim.idl g-tool: ihc g-option: -fno-export-list -fappend-interface-short-name -fno-qualified-names -fno-imports -fno-export-lists -fkeep-hresult -fout-pointers-are-not-refs generatable: AutoPrim g-sources: AutoPrim.idl g-tool: ihc g-option: -fno-qualified-names -fno-imports -fno-export-lists -fkeep-hresult -fout-pointers-are-not-refs This is rough sketch. But I think this is useful for discussion about "Cabal with Generating files". -- shelarcy <shelarcy capella.freemail.ne.jp> http://page.freett.com/shelarcy/

6849

Age (days ago)

6851

Last active (days ago)

List overview

Download

18 comments

6 participants

participants (6)

Duncan Coutts
Esa Ilari Vuokko
John Meacham
Neil Mitchell
Niklas Broberg
shelarcy

Cabal Questions

tags

participants (6)