Adding another data file for dependency "hints"

Hello, Since pacman's dependency specification is much less powerful than in Cabal, cabal2arch may output wrong version constraints while trying to approximate the true specified dependencies. I prefer this to dropping all version requirements when they cannot be expressed in the pacman's way. In order to get improved PKGBUILDs I thought we could have a better usage of the dependency resolution capabilities in the Cabal library, this is summed up in this commit http://github.com/remyoudompheng/archhaskell/commit/2e614b2 This adds an extra file, platform-provides.txt, that lists our preferred versions for various packages. For the moment I have filled it with the contents of Haskell Platform. Regards, -- Rémy.

On Tue, Dec 28, 2010 at 09:08, Rémy Oudompheng
Hello,
Since pacman's dependency specification is much less powerful than in Cabal, cabal2arch may output wrong version constraints while trying to approximate the true specified dependencies. I prefer this to dropping all version requirements when they cannot be expressed in the pacman's way.
This sounds reasonable. However, do you have examples of dependencies that CABAL can express but pacman can't? (I was under the impression that the difference was that CABAL allowed more terse expressions, not that it allowed more.)
In order to get improved PKGBUILDs I thought we could have a better usage of the dependency resolution capabilities in the Cabal library, this is summed up in this commit
http://github.com/remyoudompheng/archhaskell/commit/2e614b2
This adds an extra file, platform-provides.txt, that lists our preferred versions for various packages. For the moment I have filled it with the contents of Haskell Platform.
Is it worth considering downloading this file dynamically, rather than compiling it into the executable statically? /M -- Magnus Therning OpenPGP: 0xAB4DFBA4 email: magnus@therning.org jabber: magnus@therning.org twitter: magthe http://therning.org/magnus

On 2010/12/30 Magnus Therning
On Tue, Dec 28, 2010 at 09:08, Rémy Oudompheng
wrote: Hello,
Since pacman's dependency specification is much less powerful than in Cabal, cabal2arch may output wrong version constraints while trying to approximate the true specified dependencies. I prefer this to dropping all version requirements when they cannot be expressed in the pacman's way.
This sounds reasonable. However, do you have examples of dependencies that CABAL can express but pacman can't? (I was under the impression that the difference was that CABAL allowed more terse expressions, not that it allowed more.)
See the issue reported by https://github.com/archhaskell/cabal2arch/issues#issue/19. PKGBUILDs can only specify dependency requirements with one inequality (<, >, <= or >=) whereas Cabal can specify any Boolean combination of inequalities.
In order to get improved PKGBUILDs I thought we could have a better usage of the dependency resolution capabilities in the Cabal library, this is summed up in this commit
http://github.com/remyoudompheng/archhaskell/commit/2e614b2
This adds an extra file, platform-provides.txt, that lists our preferred versions for various packages. For the moment I have filled it with the contents of Haskell Platform.
Is it worth considering downloading this file dynamically, rather than compiling it into the executable statically?
It's been some time since these lists were not compiled in the executable. I have added a flag to cabal2arch (in my copy http://github.com/remyoudompheng/cabal2arch) to handle a custom directory or URL where these files could be stored. I am using mtl for error handling. Regards, -- Rémy.

Rémy Oudompheng wrote:
This sounds reasonable. However, do you have examples of dependencies that CABAL can express but pacman can't? (I was under the impression that the difference was that CABAL allowed more terse expressions, not that it allowed more.)
See the issue reported by https://github.com/archhaskell/cabal2arch/issues#issue/19. PKGBUILDs can only specify dependency requirements with one inequality (<, >, <= or >=) whereas Cabal can specify any Boolean combination of inequalities.
Have you seen this? http://mailman.archlinux.org/pipermail/pacman-dev/2010-December/012065.html It would enable "parsec >= 3.0 || == 2.1" to be translated to "haskell-parsec>=3.0 || haskell-parsec=2.1". It obviously won't be included any time soon, but it would provide a long-term solution. Regards, Xyne

Rémy Oudompheng writes:
the gist of this issue is that "parsec >= 3.0 || == 2.1.*" is translated to "parsec>=3.0", which is just plain wrong. Now, wouldn't it be easiest to modify cabal2arch so that it translates that specification to "parsec>=2.1"? A straight-forward algorithm to accomplish that would be to use the *lowest* version bound in these kinds of alternatives. Right now, cabal2arch appears to be using the *first* version bound, which is not exactly optimal. Am I missing something? Take care, Peter

On Sun, Jan 2, 2011 at 11:39, Peter Simons
Rémy Oudompheng writes:
> https://github.com/archhaskell/cabal2arch/issues#issue/19.
the gist of this issue is that "parsec >= 3.0 || == 2.1.*" is translated to "parsec>=3.0", which is just plain wrong. Now, wouldn't it be easiest to modify cabal2arch so that it translates that specification to "parsec>=2.1"? A straight-forward algorithm to accomplish that would be to use the *lowest* version bound in these kinds of alternatives. Right now, cabal2arch appears to be using the *first* version bound, which is not exactly optimal.
Am I missing something?
Well, there's an obvious problem with what you suggest, since "parsec>=2.1" is plain wrong too! How common is this type of dependencies in practice? Personally I wouldn't write a dependency like that, instead I'd use a flag, but trying to enforcing that behaviour would be plain silly :) /M -- Magnus Therning OpenPGP: 0xAB4DFBA4 email: magnus@therning.org jabber: magnus@therning.org twitter: magthe http://therning.org/magnus

Hi Magnus,
the gist of this issue is that "parsec >= 3.0 || == 2.1.*" is translated to "parsec>=3.0", which is just plain wrong. Now, wouldn't it be easiest to modify cabal2arch so that it translates that specification to "parsec>=2.1"? A straight-forward algorithm to accomplish that would be to use the *lowest* version bound in these kinds of alternatives. Right now, cabal2arch appears to be using the *first* version bound, which is not exactly optimal.
Well, there's an obvious problem with what you suggest, since "parsec>=2.1" is plain wrong too!
why is that? Take care, Peter

On Sun, Jan 2, 2011 at 12:20, Peter Simons
Hi Magnus,
>> the gist of this issue is that "parsec >= 3.0 || == 2.1.*" is translated to >> "parsec>=3.0", which is just plain wrong. Now, wouldn't it be easiest to >> modify cabal2arch so that it translates that specification to "parsec>=2.1"? >> A straight-forward algorithm to accomplish that would be to use the *lowest* >> version bound in these kinds of alternatives. Right now, cabal2arch appears >> to be using the *first* version bound, which is not exactly optimal. > > Well, there's an obvious problem with what you suggest, since > "parsec>=2.1" is plain wrong too!
why is that?
Now you make me think that I might have missed something, but surely parsec version 2.2 wouldn't satisfy the former, but would satisfy the latter. /M -- Magnus Therning OpenPGP: 0xAB4DFBA4 email: magnus@therning.org jabber: magnus@therning.org twitter: magthe http://therning.org/magnus

Hi Magnus,
Now you make me think that I might have missed something, but surely parsec version 2.2 wouldn't satisfy the former, but would satisfy the latter.
yes, this is true, of course. It is impossible to map that Cabal expression to Pacman without losing some information. In this particular case, that restriction is lost. If a version 2.2 of parsec were to come out, and if we were to decide to ship that version, then we'd have to update the email-validate PKGBUILD in order to fix the build. Now, the alternative is to encode the Cabal expression in question as ('parsec>=2.1.0' 'parsec<2.2'). That choice means that email-validate won't build anymore if we decide to upgrade to parsec 3. In a way, both solutions are equally inaccurate, so we can choose either one. I just happen to prefer the first solution because I feel that an upgrade to parsec 3 is far more likely to occur than an upgrade to 2.2, simply because parsec 3 exists, but parsec 2.2 does not. Does that make sense? Take care, Peter

On 03/01/11 19:49, Peter Simons wrote:
Hi Magnus,
Now you make me think that I might have missed something, but surely parsec version 2.2 wouldn't satisfy the former, but would satisfy the latter.
yes, this is true, of course.
It is impossible to map that Cabal expression to Pacman without losing some information. In this particular case, that restriction is lost. If a version 2.2 of parsec were to come out, and if we were to decide to ship that version, then we'd have to update the email-validate PKGBUILD in order to fix the build.
Now, the alternative is to encode the Cabal expression in question as ('parsec>=2.1.0' 'parsec<2.2'). That choice means that email-validate won't build anymore if we decide to upgrade to parsec 3.
In a way, both solutions are equally inaccurate, so we can choose either one. I just happen to prefer the first solution because I feel that an upgrade to parsec 3 is far more likely to occur than an upgrade to 2.2, simply because parsec 3 exists, but parsec 2.2 does not.
Does that make sense?
In this particular example I have a feeling that it's parsec that is the problem ;-) Staying with parsec though, would it make sense to take HP in consideration when converting the dependencies? I.e, when the CABAL dependency says "parsec >= 3.0 || == 2.1.*" then we choose "parsec>=2.1.0, parsec<2.2.0" simply because that's the version in HP. /M -- Magnus Therning OpenPGP: 0xAB4DFBA4 email: magnus@therning.org jabber: magnus@therning.org twitter: magthe http://therning.org/magnus

On 2011/1/3 Magnus Therning
On Sun, Jan 2, 2011 at 12:20, Peter Simons
wrote: Hi Magnus,
>> the gist of this issue is that "parsec >= 3.0 || == 2.1.*" is translated to >> "parsec>=3.0", which is just plain wrong. Now, wouldn't it be easiest to >> modify cabal2arch so that it translates that specification to "parsec>=2.1"? >> A straight-forward algorithm to accomplish that would be to use the *lowest* >> version bound in these kinds of alternatives. Right now, cabal2arch appears >> to be using the *first* version bound, which is not exactly optimal. > > Well, there's an obvious problem with what you suggest, since > "parsec>=2.1" is plain wrong too!
why is that?
Now you make me think that I might have missed something, but surely parsec version 2.2 wouldn't satisfy the former, but would satisfy the latter.
I think we have reasons to translate that to 'parsec>=2.1' or 'parsec<2.2' (I would personally use even parsec=2.1.0.1) * we won't upgrade to parsec 2.2 unless that is included in some Haskell Platform * we can expect Parsec 2.2 to be mostly API-compatible with 2.1 if it comes to existence (but the author of the package didn't want to make this assumption) * if HP includes Parsec 3, PKGBUILDs will nevertheless be regenerated to include this new information. -- Rémy.

On 2011/1/2 Peter Simons
Rémy Oudompheng writes:
> https://github.com/archhaskell/cabal2arch/issues#issue/19.
the gist of this issue is that "parsec >= 3.0 || == 2.1.*" is translated to "parsec>=3.0", which is just plain wrong. Now, wouldn't it be easiest to modify cabal2arch so that it translates that specification to "parsec>=2.1"? A straight-forward algorithm to accomplish that would be to use the *lowest* version bound in these kinds of alternatives. Right now, cabal2arch appears to be using the *first* version bound, which is not exactly optimal.
Am I missing something?
Dependencies are expressed as a union of intervals : here [2.1, 2.2) union [3.0, infinity) (square brackets mean inclusive, parentheses mean exclusive bounds). * how should multiple intervals be treated ? I currently use the last one, Peter suggests to use the first one, maybe we should use the convex hull of possible dependencies ? * when the dependency spec is reduced to a single interval [a,b] should we output >=a or <=b ? if the interval is unbounded then there is no problem. I can suggest the attached patch, that would lead to the output you want. -- Rémy.

On 2011-01-02 12:18 +0100 (00:7) Rémy Oudompheng wrote:
On 2011/1/2 Peter Simons
wrote: Rémy Oudompheng writes:
> https://github.com/archhaskell/cabal2arch/issues#issue/19.
the gist of this issue is that "parsec >= 3.0 || == 2.1.*" is translated to "parsec>=3.0", which is just plain wrong. Now, wouldn't it be easiest to modify cabal2arch so that it translates that specification to "parsec>=2.1"? A straight-forward algorithm to accomplish that would be to use the *lowest* version bound in these kinds of alternatives. Right now, cabal2arch appears to be using the *first* version bound, which is not exactly optimal.
Am I missing something?
Dependencies are expressed as a union of intervals : here [2.1, 2.2) union [3.0, infinity) (square brackets mean inclusive, parentheses mean exclusive bounds).
* how should multiple intervals be treated ? I currently use the last one, Peter suggests to use the first one, maybe we should use the convex hull of possible dependencies ? * when the dependency spec is reduced to a single interval [a,b] should we output >=a or <=b ? if the interval is unbounded then there is no problem.
The wording of your reply implies that you are unaware that you can specify single intervals. Perhaps that was unintentional, but in case it wasn't I give the following example: depends=('foo>=a' 'foo<=b'). You can also use ">" and "<", so a single interval, bounded or unbounded, is not a problem. For multiple versions it makes sense to use the one that is satisfiable by the current set of packages. If that doesn't work, can you give an example of when a package actually needs multiple intervals?

On 2011/1/2 Xyne
On 2011-01-02 12:18 +0100 (00:7) Rémy Oudompheng wrote:
Dependencies are expressed as a union of intervals : here [2.1, 2.2) union [3.0, infinity) (square brackets mean inclusive, parentheses mean exclusive bounds).
* how should multiple intervals be treated ? I currently use the last one, Peter suggests to use the first one, maybe we should use the convex hull of possible dependencies ? * when the dependency spec is reduced to a single interval [a,b] should we output >=a or <=b ? if the interval is unbounded then there is no problem.
The wording of your reply implies that you are unaware that you can specify single intervals. Perhaps that was unintentional, but in case it wasn't I give the following example: depends=('foo>=a' 'foo<=b'). You can also use ">" and "<", so a single interval, bounded or unbounded, is not a problem. For multiple versions it makes sense to use the one that is satisfiable by the current set of packages.
I suggested to use this trick some time ago, and I don't remember getting any answer, so I thought it was either undesirable, ugly or not working. My point of view on this is that we should select a preferred version depending on what is in the repository, and this is the point of the new data file. -- Rémy.

Hi Xyne,
depends=('foo>=a' 'foo<=b').
thank you very much for the clarification. Now, the only issue left is how to translate a specification like "parsec >= 3.0 || == 2.1.*". Since you know Pacman very well, I wonder whether you'd have a suggestion? Take care, Peter

Peter Simons wrote:
Hi Xyne,
depends=('foo>=a' 'foo<=b').
thank you very much for the clarification. Now, the only issue left is how to translate a specification like "parsec >= 3.0 || == 2.1.*". Since you know Pacman very well, I wonder whether you'd have a suggestion?
Take care, Peter
There should only be one version of parsec available at any given time. If the 2.1.* series is still in the repos then use ('parsec>=2.1.0' 'parsec<2.2'). If we've moved on to the 3.0 series then use ('parsec>=3.0'). Maybe I'm being overly simplistic here, but I don't see the problem. When would we need to have multiple versions of parsec in the repos? The real issue seems to be one of feeding this information to cabal2arch. It needs to know at the time of generating the PKGBUILD which subset of packages to expect. This should be provided in an external file that would allow use to maintain a list of the current repo package versions (with flexible syntax that allows the specification of series, e.g. foo>=3.0 to indicate that we're now using foo 3.0 or above in the repos). This would also allow users to customize this list for their own needs. I'm sorry that I don't have the time to provide actual code right now, but I would try to create a graph that walks the cabal files and detects dependency intervals etc, then try to create a tool to let the user trace a path through the graph to generate the list of versions. Basically, the user would be able to select target nodes (i.e. specific versions of packages), and then a path could be traced to satisfy all of those dependencies.

On 2011/1/3 Xyne
The real issue seems to be one of feeding this information to cabal2arch. It needs to know at the time of generating the PKGBUILD which subset of packages to expect. This should be provided in an external file that would allow use to maintain a list of the current repo package versions (with flexible syntax that allows the specification of series, e.g. foo>=3.0 to indicate that we're now using foo 3.0 or above in the repos). This would also allow users to customize this list for their own needs.
Do you read my messages? My original message to the list stated I have done exactly this. -- Rémy.

Rémy Oudompheng wrote:
Do you read my messages? My original message to the list stated I have done exactly this.
-- Rémy.
I replied to this but didn't notice that it was directly sent to me, so my reply only went back to Rémy. I'm reposting it below to keep it on the list: Xyne wrote:
Sorry, I got lost in the discussion about intervals and forgot about the original message. From the other posts it seemed that this issue was still unresolved.
Look at the bright side: my posts argue for your approach ;)
Thanks for your work and sorry for the confusion.
p.s. Please send messages directly to the list and CC me anything that requires my direct attention if you're worried that I might miss it.

Hi Xyne,
If the 2.1.* series is still in the repos then use ('parsec>=2.1.0' 'parsec<2.2').
I don't understand why you propose setting that upper bound. Won't this restriction mean that the PKGBUILD file will have to be re-generated when we update to parsec 3? Why do you think that behavior would be desirable?
Maybe I'm being overly simplistic here, but I don't see the problem.
Well, I'm not sure why this issue appears to be controversial either. Personally, I suggested to translate that dependency to "parsec>=2.1.0", but no-one seems to agree with me, so I'm trying to figure out a solution that we're all happy with.
The real issue seems to be one of feeding this information to cabal2arch.
Actually, I believe that this is a secondary issue right now. We ought to agree on the desired result before we begin to worry about how to achieve that result. Take care, Peter

On 2011/1/3 Peter Simons
> Maybe I'm being overly simplistic here, but I don't see the problem.
Well, I'm not sure why this issue appears to be controversial either. Personally, I suggested to translate that dependency to "parsec>=2.1.0", but no-one seems to agree with me, so I'm trying to figure out a solution that we're all happy with.
I perfectly remember agreeing with you on this: I suggested using the convex hull of possible versions instead of only the first or last interval, and even sent a patch doing this, should I file a pull request or commit directly instead ? -- Rémy.

Hi Rémy,
I perfectly remember agreeing with you on this.
oh, apparently I wasn't paying attention. :-( I am sorry.
I even sent a patch doing this. Should I file a pull request or commit directly instead?
Personally, I'm fine with committing the patch you posted earlier. It seems to do pretty much what I feel is the best solution. I'd say let's give everyone a chance to think about it and raise objections. If none surface in the next two or three days, then let's do it! Take care, Peter

On Sat, Jan 1, 2011 at 20:55, Rémy Oudompheng
On 2010/12/30 Magnus Therning
wrote: On Tue, Dec 28, 2010 at 09:08, Rémy Oudompheng
wrote: Hello,
Since pacman's dependency specification is much less powerful than in Cabal, cabal2arch may output wrong version constraints while trying to approximate the true specified dependencies. I prefer this to dropping all version requirements when they cannot be expressed in the pacman's way.
This sounds reasonable. However, do you have examples of dependencies that CABAL can express but pacman can't? (I was under the impression that the difference was that CABAL allowed more terse expressions, not that it allowed more.)
See the issue reported by https://github.com/archhaskell/cabal2arch/issues#issue/19. PKGBUILDs can only specify dependency requirements with one inequality (<, >, <= or >=) whereas Cabal can specify any Boolean combination of inequalities.
I think it's more correct to say that PKGBUILDS only can specify conjuctions of dependencies, which does allow it to specify both an upper and lower bound but not the more complicated dependencies that CABAL allows. However, is this really a limitation in practice for us? (I do understand that it's a limitation in that it makes it more difficult to automatically translate dependencies but I see that as a separate issue.)
In order to get improved PKGBUILDs I thought we could have a better usage of the dependency resolution capabilities in the Cabal library, this is summed up in this commit
http://github.com/remyoudompheng/archhaskell/commit/2e614b2
This adds an extra file, platform-provides.txt, that lists our preferred versions for various packages. For the moment I have filled it with the contents of Haskell Platform.
Is it worth considering downloading this file dynamically, rather than compiling it into the executable statically?
It's been some time since these lists were not compiled in the executable. I have added a flag to cabal2arch (in my copy http://github.com/remyoudompheng/cabal2arch) to handle a custom directory or URL where these files could be stored. I am using mtl for error handling.
Why not remove the file from github completely and host it in a well-known location then? /M -- Magnus Therning OpenPGP: 0xAB4DFBA4 email: magnus@therning.org jabber: magnus@therning.org twitter: magthe http://therning.org/magnus

2011/1/2 Magnus Therning
On Sat, Jan 1, 2011 at 20:55, Rémy Oudompheng
wrote: On 2010/12/30 Magnus Therning
wrote: On Tue, Dec 28, 2010 at 09:08, Rémy Oudompheng
wrote: In order to get improved PKGBUILDs I thought we could have a better usage of the dependency resolution capabilities in the Cabal library, this is summed up in this commit
http://github.com/remyoudompheng/archhaskell/commit/2e614b2
This adds an extra file, platform-provides.txt, that lists our preferred versions for various packages. For the moment I have filled it with the contents of Haskell Platform.
Is it worth considering downloading this file dynamically, rather than compiling it into the executable statically?
It's been some time since these lists were not compiled in the executable. I have added a flag to cabal2arch (in my copy http://github.com/remyoudompheng/cabal2arch) to handle a custom directory or URL where these files could be stored. I am using mtl for error handling.
Why not remove the file from github completely and host it in a well-known location then?
Because I just did that yesterday :) I have put examples of files at http://dev.archlinux.org/~remy/arch-haskell/ For example cabal2arch --sysinfo=http://dev.archlinux.org/~remy/arch-haskell/hp2011.0 will produce PKGBUILDs which are forced to use library versions specified in HP2011.0 while cabal2arch --sysinfo=http://dev.archlinux.org/~remy/arch-haskell/default will use the behaviour we had before. -- Rémy.
participants (4)
-
Magnus Therning
-
Peter Simons
-
Rémy Oudompheng
-
Xyne