
Hi all, as some of you may have noticed Lambdabot doesn't build on GHC 7.6.1 due to OldException being removed (and a few other changes). I updated the code so that it builds on latest GHC release. The updated code is available here: https://github.com/killy9999/lambdabot/tree/upstream It builds on Linux, but I am not 100% certain that it will build on Windows. If it doesn't I will be grateful for reporting this (and any other) issues on the bug tracker: https://github.com/killy9999/lambdabot/issues The changes required to make the code run weren't as small as I initially thought they will be. In some places this resulted in a bit ugly code, but aside from that I think that everything should work correctly. Now I would like to do a bit of maintanance on the project, since the code is a bit of a mess and it could use some cleanup. My plans are: - clean up the package namespace by placing existing modules in Lambdabot module - integrate lambdabot-utils into lambdabot (already done) - reformat the code so that it's easier to read ans if time allows: - cleanup some messy code - update the package dependencies so that lambdabot relies on maintained libraries (e.g. I thought about replacing readline with haskeline). This changes would be quite invasive and code wouldn't be compatible with the lambdabot repo on haskell.org. So before I start making any of them I would like to hear from the community if such changes in the source code of lambdabot would be considered helpful and acceptable. Janek

On 17 February 2013 18:03, Jan Stolarek
... This changes would be quite invasive and code wouldn't be compatible with the lambdabot repo on haskell.org. So before I start making any of them I would like to hear from the community if such changes in the source code of lambdabot would be considered helpful and acceptable.
Janek
I say go for it! I'll be quite happy to start running your new code in #haskell as soon as we get GHC 7.6 installed on Jason's Linode account. :) - Cale

On Sun, Feb 17, 2013 at 8:53 PM, Cale Gibbard
On 17 February 2013 18:03, Jan Stolarek
wrote: ... This changes would be quite invasive and code wouldn't be compatible with the lambdabot repo on haskell.org. So before I start making any of them I would like to hear from the community if such changes in the source code of lambdabot would be considered helpful and acceptable.
Janek
I say go for it! I'll be quite happy to start running your new code in #haskell as soon as we get GHC 7.6 installed on Jason's Linode account. :)
My name is Jason and I endorse this message :) Jason

I'm happy to hear your approval. I've spent some time yesterday cleaning up the code and writing down all things that do not work. The list I made is avaliable on github: https://github.com/killy9999/lambdabot/issues There are 17 open issues at the moment and I know I will not have enough time to resolve all of them on my own. There are a few blockers, so the current code is definitely not yet suitable for any kind of release. With all that said I would appreciate help of the community :-) There's a lot to do and the tasks vary in difficulty: from updating documentation and fixing unused imports to fixing exceptions. If anyone is willing to help please contact me via email or comments on github. Janek P.S. The development is done in the 'upstream' branch

For what it's worth, I also have a fork of lambdabot[1] I've been using for quite a while now, which may provide a cleaner starting point. In particular, it updates the plugin and command API to be (IMO) quite a bit cleaner and easier to use and understand. It could probably use some attention to security in the eval module (I've relaxed it quite a bit because I only run it on a private IRC server), but it also includes a lot of fixes and updates to other modules.
[1] https://github.com/mokus0/lambdabot
On Feb 19, 2013, at 2:00 AM, Jan Stolarek
I'm happy to hear your approval.
I've spent some time yesterday cleaning up the code and writing down all things that do not work. The list I made is avaliable on github:
https://github.com/killy9999/lambdabot/issues
There are 17 open issues at the moment and I know I will not have enough time to resolve all of them on my own. There are a few blockers, so the current code is definitely not yet suitable for any kind of release.
With all that said I would appreciate help of the community :-) There's a lot to do and the tasks vary in difficulty: from updating documentation and fixing unused imports to fixing exceptions. If anyone is willing to help please contact me via email or comments on github.
Janek
P.S. The development is done in the 'upstream' branch
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Wow, this indeed looks like a nice starting point, though I can't build lambdabot from your repo - seems that dice package is not on Hackage. Is this the package that you rely on: https://github.com/serialhex/dice ? Anyway, how would you feel about changes that I would like to make: - move all modules into Lambdabot. namespace - remove unlambda, brainfuck and show from the repo. They are on hackage, no need to keep them here - these packages aren't even used in the build process. - cleanup scripts - add package versions to cabal file This is mostly code refactoring. Janek

Sorry, I uploaded it this morning since I knew it wasn't there (it's the "dice" repo from my github account, mokus0). Have you run "cabal update" in the last 4 or 5 hours?
On Feb 19, 2013, at 2:36 PM, Jan Stolarek
Wow, this indeed looks like a nice starting point, though I can't build lambdabot from your repo - seems that dice package is not on Hackage. Is this the package that you rely on:
https://github.com/serialhex/dice ?
Anyway, how would you feel about changes that I would like to make: - move all modules into Lambdabot. namespace - remove unlambda, brainfuck and show from the repo. They are on hackage, no need to keep them here - these packages aren't even used in the build process. - cleanup scripts - add package versions to cabal file
This is mostly code refactoring.
Janek

On Tue, Feb 19, 2013 at 5:36 PM, Jan Stolarek
- remove unlambda, brainfuck and show from the repo. They are on hackage, no need to keep them here - these packages aren't even used in the build process.
Where will they go? -- gwern http://www.gwern.net

Dnia wtorek, 19 lutego 2013, Gwern Branwen napisał:
On Tue, Feb 19, 2013 at 5:36 PM, Jan Stolarek
wrote: - remove unlambda, brainfuck and show from the repo. They are on hackage, no need to keep them here - these packages aren't even used in the build process.
Where will they go? These packages are already on hackage:
http://hackage.haskell.org/package/brainfuck http://hackage.haskell.org/package/show http://hackage.haskell.org/package/unlambda No need to keep them in the lambdabot repo. Janek

On 20/02/13 08:13, Jan Stolarek wrote:
Dnia wtorek, 19 lutego 2013, Gwern Branwen napisał:
On Tue, Feb 19, 2013 at 5:36 PM, Jan Stolarek
wrote: - remove unlambda, brainfuck and show from the repo. They are on hackage, no need to keep them here - these packages aren't even used in the build process.
Where will they go? These packages are already on hackage:
http://hackage.haskell.org/package/brainfuck http://hackage.haskell.org/package/show http://hackage.haskell.org/package/unlambda
No need to keep them in the lambdabot repo.
They are in the lambdabot *repository*, but in separate hackage packages. Are you suggesting that the source code of these packages is moved out to their own darcs repositories? Twan

Are you suggesting that the source code of these packages is moved out to their own darcs repositories? Exactly. This allows to use and develop these packages independently of lambdabot and I consider that a Good Thing. I'm also much in favor of using git, because github allows easy collaboration between community members.
Janek

On Wed, Feb 20, 2013 at 12:59 PM, Jan Stolarek
Exactly. This allows to use and develop these packages independently of lambdabot and I consider that a Good Thing. I'm also much in favor of using git, because github allows easy collaboration between community members.
It may be a good thing, but speaking as the de facto maintainer of lambdabot for the past few years, it's a very small good thing and the goodness may be outweighed by the costs of switching: hardly anyone ever sends in patches for lambdabot proper, and even fewer for those add-on runtime dependencies. I am reminded of a recent incident on the XMonad mailing list: an enthusiastic young member proposed changing the entire infrastructure to Github because Github is the new hotness and it would surely promote easy collaboration between community members and so on and so forth. He put in a bunch of work in making copies and converting repos etc, and... nothing happened. His effort was wasted. Turns out the reason for people not submitting patches had more to do with things besides not being hosted on Github. -- gwern http://www.gwern.net

goodness may be outweighed by the costs of switching: Could you specify what are those "cost of switching"?
Turns out the reason for people not submitting patches had more to do with things besides not being hosted on Github. Of course. I don't clain this is the reason for people not submitting patches. All I say is that I consider git+github to be a much easier platform for collaboration than darcs. I think many people would agree.
Gwern, and what do you think about James' fork of lambdabot? It seems that there was a lot of work put into it and that this is indeed a good starting point to continue development. Janek

My 2 cents: Switching to github: +0.1 Splitting into core & contrib: +1 Okay I suppose that is more like 1.1 cents. I would love to help maintain lambdabot and help brush off the bitrot, but am personally quite overextended so I can't really promise any amount of dedication for at least the next few weeks. I can promise to occasionally throw my opinion around as if it mattered, though. ;) -- Dan Burton

On Wed, Feb 20, 2013 at 1:35 PM, Jan Stolarek
Gwern, and what do you think about James' fork of lambdabot? It seems that there was a lot of work put into it and that this is indeed a good starting point to continue development.
I haven't looked at the diffs; if as he says the security around the evaluation has been weakened, that's a deal-breaker until it's fixed. lambdabot can't be insecure since it will be run in a public IRC. -- gwern http://www.gwern.net

On Wed, Feb 20, 2013 at 11:19 AM, Gwern Branwen
Gwern, and what do you think about James' fork of lambdabot? It seems
On Wed, Feb 20, 2013 at 1:35 PM, Jan Stolarek
wrote: that there was a lot of work put into it and that this is indeed a good starting point to continue development.
I haven't looked at the diffs; if as he says the security around the evaluation has been weakened, that's a deal-breaker until it's fixed. lambdabot can't be insecure since it will be run in a public IRC.
I absolutely agree. There are sandboxing things we can do around the lambdabot instance, but Haskell has a lot of opportunities for statically disallowing questionable things. I would like to start our defense there and add layers around that. My real reason for reviving this thread: Can I get a status update, please? Thanks, Jason

On Mar 14, 2013, at 11:08 PM, Jason Dagit
My real reason for reviving this thread: Can I get a status update, please?
Sure. I don't have as much time as I'd like these days for open-source projects, but with Jan's help the code has been cleaned up quite a bit in general, and a lot of old bit-rot has been fixed. I have not specifically addressed security yet, but it's not as dire a situation as I think it sounded from my earlier remarks. Basically, security is currently a bit "DIY". If there are any holes, they are probably quite subtle because there are very, very few moving parts on lambdabot's side. mueval and Safe Haskell are used to enforce resource limitations and type-safety, respectively, but -fpackage-trust is not (yet) turned on. So all packages installed on the system are available for the interpreted code (if imported using a command such as "@let import Foo.Bar"), as long as Safe Haskell permits. This is the main potential security hole - the system may have modules installed and marked "Trustworthy" that the administrator has not explicitly decided whether to trust, and which are not, in fact, as safe as the author asserts. Currently, lambdabot trusts such packages. My intention is to add some commands (available only to lambdabot admins) or maybe a static configuration option for managing a list of packages to explicitly trust, with all others being untrusted. And of course, for a "production" system an OS-enforced sandbox is a great idea no matter how secure you believe the code do be. Aside from that caveat, I think that the code could be put on hackage today and I'd have few, if any, reservations about it. -- James

On Fri, Mar 15, 2013 at 9:19 AM, James Cook
On Mar 14, 2013, at 11:08 PM, Jason Dagit
wrote: My real reason for reviving this thread: Can I get a status update, please?
Sure. I don't have as much time as I'd like these days for open-source projects, but with Jan's help the code has been cleaned up quite a bit in general, and a lot of old bit-rot has been fixed. I have not specifically addressed security yet, but it's not as dire a situation as I think it sounded from my earlier remarks.
Basically, security is currently a bit "DIY". If there are any holes, they are probably quite subtle because there are very, very few moving parts on lambdabot's side. mueval and Safe Haskell are used to enforce resource limitations and type-safety, respectively, but -fpackage-trust is not (yet) turned on. So all packages installed on the system are available for the interpreted code (if imported using a command such as "@let import Foo.Bar"), as long as Safe Haskell permits. This is the main potential security hole - the system may have modules installed and marked "Trustworthy" that the administrator has not explicitly decided whether to trust, and which are not, in fact, as safe as the author asserts. Currently, lambdabot trusts such packages. My intention is to add some commands (available only to lambdabot admins) or maybe a static configuration option for managing a list of packages to explicitly trust, with all others being untrusted.
And of course, for a "production" system an OS-enforced sandbox is a great idea no matter how secure you believe the code do be.
Aside from that caveat, I think that the code could be put on hackage today and I'd have few, if any, reservations about it.
I haven't been following the thread closely. Is there also a github? If so, where? Some of us figured out a bug fix for the quotes plugin and I'll send a pull request if I get a chance. Thanks for the update, Jason

On Mar 15, 2013, at 2:45 PM, Jason Dagit
I haven't been following the thread closely. Is there also a github? If so, where? Some of us figured out a bug fix for the quotes plugin and I'll send a pull request if I get a chance.
Yep, there is[1]. I'm not sure what the specific bug is that you are referring to, but it's possible it doesn't exist anymore - a large part of the quotes plugin has been rewritten (actually outsourced to a fortune-mod clone written in Haskell called "misfortune"). If it still does, then of course I'd be happy to accept a fix :) [1] https://github.com/mokus0/lambdabot

On Fri, Mar 15, 2013 at 3:30 PM, James Cook
On Mar 15, 2013, at 2:45 PM, Jason Dagit
wrote: I haven't been following the thread closely. Is there also a github? If so, where? Some of us figured out a bug fix for the quotes plugin and I'll send a pull request if I get a chance.
Yep, there is[1]. I'm not sure what the specific bug is that you are referring to, but it's possible it doesn't exist anymore - a large part of the quotes plugin has been rewritten (actually outsourced to a fortune-mod clone written in Haskell called "misfortune"). If it still does, then of course I'd be happy to accept a fix :)
Awesome. I believe the bug is still there. The type for the quote db is: type Key = P.ByteString type Quotes = M.Map Key [P.ByteString] Which leaves the possibility that a key exists but there are no quotes. This is problematic for the current version of random. I glanced at your new version and it wasn't clear to me if it's still a problem (I suspect it is). One bandaid for this is to change the lines below: https://github.com/mokus0/lambdabot/blob/master/src/Lambdabot/Plugin/Quote.h... https://github.com/mokus0/lambdabot/blob/master/src/Lambdabot/Plugin/Quote.h... In both cases "Just qs" could be changed to "Just qs@(_:_)" and then empty lists would fall through to the default case. The other fix is to prune out degenerate entries (where key maps to the empty list). I believe that would be fixed in the serialization function: moduleSerialize = Just mapListPackedSerial Changing that to something like: moduleSerialize = Just mapListPackedSerialSansEmpties where mapListPackedSerialSansEmpties = mapListPackedSerial { serialize = (serialize mapListPackedSerial) . Map.filter (not.null) } Perhaps that should be added to the Serial module as an alternative to mapListPackedSerial. I haven't tested any of the above code (or even tried to compile it). Jason

On Fri, Mar 15, 2013 at 4:31 PM, Jason Dagit
On Fri, Mar 15, 2013 at 3:30 PM, James Cook
wrote: On Mar 15, 2013, at 2:45 PM, Jason Dagit
wrote: I haven't been following the thread closely. Is there also a github? If so, where? Some of us figured out a bug fix for the quotes plugin and I'll send a pull request if I get a chance.
Yep, there is[1]. I'm not sure what the specific bug is that you are referring to, but it's possible it doesn't exist anymore - a large part of the quotes plugin has been rewritten (actually outsourced to a fortune-mod clone written in Haskell called "misfortune"). If it still does, then of course I'd be happy to accept a fix :)
Awesome.
I believe the bug is still there. The type for the quote db is: type Key = P.ByteString type Quotes = M.Map Key [P.ByteString]
Which leaves the possibility that a key exists but there are no quotes. This is problematic for the current version of random. I glanced at your new version and it wasn't clear to me if it's still a problem (I suspect it is). One bandaid for this is to change the lines below:
https://github.com/mokus0/lambdabot/blob/master/src/Lambdabot/Plugin/Quote.h...
https://github.com/mokus0/lambdabot/blob/master/src/Lambdabot/Plugin/Quote.h...
In both cases "Just qs" could be changed to "Just qs@(_:_)" and then empty lists would fall through to the default case.
The other fix is to prune out degenerate entries (where key maps to the empty list). I believe that would be fixed in the serialization function: moduleSerialize = Just mapListPackedSerial
Changing that to something like: moduleSerialize = Just mapListPackedSerialSansEmpties where mapListPackedSerialSansEmpties = mapListPackedSerial { serialize = (serialize mapListPackedSerial) . Map.filter (not.null) }
Perhaps that should be added to the Serial module as an alternative to mapListPackedSerial.
I haven't tested any of the above code (or even tried to compile it).
I was going to start making these changes and I noticed that it doesn't currently build with ghc 7.4.1 w/Haskell Platform: https://travis-ci.org/dagit/lambdabot/builds/5541375 Do you know if the constraints on: regex-posix-0.95.1 regex-compat-0.95.1 Need to be what they are? Could we relax them without breaking anything? Jason

On Mar 15, 2013, at 5:33 PM, Jason Dagit
Do you know if the constraints on: regex-posix-0.95.1 regex-compat-0.95.1
Need to be what they are? Could we relax them without breaking anything?
The constraints were added recently, and I believe they were a very conservative estimate based on what versions were specifically present on Jan's machine at the time. Almost all of them could probably be relaxed.

On Mar 15, 2013, at 5:33 PM, Jason Dagit
I was going to start making these changes and I noticed that it doesn't currently build with ghc 7.4.1 w/Haskell Platform: https://travis-ci.org/dagit/lambdabot/builds/5541375
Do you know if the constraints on: regex-posix-0.95.1 regex-compat-0.95.1
Need to be what they are? Could we relax them without breaking anything?
I've relaxed those, along with many others that I think were unnecessarily strict, and it now builds on travis: https://travis-ci.org/mokus0/lambdabot/builds/5674142 -- James

On Feb 19, 2013, at 2:36 PM, Jan Stolarek
Anyway, how would you feel about changes that I would like to make: - move all modules into Lambdabot. namespace - remove unlambda, brainfuck and show from the repo. They are on hackage, no need to keep them here - these packages aren't even used in the build process. - cleanup scripts - add package versions to cabal file
I have no objections to any of these, though I would recommend as Gwern hinted that if related packages are to be removed that they should also be given new homes - I believe that the lambdabot source is currently the main home of these packages. I tend to prefer not to give package version upper bounds, but that's more of a preference than an objection ;)

On Tue, Feb 19, 2013 at 3:01 PM, James Cook
On Feb 19, 2013, at 2:36 PM, Jan Stolarek
wrote: Anyway, how would you feel about changes that I would like to make: - move all modules into Lambdabot. namespace - remove unlambda, brainfuck and show from the repo. They are on hackage, no need to keep them here - these packages aren't even used in the build process. - cleanup scripts - add package versions to cabal file
I have no objections to any of these, though I would recommend as Gwern hinted that if related packages are to be removed that they should also be given new homes - I believe that the lambdabot source is currently the main home of these packages.
Random thought, feel free to ignore it: Would it make sense to split lambdabot up into core and contrib like is done with xmonad? Contrib could contain the sillier things like bf, unlambda, show, etc and would have a lower bar for contributors. Core would be the standard things and the essential things. It seems that people don't really contribute new plugins these days but it would be great if they did. For example, having a plugin for liquid types would be super spiffy. Also, any plugin that helps people to reason about other code (like vacuum). Jason

On Feb 19, 2013, at 9:54 PM, Jason Dagit
Random thought, feel free to ignore it: Would it make sense to split lambdabot up into core and contrib like is done with xmonad? Contrib could contain the sillier things like bf, unlambda, show, etc and would have a lower bar for contributors. Core would be the standard things and the essential things.
Sounds like a good idea to me. I could probably do that this weekend if nobody else does sooner. Any opinion about whether it's better to put them in the same or separate actual repos? I've tried both with different collections of related packages and have no strong opinion, myself.
It seems that people don't really contribute new plugins these days but it would be great if they did. For example, having a plugin for liquid types would be super spiffy. Also, any plugin that helps people to reason about other code (like vacuum).
I suspect there are two big reasons for that. The biggest reason is probably that lambdabot has been getting long in the tooth, and the barrier to entry is sorting through a somewhat bit- rotted API with little to no documentation. That's why a lot of the work I did to clean it up for myself was API-related. The other is that there seemed to be, for a long time, no interest in maintaining it. Several years ago I did some work on fixing up some plugins, for example, but couldn't get a response from whoever was the current maintainer (I think it might have been dons, who was probably overextended at the time). Hopefully the API can be further simplified and documented; if it can be reduced to a few well-documented "core" modules and a few auxiliary utility/helper ones, that would probably make the educational barrier to entry quite a bit lower, and I'm happy to maintain the repo or willing to let others do so, so hopefully we can reduce the social barrier too. I don't currently have a lot of time to devote to hacking on it, but I try to be a responsive maintainer of all my projects. -- James

Any opinion about whether it's better to put them in the same or separate actual repos? The general rule in git is that a repo should contain a single project. There are some projects that violate this rule - e.g. cabal stores both Cabal and cabal-install in the same repository - but with such a setup it is harder to manage commits in case some of them need to be reverted. I'd say that if we split lambdabot in such a way then core should be one repository and contrib should be another one (separate). Contrib repository should contain only one package (instead of bunch of smaller ones).
couldn't get a response from whoever was the current maintainer Speaking of which, I think you could become the current maintainer since both Cale and Don agreed that the project should be maintained and you seem to have done a lot to improve it. Not saying that you should do all the development, but you could manage releases and uploads to Hackage.
I don't have too much time to spend on this, but I will try to regularly spend 2-3 hours a week on lambdabot. Janek

I've catalogued a list of issues on github: https://github.com/mokus0/lambdabot/issues Let me know if you have objections against any of them. If no then I will try to fix them gradually. Right now a major problem for me are exceptions in Data.Binary, but I don't think I will be able to fix them. Janek

I have no objections to any of these, Great. Then I will start making mentioned changes and sending pull requests when I find some free time.
though I would recommend as Gwern hinted that if related packages are to be removed that they should also be given new homes - I believe that the lambdabot source is currently the main home of these packages. OK, now I see. In that case we should create separate repositories for these packages.
I tend to prefer not to give package version upper bounds Seconded. I only want to specify lower bounds. I have already determined them so that code builds on GHC 7.4.2.
Random thought, feel free to ignore it: Would it make sense to split lambdabot up into core and contrib like is done with xmonad? I don't have opinion on this one.
Janek

Sounds great. Lambdabot is an important icon to the Haskell community; it
will be nice to brush off the bitrot and make lambdabot easier for the
average Haskeller to install without having to rely on Cale keeping it
running on irc (grateful, though we are).
-- Dan Burton
On Feb 17, 2013 3:04 PM, "Jan Stolarek"
Hi all,
as some of you may have noticed Lambdabot doesn't build on GHC 7.6.1 due to OldException being removed (and a few other changes). I updated the code so that it builds on latest GHC release. The updated code is available here:
https://github.com/killy9999/lambdabot/tree/upstream
It builds on Linux, but I am not 100% certain that it will build on Windows. If it doesn't I will be grateful for reporting this (and any other) issues on the bug tracker:
https://github.com/killy9999/lambdabot/issues
The changes required to make the code run weren't as small as I initially thought they will be. In some places this resulted in a bit ugly code, but aside from that I think that everything should work correctly.
Now I would like to do a bit of maintanance on the project, since the code is a bit of a mess and it could use some cleanup. My plans are:
- clean up the package namespace by placing existing modules in Lambdabot module - integrate lambdabot-utils into lambdabot (already done) - reformat the code so that it's easier to read
ans if time allows:
- cleanup some messy code - update the package dependencies so that lambdabot relies on maintained libraries (e.g. I thought about replacing readline with haskeline).
This changes would be quite invasive and code wouldn't be compatible with the lambdabot repo on haskell.org. So before I start making any of them I would like to hear from the community if such changes in the source code of lambdabot would be considered helpful and acceptable.
Janek
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
participants (7)
-
Cale Gibbard
-
Dan Burton
-
Gwern Branwen
-
James Cook
-
Jan Stolarek
-
Jason Dagit
-
Twan van Laarhoven