LFM for Haskell bittorrent effort

Hi, For this or another reason, no bittorrent libraries/clients available today seem to suit my needs: they are either too slow, too memory inefficient, bundled with a client which often can't run in a headless environment or broken in some other way or don't implement all the common BEPs. I came to the conclusion that I'll probably have to write one. Happily, whenever I mentioned the topic in #haskell, there was quite a bit of interest each time so I'm looking to recruit people to join me in this effort. First a brief overlook of what's already available in the Haskell environment: * Combinatorrent[1] – a rather old client, no longer maintained with the last Hackage upload in 2010. Only supports few BEPs and was not able to load in the .torrent file provided by Ubuntu. I have spoken to jlouis who wrote this, you can read the conversation at [2]. He himself says that nowadays you can do better performance wise with available libraries. Additionally, this comes with a client on top rather than just being a library. * conjure – not touched since 2008 * bittorrent – seems to be the closest to what I'd like but it is still incomplete and the library is in limbo. There is a lot of stuff that does not work or is commented out and for various reasons the maintainer is unavailable to help out. I have spoken to the maintainer and he himself would like to redesign the library. That's about it. There seem to be a few related tools for bencoding &c but nothing huge. I think the best way to go about this is to learn from Combinatorrent and bittorrent and start something fresh. I'll outline the goals I personally am looking for even though they might be obvious: * efficient, scalable – the library needs to be able to handle tens of thousands of active (announcing, not necessarily downloading) torrents at once. For reference, libtorrent-rakshasa + rtorrent is able to run about ~10k torrents in about 1GB of memory with low CPU usage. The downside is that the library crashes like no tomorrow and the client itself sucks. A slow library is useless to me but so is a fast but unstable one. * implements all the common BEPs – no use for a library if it doesn't work for a lot of cases. It needs to implement everything from [3] maybe except BEP 8. * sane interface – in the end we need to be able to build clients on top. Fortunately we can leverage Haskell's types to do this whereas the existing C/C++ bittorrent libraries seems to suffer badly here. * resilient – if I wanted a fast library which crashes every ten minutes I'd stick with libtorrent Basically, I'm looking to build a fully usable bittorrent library. Notably, I'm not looking for people to help to write a client (yet!). I plan to start as soon as time allows but it'd be great if there was more than just myself on board. Let me know one way another if you're interested in contributing. If you're just interested in the final product then that's fine too, I could always use more motivation! [1]: https://github.com/jlouis/combinatorrent [2]: http://ircbrowse.net/browse/haskell?id=18701612×tamp=1406738464#t1406738464 [3]: http://www.bittorrent.org/beps/bep_0000.html -- Mateusz K.

On 08/31/2014 09:04 PM, Mateusz Kowalczyk wrote:
Hi,
For this or another reason, no bittorrent libraries/clients available today seem to suit my needs: they are either too slow, too memory inefficient, bundled with a client which often can't run in a headless environment or broken in some other way or don't implement all the common BEPs. I came to the conclusion that I'll probably have to write one. Happily, whenever I mentioned the topic in #haskell, there was quite a bit of interest each time so I'm looking to recruit people to join me in this effort.
[snip]
I had a few replies already which is great news for me. If you're interested in contributing then send me your GitHub username (or SSH key if you don't like GitHub) and I'll create an organisation later. There is a #haskell-bittorrent channel that was previously used for the ‘bittorrent’ library which I think we can use, if you use IRC of course. -- Mateusz K.
participants (1)
-
Mateusz Kowalczyk