April 2003 - Libraries

RE: FW: First Attempt at Crypto Library
by Simon Marlow 25 Apr '03

25 Apr '03

> > I initially thought along these lines, but then concluded > that it's not > > that simple (or at least, that's not the only interpretation of the > > classification). > > Is a deep hierarchy really necessary? Well, it seems useful to group codecs by purpose: Audio, Video, Compression, etc. so I believe we're not into diminishing returns at this level. Matt Harden said: > I would agree with Codec.Binary.Base64. Codec.Text should be > reserved for codecs that deal with text data; i.e. translating > text between various encodings like unicode, ISO 8859-1, ascii, > ebcdic, etc. That's a good point. Unless anyone objects, let's reserve Codec.Text for the text encodings (UTF-8 and so on). The options for Base64 and friends are therefore: Codec.Binary.Base64, Codec.Base64, or Codec.General.Base64. Or even Codec.BinaryToText.Base64? None stands out as the obvious choice to me, but we could defer the decision until the library actually exists ;-) Cheers, Simon

8 10

RE: FW: Library infrastructure
by Simon Marlow 24 Apr '03

24 Apr '03

Some people have mentioned other systems that might be relevant: the BSD ports tree, CPAN, etc. I'd just like to clarify what (IMHO) we're not aiming for here: in particular, we don't need a system that - downloads source - manages dependencies (beyond what the compiler needs to know) And I'd argue that we don't even need a system that builds binary distributions. The reason is that these tasks are all handled by the various OS package systems: BSD ports, RPM, apt-get, emerge, and so on. We don't want to duplicate or worse, bypass, the local package management system. We do need a system that helps you generate compliant package skeletons in the local native format: an RPM spec, a BSD ports Makefile, or a Gentoo ebuild. Cheers, Simon

7 11

FW: Library infrastructure
by Dominic Steinitz 23 Apr '03

23 Apr '03

I'm not clear what happens at the moment. I had naively assumed that having created my library that it would be put into the ghc cvs repository and would then be available in the same way other libraries are when you install ghc. Do libraries get put into the ghc cvs repository? If they do, do they get put into the ghc release? If they don't then how does someone who wants to use one get hold of it? My thinking with the crypto library was that there were quite a few crypto functions available but they were spread out all over the place and had different interfaces and wouldn't it be useful if there were one library with a uniform interface. I'd be happy to help in building a library infrastructure but I think we should do the minimum possible to deliver the absolutely essential elements only. How do we capture the requirements? Do we have anything at all that does some of the job that we can build on? >From my reading of this thread, I thought Malcolm Wallace had something. Dominic Steinitz

2 1

Re: First Attempt at Crypto Library
by Dominic Steinitz 23 Apr '03

23 Apr '03

I'm not sure where we are on the name for the proposed crypto library. Are we going for: Codec.Encryption.DES Codec.Encryption.RSA Codec.Encryption.Blowfish If so I'm not sure how modes such as Cipher Block Chaining (see below) fit in as they apply to all block ciphers. There is also the issue of padding schemes. Are we saying that there should be: Codec.Encryption.Modes Codec.Encryption.Padding I'm not sure what will happen when we add asymmetric algorithms but it seems ok for now. Should what Codec.Encryption.DES exports be des & unDes or should they be encrypt and decrypt? I presume the latter. -- * CBC or Cipher Block Chaining Mode -- | In CBC or Cipher Block Chaining mode each block is XORed with -- the previous enciphered block before encryption. For the first -- block we start with an initialization vector. cbc :: Bits block => (key -> block -> block) -> block -> key -> [block] -> [block] cbc e iv k ps = ciphers where ciphers = map (e k) feedIns feedIns = zipWith xor (iv : ciphers) ps Dominic Steinitz ----- Original Message ----- From: "Dominic Steinitz" <dominic.steinitz(a)blueyonder.co.uk> To: "Simon Peyton-Jones" <simonpj(a)microsoft.com>; "Simon Marlow" <simonmar(a)microsoft.com> Cc: <libraries-request(a)haskell.org> Sent: Monday, April 21, 2003 6:46 PM Subject: First Attempt at Crypto Library > Simon, Simon, > > Here's my first attempt at a crypto library. It compiles and I can run a > test using ghc-inplace. The test checks with the example in > http://www.itl.nist.gov/fipspubs/fip81.htm (except I couldn't find an > example with PKCS#5 padding). I'm not sure what the next steps are. > > Dominic Steinitz >

1 0

FW: Library infrastructure
by Simon Peyton-Jones 23 Apr '03

23 Apr '03

Gentle Haskellers Haskell has now got a decent, hierarchical module system, supported by GHC, Hugs and NHC, and the library structure is now quite well populated. For example, here are the "base" libraries shipped with GHC (similar libraries come with Hugs and Nhc): http://www.haskell.org/ghc/docs/latest/html/base/index.html Languages flourish when libraries are plentiful, reliable, and well documented. In an open-source community like the Haskell community, plentiful libraries depend on contributions from the community. The difficulty is that it is currently not obvious what Joe Programmer should do to contribute a library to the community. I think this difficulty is a major obstacle to Haskell's progress, perhaps even the largest single obstacle today. One route to getting a library widely distributed is to get it into the "libraries" source tree used by GHC, Hugs, and NHC. That has the merit that the library will be distributed with each of these compilers. But it's too much of a "cathedral" approach. There is no way to distribute libraries between releases, and it makes the distributions ever larger and more complex for the GHC/Hugs/Nhc teams to build. To complement this, there ought to be an easy "bazaar" way for Joe Programmer to implement a library, build distributions, put them up on his web site for others to download, and add a link to some central library index so that people can find it. Sadly this is not an entirely simple task, because (a) the library should work on all Haskell implementations, and (b) should be packaged for binary installation on a variety of platforms (Windows, Linux, OpenBSD, Mac etc). Indeed, since GHC, Hugs and Nhc have different packaging mechanisms, there's a kind of (a)*(b) product involved. The bottom line is that today the obstacles are high enough to discourage all but the most motivated. What is needed is some kind of software infrastructure to automate the common tasks, and to give clear instructions about the non-automatable parts. This infrastructure should not be part of the (complicated) GHC (or Hugs, or Nhc) build systems; indeed, while it will certainly contain implementation-specific parts, its structure should be implementation independent. Simon Marlow emitted a message a while ago about this, but discussion petered out: http://www.haskell.org/pipermail/glasgow-haskell-users/2002-November/thr ead.html#4322 The lack of this library-building infrastructure is a serious problem. Solving it, or even partially solving it, would be a huge step forward. So this message is really to try to kick-start the process again. Is anyone willing to invest effort in making this happen? It's probably a task that is too big for one person. I think what we need is A library-infrastructure Czar, who will a) do the core design and implementation b) coordinate the whole thing to make sure progress happens A bunch of library-infrastructure Supporters, who are willing a) to contribute thoughtfully to the design b) to implement pieces of it, for example pieces specific to one implementation (e.g. GHC) or one target architecture (e.g. PowerPC) Perhaps the first thing to do is to assemble list of people who would be willing to contribute as supporters (in the above sense), hoping that a Czar may then emerge by popular acclaim. Would you like to reply, perhaps to the 'libraries(a)haskell.org' list, so indicate interest. I really think this is important. If we do not collectively evolve a way to develop libraries in "bazaar" mode, I think Haskell will at best develop more slowly, and at worst wither away. Simon PS1: This is all distinct from the question of library *content* --- that is, what the module name, discussion about the signature of the exported types and functions are --- which does seem be handled OK by the 'libraries' list. This message concerns library-building *infrastructure*. PS2: Another entirely-orthogonal question is where library source code *lives*. One possibility is to use the same CVS repository that GHC/Hugs/Nhc use for libraries, albeit in a separate subtree. Perhaps fptools/user-libs/joe-bloggs/*. Another possibility is SourceForge. Finding backed-up CVS space for source code is not a problem.

9 10

RE: bug in hPutPS
by Simon Marlow 23 Apr '03

23 Apr '03

> In local.libraries, you wrote: > > hPutPS (at least as packaged in ghc 5.04.2--I haven't yet tried with > > 5.04.3) fails when given an empty string with the error: > > > > Action: hPutArray > > Handle: {loc=<stdout>,type=writable,binary=False,buffering=none} > > Reason: illegal buffer size 0 > > > > I guess maybe this is actually a bug in hPutArray (which I > know nothing > > about), but I'm sure that one should be able to output an > empty string. > > hPutArray with count = 0 probably shouldn't fail, I'll check this. > If generating an empty array in hPutPS is a bug I cannot tell. I just fixed this. Cheers, Simon

1 0

bug in hPutPS
by David Roundy 23 Apr '03

23 Apr '03

hPutPS (at least as packaged in ghc 5.04.2--I haven't yet tried with 5.04.3) fails when given an empty string with the error: Action: hPutArray Handle: {loc=<stdout>,type=writable,binary=False,buffering=none} Reason: illegal buffer size 0 I guess maybe this is actually a bug in hPutArray (which I know nothing about), but I'm sure that one should be able to output an empty string. -- David Roundy http://www.abridgegame.org

2 1

RE: FW: First Attempt at Crypto Library
by Simon Marlow 23 Apr '03

23 Apr '03

> > Since we already have a little hierarchy based on the > codec's purpose: > > Codec.Image (e.g. Codec.Image.Jpeg) > > Codec.Video (e.g. Codec.Video.Mpeg2) > > then how about adding > > Codec.Text (e.g. Codec.Text.Base64) > > because its role, like uuencoding, is to convert binary streams to > > 7-bit ASCII text i.e. transmissible by email. > > I would have thought Codec.Binary.Base64, since in the other > cases (Image > and Video), the name is for the content you wish to code, not > the format in > which it is encoded. e.g. Jpeg is a format for encoding an image in > binary, while xpm is a format for encoding an image as text, > but both would go under Codec.Image. I initially thought along these lines, but then concluded that it's not that simple (or at least, that's not the only interpretation of the classification). For example, Jpeg is a coding system which encodes raw binary image data into jpeg image data: both the source and encoded result are image data, and they're both binary. So the term 'Image' refers to the purpose of the codec, not the source or destination format. Looking at it this way, 'Text' is a reasonable classification for a codec that encodes arbitrary raw binary data into ascii text: the purpose of the codec is to generate text. Ok, it's a little bit flimsy, but IMHO better than Codec.Binary which doesn't tell you anything: most codecs take binary input. Cheers, Simon

2 1

Re: FW: Library infrastructure
by Yoann Padioleau 22 Apr '03

22 Apr '03

"Simon Peyton-Jones" <simonpj(a)microsoft.com> writes: > Gentle Haskellers > > Haskell has now got a decent, hierarchical module system, supported by > GHC, Hugs and NHC, and the library structure is now quite well > populated. For example, here are the "base" libraries shipped with GHC > (similar libraries come with Hugs and Nhc): > http://www.haskell.org/ghc/docs/latest/html/base/index.html > > Languages flourish when libraries are plentiful, reliable, and well > documented. In an open-source community like the Haskell community, > plentiful libraries depend on contributions from the community. > > The difficulty is that it is currently not obvious what Joe > Programmer > should do to contribute a library to the community. > > I think this difficulty is a major obstacle to Haskell's > progress, > perhaps even the largest single obstacle today. I always find cvs difficult to use, why not put a wiki website for the library code where everyone could *easily* contribute. There needs some control, but simply not allowing a user to add some code that does not compile can do the tricks. -- Yoann Padioleau, INSA de Rennes, France, Opinions expressed here are only mine. Je n'écris qu'à titre personnel. **____ Get Free. Be Smart. Simply use Linux and Free Software. ____**

2 1

RE: FW: First Attempt at Crypto Library
by Simon Marlow 22 Apr '03

22 Apr '03

> This reminds me, for my ginsu project > http://repetae.net/john/computer/ginsu/ > I implemented interfaces to curses and part of OpenSSL's cryptography > routines. Would there be interest in putting those into the general > haskell librarys as well? As far as including more code in the libraries that we ship with GHC goes, the gates are pretty much closed right now - if your library is small enough it might be able to sneak in, but we're actively trying to keep a bound on the amount of code we have to build/ship with GHC. We're aware that this makes life difficult for those trying to supply libraries for use with GHC, and both Simon & I think this is a big problem. We've suggested[1] that what the community as a whole needs is an independently-maintained library infrastructure that supports building and packaging libraries in a system- and compiler-independent way. This is a BIG undertaking, which we don't have the time to tackle. So far no-one has stepped up to the plate (with the possible exception of Henrik Nilsson who recently distributed some code which takes an initial stab at the problem). This doesn't mean that using the CVS repository and putting your libraries under fptools/libraries is ruled out; indeed there are lots of libraries under fptools/libraries that aren't shipped with GHC by default (ObjectIO, GLUT, X11, HGL, HaXml, etc.). We welcome more libraries in the source tree, and that's one way to get access to a build system which already does a lot of the work for you - but the fptools build system doesn't provide the packaging facilities that we'd want from a more general library infrastructure. Cheers, Simon [1] http://www.haskell.org/pipermail/glasgow-haskell-users/2002-November/004 322.html

2 1