April 2006 - Libraries

RE: Idea to allow people to comment on Haskell docs
by Simon Peyton-Jones 09 Jan '07

09 Jan '07

| I've had a go at patching Haddock to put a link on the page for each | module. It's controlled by an optional --wiki=URL flag. | | As an example, I used this with the (devel version of the) Gtk2Hs docs: | | http://haskell.org/gtk2hs/docs/devel/ | | For example this page: | | http://haskell.org/gtk2hs/docs/devel/Graphics-Rendering-Cairo-SVG.html It looks great! But I'd never have found the link if you hadn't pointed me at it. Rather than "Wiki" say "User comments". And I think the per-entry [User comment] link is essential. Typically people are going to be looking at a particular thing. For example, here http://haskell.org/gtk2hs/docs/devel/Graphics-Rendering-Cairo-SVG.html#v %3AsvgRenderFromString is where I'd look for the link for svgRenderFromString. Does MediaWiki support anchors in the middle of a page? That's essential here -- we want one Wiki page for one Haddock page. On a closely related matter, Benjamin Pierce told me how often he wished that there was a link from the Haddock documentation to the source code for a function. Especially for Prelude functions, he's dead right. So another really useful Haddock enhancement would be the ability to add links to the source code from the Haddock docs --- again on a per-function basis. Do you feel like adding that too? Where should the code live? I'd suggest snap-shoting it at the moment that the Haddock docs are generated, and putting the files with the Haddock docs. Simon

12 15

RE: Reflection API for Haskell.
by Simon Peyton-Jones 22 Jun '06

22 Jun '06

I'm all for this. But I think the design needs a bit of work. Firstly, remember that TypeReps at the moment are designed for *ground types*, that is types with no free type variables. A ground type always starts with a type constructor. So one can start with typeOf to get a TyCon. The TyCon might not be an algebraic data type; that's why the DataType type in Data.Generics.Basics has a DataRep thing. You could add that. (You'd want to add FunRep.) I think you could probably get rid of dataTypeOf entirely, incidentally. When it comes to the types of data constructors, they have *quantified* types, so you need a way to bind the type variables. I don't see that in your design. [This relates to another missing feature of the current design which is that it can't represent higher rank types, e..g typeOf f where f :: (forall a. a->a) -> Int But allowing this would required not only a higher-rank TypeRep, but having instances of Typeable at for-all types, which is an entirely un-explored area. Let's leave that aside.] So for data constructors you have type variables in their types, and I think you' have to represent that explicitly, which would in turn mean that a TypeRep is not simply an application of a TyCon. (I'm not keen on "tycoon tyvars", which I guess is your fix. The water seems deeper here. Are you sure you could not get away without representations of the constructor argument types? After all, SYB has already shown how to write serialisers and deserialisers. Simon | -----Original Message----- | From: libraries-bounces(a)haskell.org [mailto:libraries-bounces@haskell.org] On Behalf Of Krasimir | Angelov | Sent: 30 March 2006 12:30 | To: Haskell Libraries | Subject: Reflection API for Haskell. | | Hello Guys, | | I am interested in having better Reflection API for Haskell. Currently | we have Typeable and Data classes which provide some pieces of | information about the data types at runtime. The (typeOf :: Typeable a | => a -> TypeRep) method provides runtime information about the type of | a given variable. The (dataTypeOf :: Data a => a -> DataType) provides | almost the same information but with some extras. There is some | overlap between the TypeRep and DataType types. Some pieces of | information you can get from the TypeRep, other from the DataType and | some other from both of them. There is also an information which is | inaccessible from either TypeRep and DataType. Here is a list of the | differences: | | - TypeRep contains both the TyCon and the arguments given to it. | DataType contains only the name of the TyCon. The TyCon name is | duplicated in TypeRep and in DataType. | - DataType contains the DataRep structure which gives us an | information about the available data structures. This is the only way | to access them. | - The Constr type represents data constructors at runtime. It has | information about the constructor name, the record selectors and the | fixity (prefix/infix). Unfortunately there isn't any information about | the types of its arguments. | | What I would like to see is a better Reflection API with clear | interface. My proposal is: | | -- * Type representations | TypeRep -- abstract, instance of: Eq, Show, Typeable | TyCon -- abstract, instance of: Eq, Show, Typeable | DataCon -- abstract, instance of: Eq, Show, Typeable | | -- * Construction of type representations | mkTyCon :: String -> Arity -> [DataCon] -> Fixity -> TyCon | mkTyConVar :: Int -> Arity -> TyCon | mkTyVar :: Int -> TypeRep | mkTyConApp :: TyCon -> [TypeRep] -> TypeRep | mkDataCon :: String -> [TypeRep] -> TypeRep -> [String] -> Fixity -> DataCon | | -- * Fixity representation | Fixity(..) -- instance of: Eq, Show, Typeable | defaultFixity :: Fixity | | -- * Observation of type representations | typeRepTyCon :: TypeRep -> TyCon | typeRepArgs :: TypeRep -> [TypeRep] | | -- * Observation of type constructors | tyConString :: TyCon -> String | tyConArity :: TyCon -> Int | tyConDataCons :: TypeCon -> [DataCon] | tyConFixity :: TyCon -> Fixity | | -- * Observation of data constructors | dataConString :: DataCon -> String | dataConArgs :: DataCon -> [TypeRep] | dataConResult :: DataCon -> TypeRep | dataConFields :: DataCon -> [String] | dataConFixity :: DataCon -> Fixity | | The above API is just a reorganization of the existing API but with | some extensions: | | - In the existing API there is no way to get the TyCon fixity and | arity. In the new one there are the tyConFixity and tyConArity | functions. | - The data constructors are available from the tyConDataCons function. | There isn't need for the separated DataType and DataRep types. | - The DataCon arguments and the result type are available from the | dataConArgs and dataConResult functions. | - Since the type can be polymorphic we need to have representation for | type variables. For that reason there are the mkTyVar and mkTyConVar | functions. | | The new API can be used for both the Dynamic data type implementation | and for the generic programming API. The additional information can be | used for many kinds of serialization libraries where you would like to | be sure that you are deserialising data with the same structure as | those used in your program. In addition if you have stored the data | type meta information you can have many kinds of generic tools that | can read/write the data without need to have the same types defined in | the application. | | What do you think about this kind of generalization of the existing | API in the standard package? | | Cheers, | Krasimir | _______________________________________________ | Libraries mailing list | Libraries(a)haskell.org | http://www.haskell.org/mailman/listinfo/libraries

3 2

Non-exhaustive pattern in Socket.hsc
by Dominic Steinitz 07 May '06

07 May '06

I'm getting [root@tility pcap]# ./test test: Socket.hsc:(1434,17)-(1523,20): Non-exhaustive patterns in case The problem is caused because pcap_findalldevs is returning an address family value of 17 #define PF_PACKET 17 /* Packet family. */ #define AF_PACKET PF_PACKET and the ghc Family datatype doesn't support it :-( data Family = AF_UNSPEC | AF_UNIX | AF_INET | AF_INET6 | AF_SNA | AF_DECnet | AF_APPLETALK | AF_ROUTE | AF_X25 | AF_AX25 | AF_IPX -- Imported from Network.Socket *Main> map packFamily [AF_UNSPEC,AF_UNIX,AF_INET,AF_INET6,AF_SNA,AF_DECnet,AF_APPLETALK,AF_ROUTE,AF_X25,AF_AX25,AF_IPX] [0,1,2,10,22,12,5,16,9,3,4] I think I have two choices: 1. Amend ghc. I haven't built ghc for a long time. Would I have to build the whole of ghc? I presume I would have to because of dependencies. 2. Use Network.Alt since http://www.cs.helsinki.fi/u/ekarttun/network-alt/doc/Network.Alt.Types.html defines type Family = Int Any advice? What is the long term plan for Network anyway? Thanks, Dominic.

6 6

Unsafe Functions
by Ashley Yakeley 05 May '06

05 May '06

I would like to start a discussion on the role of unsafe functions in Haskell: unsafePerformIO :: IO a -> a unsafeInterleaveIO :: IO a -> IO a unsafeInterleaveST :: ST s a -> ST s a unsafeIOToST :: IO a -> ST s a unsafeIOToSTM :: IO a -> STM a unsafeFreeze, unsafeThaw, unsafePreservingMatrix, unsafeRenderPrimitive perhaps also unsafeForeignPtrToPtr :: ForeignPtr a -> Ptr a (which is already under Foreign.*) hGetContents :: Handle -> IO String (which is lazy rather than unsafe per se) * Do you use these, and what for? * Is there safe functionality that can currently only be obtained with them? * Do you think they should be standardised, and how? I'm thinking the unsafe functions should be moved from System.IO.Unsafe and elsewhere to Unsafe, similar to Foreign.*, to better separate them from "real Haskell" conceptually. Also, I would add: unsafeCoerce :: a -> b -- Ashley Yakeley, Seattle WA WWEWDD? http://www.cs.utexas.edu/users/EWD/

3 5

ANNOUNCE: Crypto-3.0.3
by Dominic Steinitz 29 Apr '06

29 Apr '06

I should like to announce a new version of the Haskell Cryptography Library. Version 3.0.3 supports: • DES • Blowfish • AES • Cipher Block Chaining (CBC) • PKCS#5 and nulls padding • SHA-1 • MD5 • RSA • OAEP-based encryption (Bellare-Rogaway) • PKCS#1v1.5 signature scheme • ASN.1 • PKCS#8 • X.509 Identity Certificates • X.509 Attribute Certificates It also includes Base64, a hexdump function, support for Word128, Word192, Word256 (key sizes for AES) and beyond. The ASN.1 handling has been re-written. Details on how to use it are contained in the user guide http://www.haskell.org/crypto/ReadMe.pdf. Haddock documentation is available at http://www.haskell.org/crypto/doc/html. See the home page (http://www.haskell.org/crypto) for downloading. Dominic.

2 2

[RFC] Faster implementation of Integer to string conversion.
by Bertram Felgenhauer 28 Apr '06

28 Apr '06

Hi, several people have noted that printing integers is slow. The attached patch implements a faster version of the jtos (read: positive integer to string) function in GHC's Num library. The algorithm is a divide and conquer algorithm, that converts numbers to base b by converting them to base b^2 first, and then splitting the digits. Notes: - I've tested the speed on my computer (which has an Athlon XP processor) and it seems to be of comparable speed as the original for small numbers and way faster for big numbers (read: a few hundred or thousand digits). - The changed conversion code is a much better list producer than the original one, which generates digits starting with the last. - There's similar code in Numeric.lhs, that deals with arbitrary bases. The current patch does not deal with this. (see questions below) - musabi on #haskell mentioned that there's interest in replacing GMP as the bignum implementation. This is independent of changing this string conversion, as far as I can see. Questions: - Is this really GHC specific? - Unfortunately I see no nice way to share code between the Numeric and Num modules. Does anyone have an idea how to do this without sacrificing performance for the common base 10 case? regards, Bertram

2 2

FPS/Data.ByteString candidate
by dons＠cse.unsw.edu.au 28 Apr '06

28 Apr '06

Following discussion, I've tagged FPS 0.4, a candidate for the base library. Changes: * Renamed to Data.ByteString(ByteString) * Improved documentation * Tweaks to build under ghc 6.6 * Added: getLine, getContents, putStr, putStrLn, zip, unzip, zipWith * Much faster: elemIndices, lineIndices, split, replicate * More automagic benchmarks and QuickCheck tests. As usual, code is here: http://www.cse.unsw.edu.au/~dons/fps.html -- Don

10 41

implementation of UTF-8 conversion for text I/O: iconv vs hand-made
by Bulat Ziganshin 26 Apr '06

26 Apr '06

Hello all this letter describes why i think that using hand-made (de)coder for support of UTF-8 encoded files is better than using iconv. to let other readers know, iconv is wide-spread C library that performs buffer-to-buffer conversion between any text encodings (utf-8, utf-16, latin-1, ucs-2, ucs-4 and more). hand-made (en)coder implemented by me is just "converter", i.e. high-order function, between the getByte/putByte and getChar/putChar operations. so it can be used in any monad and with any purposes, not only for text I/O one can find example of library that uses iconv in the "System\IO\Text.hs" module from http://haskell.org/~simonmar/new-io.tar.gz and example of hand-made encoder in module "Data\CharEncoding.hs" and its usage - in "System\Stream\Transformer\CharEncoding.hs" from http://freearc.narod.ru/Streams.tar.gz i crossposted this letter to Marcin and Simon because you have discussed with me this question and to Einar because he once asked me about one specific feature in this area. why iconv is better: 1) it's lightning fast, making virtually zero speed overhead 2) it's robust 3) it contains already implemented and debugged algorithms for all possible encodings we can encounter 4) it has highly developed error processing facilities (i mean signalling about errors in input data and/or masking them) why hand-made conversion is better: 1) i don't know whether iconv will be available on every Hugs and GHC installation? 2) Einar once asked me about changing the encoding on the fly, that is needed for some HTML processing. it is also possible that some program will need to intersperse text I/O with buffer/array/byte/bits I/O. it's a sort of things that are absolutely impossible with iconv 3) my library support Streams that works in ANY monad (not only IO, ST and their derivatives). it's impossible to implement iconv conversion for such stream types as you can see, while the last arguments says about very specific situations, these situations absolutely can't be handled by iconv, so we need to implement hand-made conversions anyway. on the other side, iconv strong points don't have principal meaning - the speed with hand-made routines will be enough, about several mb/s; all possible encodings can be implemented and debugged sooner or later; only processing of errors in input data is weak point of the current design itself moreover, there are implementation issues that make me more enthusiastic about hand-made solution. it just already implemented and really works. implementation of the CharEncoding for streams is in module "System\Stream\Transformer\CharEncoding.hs", which is very trivial. implementation of different encoders in "Data\CharEncoding.hs" is slightly more complex, but these routines also used in "instance Binary String", i.e. to serialize strings. also, i think that "Data\CharEncoding.hs" module should be a part of standard Haskell library, so implementation of CharEncoding stream transformer is almost "free" on the other side, implementation of text encoding in "new I/O" library is about 1000 lines long. while i don't need to copy them all, using iconv anyway will be much more complex than using hand-made routines. this include complexity of interaction with iconv itself and complexity of implementing various I/O operations over the buffer that contains 4-byte characters. i already implemented 3 buffering transformers and adding one more buffering scheme is the last thing i want to do. vice versa - now i'm searching for ways to omit repetitions of code by joining them all into one. it's very boring - to have 3 or 4 similar things and replicate every change to them all at the same time, the library design is open and it's entirely possible to have two alternative char encoding transformers. everyone can develop additional transformers even without interaction with me - in this case, it should just implement vGetChar/bPutChar operations via the vGetBuf/vPutBuf ones. i just propose to leave the things as they are, and go to implementing of iconv-based transformer only when we will be actually bothered by it's restrictions -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com

4 4

Cabal feature request - dates in versions
by Einar Karttunen 24 Apr '06

24 Apr '06

Hello When developing a library I would like to have cabal files with something like: Version: 0.9.<date> where Cabal would automatically generate the exact version based on the current date (e.g. 0.9.20060422). This would be very useful for development versions of libraries. - Einar Karttunen

2 3

libpcap Binding & Cabal Question
by Dominic Steinitz 23 Apr '06

23 Apr '06

I am trying to build the binding for libpcap. http://www.haskell.org/pipermail/libraries/2004-June/002285.html http://www.haskell.org//pipermail/libraries/attachments/20040630/4469b357/P… Unfortunately when I run hsc2hs I get errors. [dom@tility pcap]$ hsc2hs Pcap.hs Pcap.hs:121:21: net/bpf.h: No such file or directory Pcap.hs: In function `main': Pcap.hs:461: error: structure has no member named `sa_len' The first one is easily fixed. The version of libpcap I downloaded from www.tcpdump.org (LIBPCAP 0.9.4) contains pcap-bpf.h not net/bpf.h (so I replaced #inlcude <net/bpf.h> with #include <pcap-bpf.h>). The second one is a result because in some systems socket addresses are fixed-length and sizeof (struct sockaddr) gives the size of the structure but in other systems the socket address is variable length and there's an sa_len field giving the length of the structure. The libpcap authors handle this by using configure to see if the sa_len field exists. If it does then #define SA_LEN(addr) (get_sa_len(addr)) If does not then (actually it's more complicated than this but I don't want to clutter up an already cluttered email) #define SA_LEN(addr) (sizeof (struct sockaddr)) Finally to my questions. 1. How do I get Cabal to check for the existence of sa_len? I'm guessing I follow the instructions in "2.2. System-dependent parameters" but I have never used autoconf in my life so if there were another easier way I'd be happy to hear about it. 2. How do I tell Cabal I need LIBPCAP 0.9.4? Dominic.

2 4