- Libraries - Haskell.org

Packages and .NET assemblies
by Simon Marlow 12 Jul '06

12 Jul '06

I just took a deeper look at how .NET handles packages, since they have thought through many of the same issues and have a scheme that scales well. There's a great deal of similarity with the schemes we're thinking about for Haskell. The executive summary: - A .NET assembly fulfills the same function as a Haskell package: an assembly is the unit of versioning and distribution, and carries with it a bunch of metadata (author, copyright, etc.). - An assembly is uniquely identified by a triple (name,version,culture). - An assembly travels in a single file (.DLL) - An assembly may define multiple namespaces (like A.B.C), and entities within namespaces. Much like modules in Haskell. - Assemblies are not named directly in source code. C# code (for example) refers to namespaces and entities only. - When compiling C# code, you name the DLLs that contain assemblies to bring into scope (in fact, the transitive closure) using command-line options. - The C# compiler brings a set of assemblies into scope by default (== exposed packages in GHC). - The binary has baked into it the exact identity of the assemblies it was compiled against. - At runtime, the binary might be rebound to different versions of the assemblies it depends on, via a complicated structure of configuration files. So there's no grafting, or even assembly-qualified imports. How do they get away with this? Well, one reason is that namespaces are generally globally unique, because the convention is to use names beginning with an organisation (eg. Microsoft.System). So when you bring a bunch of assemblies into scope, their namespaces generally don't conflict. Interestingly, the assembly story is almost identical to what GHC supports right now, and when we change module identities to include the package name (which I'm working on now), it will be even closer. Not that I necessarily think we can get away with doing nothing to the language, but at lesat this seems to argue for being conservative. Cheers, Simon

2 1

updates to HaXml (1.13.1 and 1.16)
by Malcolm Wallace 12 Jul '06

12 Jul '06

For those who use the current stable version of HaXml, I'd like to announce a new patch-level release, 1.13.1, which contains the following bugfixes: * permit percent character in attribute values * parse unquoted attribute values starting '+' or '#' in HTML * keep the original DTD in the output of 'processXmlWith' See http://www.haskell.org/HaXml/ for downloads. For those living on the development edge, I'd like to report that the current darcs version darcs get http://www.cs.york.ac.uk/fp/darcs/HaXml contains a new set of parser combinators (with the same API as before) that is lazier, whilst still allowing backtracking. By lazy, I mean it can start to return partial values as soon as it has consumed e.g. the start tag of an element, without waiting to check that the close tag matches. This has two good effects: * your program will run faster * it will consume less memory and two bad effects: * if there are errors in the document, they will throw an exception in the middle of your processing * the error message in the exception may be rather less accurate about the cause and location than previously. The older XML parser has also been retained, since the lazy version is still experimental. To use the new one, import Text.XML.HaXml.ParseLazy There are also lazy versions of the usual demo programs CanonicaliseLazy XtractLazy As an example of the improved speed, a query to extract all the <key> tags from a 3.7Mb XML document: Xtract "//key" file.xml did not give any results after more than ten minutes on my machine, but XtractLazy "//key" file.xml started producing results immediately, and completed the task in 25 seconds (returning 52584 tags). Separate website and downloads at http://www.cs.york.ac.uk/fp/HaXml-devel Regards, Malcolm

3 4

Re: Packages and modules
by Malcolm Wallace 11 Jul '06

11 Jul '06

John Meacham <john(a)repetae.net> wrote: > Package names should never appear in source files IMHO. if a package > name is in the source file, then you might as well make it part of the > module name. Packages exist for 'meta-organization' of code. A way to > deal with mapping code _outside_ of the language itself, putting > packages inside the code will force the creation of yet another level, > meta-packages, or something. packages should not be a language issue, > they are an environment one. I tend to the opposite view. The meaning of the code should be expressed in the code itself. If a module M imports A.B.C, and I can see two such modules called A.B.C, then the meaning of the code is ambiguous and ill-defined. I would rather not have to look elsewhere (in the build system? Makefile? scons? Cabal file? DOS batch file? where?) to find out how to resolve the ambiguity. Surely the programmer knew which import was intended. Is it so difficult to communicate that information somewhere close to the import itself? This is the same situation as with compiler flags for language extensions, foreign imports, and so on. If a module uses a non-standard language feature that changes the interpretation of the syntax (e.g. template haskell), it should say so in the module itself {-# OPTIONS_GHC -fth #-}, so that I as a human reader know what is going on. Likewise, if a foreign import requires a C header file, that should also be referred to in the FFI decl, not solely as a build option at some arbitrary distance in the filesystem. Regards, Malcolm

4 18

Re: Why is there no splitBy in the list module?
by Christian Maeder 11 Jul '06

11 Jul '06

Simon Marlow schrieb: > Donald Bruce Stewart wrote: > >> Hacking up your own custom split (or a tokens/splitOnGlue) must be one >> of the most common questions from beginners on the irc channel. >> >> Anyone rememeber what the result of the "let's get split into the base >> library" movement's work was? > > ISTR there wasn't a concensus, so nothing happened. Which is silly, > really - I agree we should definitely have a Data.List.split. > Maybe someone can extract a result from the old discussion about "Prelude function suggestions" http://thread.gmane.org/gmane.comp.lang.haskell.libraries/1684/focus=1684 Cheers Christian Our current (special) version is: {- | A function inspired by the perl function split. A list is splitted on a seperator element in smaller non-empty lists. The seperator element is dropped from the resulting list. -} splitOn :: Eq a => a -- ^ seperator -> [a] -- ^ list to split -> [[a]] splitOn x xs = let (l, r) = break (==x) xs in (if null l then [] else [l]) ++ (if null r then [] else splitOn x $ tail r)

3 3

RE: Packages and modules
by Simon Peyton-Jones 05 Jul '06

05 Jul '06

| So instead of just taking this simple solution, the wiki proposal is instead | destroying the beauty of the per-package namespace idea by incorporating | into it the existing shared namespaces with their attendant problems, | instead of just letting the existing messy system die a natural death | through the syntactic isolation I proposed. Brian, I think your proposal may be clearer to you than to everyone else. It's always hard to reconstruct a detailed proposal by reading long email threads. Suggestion: if you feel strongly about this, why not start a Wiki page (you can link to it from the current one) to describe the design you propose, at a comparable level of detail? Incidentally, compatibility with Cabal is a significant goal. Simon

1 0

RE: Packages and modules
by Simon Peyton-Jones 05 Jul '06

05 Jul '06

In response to Brian and Ian's helpful comments, I've added a bunch more stuff to our proposal about packages. If I have missed anything, let me know. http://hackage.haskell.org/trac/ghc/wiki/GhcPackages If you or anyone else thinks the choices made there are poor ones, continue to say so. It's a helpful process. One particular point: | 2b) Exporting the contents of an external module qualified | | module Foo | ( qualified module P | ) where While this might be a good idea, it's an orthogonal one, and the proposal doesn't tackle it. One thing at a time! Simon

1 0

QDBM / Hyper Estraier bindings for haskell
by Jun Mukai 28 Jun '06

28 Jun '06

Hello all, I pleased to announce the QDBM and Hyper Estraier bindings for haskell. If you are interested in, they can be obtained from http://www.city5.org/haskellprog/qdbm/ http://www.city5.org/haskellprog/hstraier/ These are darcs repository, so you can obtain them via darcs. These bindings are already cabalized, you have to type only as follows to install them: % runghc ./Setup.hs configure % runghc ./Setup.hs build # runghc ./Setup.hs install the online API documentation generated by haddock is also shown at http://www.city5.org/doc/qdbm/ http://www.city5.org/doc/hstraier/ * What is QDBM? QDBM, stands for Quick DBM, is a database module similar to GDBM or NDBM or Berkeley-DB, but the specific feature is performance and simple API. By now, my QDBM binding supports only Depot, Curia, Villa, and a part of Cabin. QDBM can handle several kinds of DBM, and Depot and Curia and Villa is a name of each DBM. Depot is a basic DBM. Curia is a `large' DBM and bundles many Depot files for its internal implementation. Curia also can store `large objects' as a file. Villa is B+Tree DBM and uses a function to compare keys. Villa also has faculty of cursors and transactions. Vista, `large' version of Villa, is not supported. Odeum, inverted index, is also not supported. Cabin is auxiliary functions collection module and contains a kind of utility functions of string operation such as Base64 encode/decode, URL encode/decode, quoted printable encode/decode, mime analysis, csv reading and so on. These functions will be useful for many Haskellers, and run faster. If you are interested in QDBM itself, see http://qdbm.sourceforge.net/ * What is Hyper Estraier Hyper Estraier is a full-text search system using QDBM. You can search documents according to the keywords specified and so on. My Hyper Estraier binding supports only `core' API. Core API can handle document object, conditionals, and full-text search DB. Hyper Estraier itself has `node' API, which invokes search queries to remote servers using HTTP. Node API is not supported yet in my binding now. If you are interested in Hyper Estraier itself, see http://hyperestraier.sourceforge.net/ Any comments are welcome. Thanks, Jun Mukai mukai(a)jmuk.org

2 2

Streams 0.2 - beta testing started. Good replacement for NewBinary and System.IO?
by Bulat Ziganshin 27 Jun '06

27 Jun '06

Hello all i'm going to beta testing of Streams 0.2 library. compared to previous version, it works several times faster, supports memory-mapped files, has more orthogonal API and numerous internal changes. moreover, it can be a great replacement for NewBinary and System.IO modules (more on this below) === For users of Streams 0.1 library === new version is not yet documented, and that is not big problem, because at this moment i'm mostly interested in testing it as replacement of previous library version. if you already use Streams, please check that with new version everything is still ok comparing to Streams 0.1, there a few API changes: - with* stream transformers renamed to add*: withEncoding should be changed to addEncoding withLocking -> addLocking bufferBlockStream -> "addBuffering >>= addInlining" bufferMemoryStream -> addInlining - with* functions now has more regular names: withEncoding' -> withEncoding withLocking' -> withLocking withBuffering -> "withBuffering $ \h -> withInlining h" withMemBuffering -> withBuffering - in binary i/o, upper-case LE/BE suffixes now used in all functions: putWord16be -> putWord16BE getWord64le -> getWord64LE and so on - custom serialization routines for arrays, lists and strings now has more regular names, look at comment titled "Serialization for arrays of different types" in Data.AltBinary.Custom.Array module - Stream class now split into the Stream, BlockStream, MemoryStream, ByteStream and TextStream classes; MemoryStream interface is slightly changed if i missed something that breaks backward compatibility - please let me know. and please, find the time and try new library version with your code before the library will be released === As replacement for NewBinary library and various Binary modules === Streams 0.2 includes full emulation of NewBinary library API. If you already use one of these modules in your program, just replace "import NewBinary.Binary" with "import Data.AltBinary" and enjoy! all the differences i know is: - argument order in "putBits" are different, my order is more convenient for partial application - format of written data is incompatible between libraries. on the other side, data format used in my lib is CPU-independent, so you can write data on 32-bit CPU and then read them back on 64-bit one One of most pleasing features of this emulation is that it works from several to several tens times faster than original :) === As (particular) replacement for System.IO module === If you are interested in getting several times speed improvement, you can try to replace "import System.IO" in your program with "import System.Stream.IO" depending on how you are lucky, this may or may not work. Currently this emulation don't supports LineBuffering and NoBuffering, and several functions (hLookAhead, hTellPos...). Support for stdin/stdout/stderr and appropriate functions are also disabled due to lack of LineBuffering support. if you really need to use them, do it through the regular Streams interface (read Example/wc.hs for details) === Downloading and installation === To install the library, you first need to install ArrayRef library: Download: http://www.haskell.org/library/ArrayRef.tar.gz Installation: run "make install" - Optionally, you can also add support for ByteStrings: - Download FPS 0.7 from repository (it's not yet released): - darcs get --partial http://www.cse.unsw.edu.au/~dons/code/fps - Compile and install it: - runhaskell Setup.hs configure - runhaskell Setup.hs make - runhaskell Setup.hs install Download: http://www.haskell.org/library/StreamsBeta.tar.gz - If you installed FPS library, rename Streams.cabal.fps into the Streams.cabal Installation: run "make install" OLD version docs: http://haskell.org/haskellwiki/Library/Streams -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com

1 0

RE: Reflection API for Haskell.
by Simon Peyton-Jones 22 Jun '06

22 Jun '06

I'm all for this. But I think the design needs a bit of work. Firstly, remember that TypeReps at the moment are designed for *ground types*, that is types with no free type variables. A ground type always starts with a type constructor. So one can start with typeOf to get a TyCon. The TyCon might not be an algebraic data type; that's why the DataType type in Data.Generics.Basics has a DataRep thing. You could add that. (You'd want to add FunRep.) I think you could probably get rid of dataTypeOf entirely, incidentally. When it comes to the types of data constructors, they have *quantified* types, so you need a way to bind the type variables. I don't see that in your design. [This relates to another missing feature of the current design which is that it can't represent higher rank types, e..g typeOf f where f :: (forall a. a->a) -> Int But allowing this would required not only a higher-rank TypeRep, but having instances of Typeable at for-all types, which is an entirely un-explored area. Let's leave that aside.] So for data constructors you have type variables in their types, and I think you' have to represent that explicitly, which would in turn mean that a TypeRep is not simply an application of a TyCon. (I'm not keen on "tycoon tyvars", which I guess is your fix. The water seems deeper here. Are you sure you could not get away without representations of the constructor argument types? After all, SYB has already shown how to write serialisers and deserialisers. Simon | -----Original Message----- | From: libraries-bounces(a)haskell.org [mailto:libraries-bounces@haskell.org] On Behalf Of Krasimir | Angelov | Sent: 30 March 2006 12:30 | To: Haskell Libraries | Subject: Reflection API for Haskell. | | Hello Guys, | | I am interested in having better Reflection API for Haskell. Currently | we have Typeable and Data classes which provide some pieces of | information about the data types at runtime. The (typeOf :: Typeable a | => a -> TypeRep) method provides runtime information about the type of | a given variable. The (dataTypeOf :: Data a => a -> DataType) provides | almost the same information but with some extras. There is some | overlap between the TypeRep and DataType types. Some pieces of | information you can get from the TypeRep, other from the DataType and | some other from both of them. There is also an information which is | inaccessible from either TypeRep and DataType. Here is a list of the | differences: | | - TypeRep contains both the TyCon and the arguments given to it. | DataType contains only the name of the TyCon. The TyCon name is | duplicated in TypeRep and in DataType. | - DataType contains the DataRep structure which gives us an | information about the available data structures. This is the only way | to access them. | - The Constr type represents data constructors at runtime. It has | information about the constructor name, the record selectors and the | fixity (prefix/infix). Unfortunately there isn't any information about | the types of its arguments. | | What I would like to see is a better Reflection API with clear | interface. My proposal is: | | -- * Type representations | TypeRep -- abstract, instance of: Eq, Show, Typeable | TyCon -- abstract, instance of: Eq, Show, Typeable | DataCon -- abstract, instance of: Eq, Show, Typeable | | -- * Construction of type representations | mkTyCon :: String -> Arity -> [DataCon] -> Fixity -> TyCon | mkTyConVar :: Int -> Arity -> TyCon | mkTyVar :: Int -> TypeRep | mkTyConApp :: TyCon -> [TypeRep] -> TypeRep | mkDataCon :: String -> [TypeRep] -> TypeRep -> [String] -> Fixity -> DataCon | | -- * Fixity representation | Fixity(..) -- instance of: Eq, Show, Typeable | defaultFixity :: Fixity | | -- * Observation of type representations | typeRepTyCon :: TypeRep -> TyCon | typeRepArgs :: TypeRep -> [TypeRep] | | -- * Observation of type constructors | tyConString :: TyCon -> String | tyConArity :: TyCon -> Int | tyConDataCons :: TypeCon -> [DataCon] | tyConFixity :: TyCon -> Fixity | | -- * Observation of data constructors | dataConString :: DataCon -> String | dataConArgs :: DataCon -> [TypeRep] | dataConResult :: DataCon -> TypeRep | dataConFields :: DataCon -> [String] | dataConFixity :: DataCon -> Fixity | | The above API is just a reorganization of the existing API but with | some extensions: | | - In the existing API there is no way to get the TyCon fixity and | arity. In the new one there are the tyConFixity and tyConArity | functions. | - The data constructors are available from the tyConDataCons function. | There isn't need for the separated DataType and DataRep types. | - The DataCon arguments and the result type are available from the | dataConArgs and dataConResult functions. | - Since the type can be polymorphic we need to have representation for | type variables. For that reason there are the mkTyVar and mkTyConVar | functions. | | The new API can be used for both the Dynamic data type implementation | and for the generic programming API. The additional information can be | used for many kinds of serialization libraries where you would like to | be sure that you are deserialising data with the same structure as | those used in your program. In addition if you have stored the data | type meta information you can have many kinds of generic tools that | can read/write the data without need to have the same types defined in | the application. | | What do you think about this kind of generalization of the existing | API in the standard package? | | Cheers, | Krasimir | _______________________________________________ | Libraries mailing list | Libraries(a)haskell.org | http://www.haskell.org/mailman/listinfo/libraries

3 2

darcs patch: add Control.Monad.STM to package.conf.in
by Bertram Felgenhauer 20 Jun '06

20 Jun '06

Mon Jun 19 09:33:02 CEST 2006 Bertram Felgenhauer <int-e(a)gmx.de> * add Control.Monad.STM to package.conf.in

2 1