- Libraries - Haskell.org

XML and RDF parsers
by Graham Klyne 14 Jul '04

14 Jul '04

Further to my work on HaXml announced last month [1][2][3][4], I've added two new filters to perform xml:base and xml:lang processing, and fixed up some minor problems. The code is available at [5], with browseable directories at [6]. I've also developed an RDF/XML parser as part of my Swish project [11] (the RDF/XML parser can be used separately from all the other Swish components); the code is at [7], with browseable online directories at [8]. All this has a dependency on some revised network modules [9][10]. ... As part of developing the RDF/XML parser, I believe I have identified a fairly significant problem with HaXml's handling of whitespace. The core fix is, I believe, quite easy, but there are knock-on changes required to the validation module and 20-30 of the test suite test cases. The code changes could cause some existing HaXml applications to break because they would result in additional content items in the resulting XML data containing just whitespace text -- e.g. whitespace appearing between elements would be returned to the application. #g -- [1] http://www.haskell.org//pipermail/libraries/2004-June/002243.html [2] http://www.haskell.org//pipermail/libraries/2004-June/002266.html [3] http://www.haskell.org//pipermail/libraries/2004-June/002276.html [4] http://www.haskell.org//pipermail/libraries/2004-June/002282.html [5] http://www.ninebynine.org/Software/HaskellUtils/20040713-Haxml-1.12.zip [6] http://www.ninebynine.org/Software/HaskellUtils/HaXml-1.12/ [7] http://www.ninebynine.org/Software/HaskellRDF/20040713-HaskellRDF.zip http://www.ninebynine.org/Software/HaskellUtils/20040713-HaskellUtils.zip [8] http://www.ninebynine.org/Software/HaskellRDF/ http://www.ninebynine.org/Software/HaskellUtils/ [9] http://www.ninebynine.org/Software/HaskellUtils/20040609-Network.zip [10] http://www.ninebynine.org/Software/HaskellUtils/Network/ [11] http://www.ninebynine.org/RDFNotes/Swish/Intro.html ------------ Graham Klyne For email: http://www.ninebynine.org/#Contact

1 0

Use of cpp
by Adrian Hey 14 Jul '04

14 Jul '04

Hello, I've just modified some Haskell code from the AVL library to explicitly use ghc unboxed Ints in some routines. Unfortunately, I've broken portability by doing this and was considering some cpp magic to restore portability. But IIRC, the use of cpp is discouraged for Haskell library infrastructure. Is that so? If so, what should I use instead? Is cpphs the way to go? Thanks -- Adrian Hey

5 7

RE: System.Process
by Simon Marlow 08 Jul '04

08 Jul '04

On 07 July 2004 22:59, Peter Simons wrote: > I was wondering what the state of the System.Process module > described on > > http://haskell.org/hawiki/ExternalCommands > > is? > > I have tried to use the various POpen-like implementations > out there, but _none_ of them works with the most current > GHC. Is there any way to communicate with external programs > at the moment? Posix-only would be suffice at the moment. I posted code with POSIX & Windows versions recently: http://www.haskell.org//pipermail/libraries/2004-June/002265.html It's nearly finished, I hope to get it into GHC 6.4. You need to compile with -threaded in order to get a non-blocking waitForProcess. Cheers, Simon

2 1

Subprocesses (was: [ANNOUNCE] shell-haskell 1.0.0)
by Simon Marlow 08 Jul '04

08 Jul '04

Libraries people, There are currently two ways to create and communicate with external processes: - System.Cmd.system (very basic, doesn't give you access to the process's output but you can use shell primitives to redirect its input/output). - Posix.runProcess (can specify Handles for the process's input/output, but it doesn't create pipes so you can't actually communicate with the subprocess using these Handles.). - POpen.popen (can specify an input String, and gives you lazy output Strings. No interactive communication is possible.) Ok, *three* ways. System.Cmd.system, Posix.runProcess, POpen.popen, and shell-haskell. Er. *Four* ways. David Sankel's shell-haskell creates pipes for stdin/stdout/stderr, and lets you communicate interactively with the sub-process via Handles. It could do with a way to specify the environment and working directory of the sub-process (ala POpen and runProcess), but apart from that it seems to be the most flexible of the four. It could be simplified slightly by using one Handle rather than two for stdin/stdout. We should think about what functionality we want to provide in the core libraries. My preference would be to go for a full version with three pipes (like shell-haskell) and perhaps one or two more specialised versions covering common cases (I don't know what these should be, though). What do people think? Cheers, Simon -----Original Message----- From: haskell-admin(a)haskell.org [mailto:haskell-admin@haskell.org] On Behalf Of David Sankel Sent: 06 May 2003 03:06 To: haskell(a)haskell.org Subject: [ANNOUNCE] shell-haskell 1.0.0 =================== shell-haskell 1.0.0 =================== shell-haskell provides utilities to communicate with other process's via Haskell code. This library can be used for something as simple as getting the output of another program or as complex as interacting with an interpreter. see http://www.electronconsulting.com/shell-haskell for details. David J. Sankel _______________________________________________ Haskell mailing list Haskell(a)haskell.org http://www.haskell.org/mailman/listinfo/haskell

5 10

RE: DData, again
by Simon Marlow 05 Jul '04

05 Jul '04

On 02 July 2004 21:28, Alastair Reid wrote: >> i would prefer that the library had an independent distribution, >> and can be easily downloaded and experimented with. > > Perhaps it would be a good candidate for the new packaging > infrastructure? > > btw Note that distributing it independently doesn't mean that it > can't live in the main cvs.haskell.org cvs repository. (The converse > doesn't appear to be true: if you want it to become part of the > standard distribs, it probably has to live in cvs.haskell.org just to > keep it easy for the distrib builders who tend to like to be able to > create branches, tag the repository when they make a release, etc.) As far as the main libraries repository goes, I'd like to split this into two stages: 1. Bring in Map, Set, IntMap, and IntSet. I think we're all agreed that these are the right thing. 2. Resolve the outstanding issues regarding overloaded sequences. I find Ross's arguments in favour of a Sequence class fairly persuasive, but I haven't heard many other comments on the subject. Cheers, Simon

1 0

DData
by Christian Maeder 02 Jul '04

02 Jul '04

Hi, may I warm up an old subject? I would appreciate if at least the Map and the Set module could be added to the base package, soon! DData.Set and the current Data.Set can be merged together (as Sven Panne mentioned earlier). Data.FiniteMap and Data.Map should co-exist for a while. The current Data.FiniteMap and Data.Set have various missing functions and instances. It would make sense to add Data.Bag also (but I don't use Bags currently) (I also don't care about Seq, Queue and Int-specializations.) The last proposal was made by JP Bernardy: http://users.skynet.be/jyp/DData/doc Daan wanted to look into it. My experiments (with large maps) wrt. performance (and memory consumption with and without optimization) of both Map and FiniteMap implementations revealed differences, but no crucial ones. "FiniteMap Int a" performs better than "Map Int a", but not as good as "IntMap a", but the differences are unimportant for us. (We would need a pure HashMap with O(1) lookup while extending the Map) Christian

4 3

binding for libpcap
by Gregory Wright 30 Jun '04

30 Jun '04

Hi, Attached is a binding for libpcap, the packet capture library. I'm looking for comments on how it might be improved, or any particular infelicities that you notice. As I don't use hugs regularly anymore, perhaps someone can point me to a good example of a network library binding that works on both ghc and hugs. It' s tested on FreeBSD (5.2.1-current) but should work with any *nix that can build libpcap. It does not work with Winpcap, a similar library for windows, although it shouldn't be too hard to extend it to do so. The marshalling of packet data is purposely very lightweight (data is returned as Ptr Word8). Having written a lot of embedded software for network test equipment, I prefer the flexibility of being able to marshal into IO-mutable arrays if necessary. But I don't want to impose that on someone who would be satisfied by [Word8]. Best Wishes, Greg Gregory Wright Antiope Associate LLC gwright(a)antiope.com

1 0

Re: [Haskell] Reading a directory tree
by Isaac Jones 29 Jun '04

29 Jun '04

[Followups to libraries(a)haskell.org] This thread makes me want to mention that I'm starting to put together a little library of useful path-based functions (I started it based on OS.Path in Python). Not sure how to organize it, exactly, since some things that might make sense here are already in System.Directory (and some things in there don't make sense in there) and some things may require Posix, and so should be split off into a separate module. My basic requirement is that they should be functions useful for system administration. We've sucked in some functions in from GHC and from Cabal[1]. It would be nice if we could just have a dumping ground for related functions that people have written. For now, if you have darcs[2], you can say: darcs get http://www.syntaxpolice.org/darcs_repos/OS.Path/ (And if you don't have darcs, you can browse around inside there with a web browser to get an idea of how it's shaping up.) That'll download the tree. You can make changes and additions, then use "darcs send" to email the changes to me. I think we should just dump together a bunch of these functions and worry about organizing it later (wiki-style). Here's a function[3] I wrote that's similar to what's mentioned here. It builds a tree out of a file directory. I'd like a function like "walk"[4] that walks this tree and executes an IO action (like printing out all the .txt files or something). It doesn't have to be based on Tree. peace, isaac ------------------------------------------------------------ [1] http://www.haskell.org/cabal [2] http://abridgegame.org/darcs/ [3] -- |Create a tree out of the given starting point. If the starting -- point is a directory, we recurse down and give the entire -- sub-structure, otherwise, we return a single node. -- I think it only requires Posix to test whether this is a directory, -- maybe we can do this with System.Directory? makeDirectoryTree :: FilePath -- ^Starting point (file or directory) -> IO (Tree (FilePath, Bool)) makeDirectoryTree path' = makeDirTree' Nothing path' where makeDirTree' :: Maybe FilePath -- ^Parent Dir -> FilePath -- ^Starting point -> IO (Tree (FilePath, Bool)) makeDirTree' parentIn pathIn = do let fullPath = case parentIn of Just pi' -> (dropLast pi' pathSeparator) ++ [pathSeparator] ++ pathIn Nothing -> pathIn isDir <- pathIsDirectory fullPath contents <- if isDir then getDirectoryContents fullPath else return [] subForest <- mapM (makeDirTree' (Just fullPath)) [c | c <- contents, c /= ".", c /= ['.', pathSeparator] , c /= ['.', '.', pathSeparator], c /= ".."] return $ Node (pathIn, isDir) subForest -- FIX: probably better not to use "error" here, but rather let exception occur. pathIsDirectory :: FilePath -> IO Bool pathIsDirectory p = do existsP <- doesFileExist p existsP2 <- doesDirectoryExist p when (not (existsP || existsP2)) (error $ "File does not exist: " ++ show p) status <- getFileStatus p return $ isDirectory status dropLast [] _ = [] dropLast (h:[]) toDrop | h == toDrop = [] | otherwise = [h] dropLast (h:t) toDrop = h:(dropLast t toDrop) [4] -- |Apply the given function in each directory starting at the -- root. use 'makeDirectoryTree'? FIX: how do we handle symlinks? walk :: FilePath -- ^root -> (FilePath -- current directory -> [FilePath] -- list of files in the directory -> IO a) -- ^visit function (current dir, list of files) -> IO (Tree a)

3 2

HaXml, further updates
by Graham Klyne 28 Jun '04

28 Jun '04

I've done some cleaning up on the code (removing all the old parameter entity handling logic in the parser), and fixed a serious known bug in the general entity substitution filter (recursive GE definitions would blow the stack). The GE substitution logic has been reorganized to separate the various phases. Some new test cases have been added to the test suite. The current code is at: http://www.ninebynine.org/Software/HaskellUtils/20040628-Haxml-1.12.zip http://www.ninebynine.org/Software/HaskellUtils/HaXml-1.12/ There's plenty more that I could do (e.g. see the TODO.txt), but I plan to turn my attention to other areas for the foreseeable future. #g ------------ Graham Klyne For email: http://www.ninebynine.org/#Contact

1 0

DNS and E-Mail library
by Peter Simons 25 Jun '04

25 Jun '04

Hi, I have packaged up two libraries of mine in the hope that someone will find them useful. Here they are: HsDNS: http://cryp.to/hsdns/ An asynchronous DNS resolver based on the GNU Adns library. HsEMail: http://cryp.to/hsemail/ A collection of Parsec combinators, which implement the grammars defined in RFC2821 and RFC2822 (SMTP and Internet Messag Format). The web pages and release archives now contain reference documentation courtesy of Haddock. Peter P. S.: Who would I need to contact to get these links up an the haskell.org/libraries/ page?

1 0