February 2004 - Libraries

Subprocesses (was: [ANNOUNCE] shell-haskell 1.0.0)
by Simon Marlow 08 Jul '04

08 Jul '04

Libraries people, There are currently two ways to create and communicate with external processes: - System.Cmd.system (very basic, doesn't give you access to the process's output but you can use shell primitives to redirect its input/output). - Posix.runProcess (can specify Handles for the process's input/output, but it doesn't create pipes so you can't actually communicate with the subprocess using these Handles.). - POpen.popen (can specify an input String, and gives you lazy output Strings. No interactive communication is possible.) Ok, *three* ways. System.Cmd.system, Posix.runProcess, POpen.popen, and shell-haskell. Er. *Four* ways. David Sankel's shell-haskell creates pipes for stdin/stdout/stderr, and lets you communicate interactively with the sub-process via Handles. It could do with a way to specify the environment and working directory of the sub-process (ala POpen and runProcess), but apart from that it seems to be the most flexible of the four. It could be simplified slightly by using one Handle rather than two for stdin/stdout. We should think about what functionality we want to provide in the core libraries. My preference would be to go for a full version with three pipes (like shell-haskell) and perhaps one or two more specialised versions covering common cases (I don't know what these should be, though). What do people think? Cheers, Simon -----Original Message----- From: haskell-admin(a)haskell.org [mailto:haskell-admin@haskell.org] On Behalf Of David Sankel Sent: 06 May 2003 03:06 To: haskell(a)haskell.org Subject: [ANNOUNCE] shell-haskell 1.0.0 =================== shell-haskell 1.0.0 =================== shell-haskell provides utilities to communicate with other process's via Haskell code. This library can be used for something as simple as getting the output of another program or as complex as interacting with an interpreter. see http://www.electronconsulting.com/shell-haskell for details. David J. Sankel _______________________________________________ Haskell mailing list Haskell(a)haskell.org http://www.haskell.org/mailman/listinfo/haskell

5 10

DData in hierarchical libraries
by JP Bernardy 19 Mar '04

19 Mar '04

Hello, I've compiled a haddock documentation for my proposal "DData in hierarchical libraries". I submit it to you for review. http://users.skynet.be/jyp/DData/doc/index.html The source code is available too. http://users.skynet.be/jyp/DData/doc/ddata.tar.gz Daan: I take the liberty to "publish" this without knowing the exact terms of your license. Given the audience of this list, I think it will be no problem. Cheers, JP. --------------- For easier review, a summary of changes follow. The source distribution also contains a diff. * Scc module removed More an algorithm than a data structure. Sort of (but not quite) redundant with SCC in Data.Graph * renamed "single" to "singleton" more explicit, more usual. * removed find, favouring (!) and lookup. avoids incoherency with Data.List.find Map.find :: Map k v -> k -> v List.find :: (a -> Bool) -> [a] -> Maybe a * removed (<>), prefering append More explicit. Besides, operators might be a problem in the current scheme of using modules qualified. * rename "isEmpty" to "null" to match List.null * foldL, foldR, foldI -> foldl, foldr, foldi to match Prelude.fold[lr] * Added instances * renamed "subset" to "isSubsetOf" this matches isPrefixOf, isSuffixOf * Added documentation / changed it to reflect changes incl. hints on how to use the module system. ----- I think I should explain why I wanted to change the interface of Maps. Consider the type of these functions: Set.fromList :: Ord a => [a] -> Set a Set.member :: Ord a => a -> Set a -> Bool Set.fold :: (a -> b -> b) -> b -> Set a -> b Set.filter :: Ord a => (a -> Bool) -> Set a -> Set a Map.fromList :: Ord k => [(k, a)] -> Map k a Map.member :: Ord k => k -> Map k a -> Bool Map.fold :: (a -> b -> b) -> b -> Map k a -> b Map.filter :: Ord k => (a -> Bool) -> Map k a -> Map k a With respect to fold and filter (and most functions), maps have the shape of collections of values. (The key type does not appear as such in the functions' signiatures) Yet, a few functions break this conventions. For example, "member" tests the presence of a key, not a value. This is justified by the implementation. I've come to think it is the way to go (given it is a "concrete" library). It makes me slightly uneasy that a few functions stand out, still. __________________________________ Do you Yahoo!? Get better spam protection with Yahoo! Mail. http://antispam.yahoo.com/tools

4 3

DData in hierarchical libraries
by JP Bernardy 13 Mar '04

13 Mar '04

Dear Haskell Librairies Mailing List, Some time ago we discussed the inclusion of DData in hierachical libraries. I remember Simon Marlow proposing someone "thrashes out the details" and present the result for review. So, I've played with the code, to come up with a list of proposed changes. The rationale behind most of these changes is better consistency (so enventually the library can use overloading), simplicity, and readability. I understand that most of these are trival, mostly naming issues, and therefore, open the door to potentially endless and futile disscussions. My goal is just to pinpoint potential flaws, and have your opinions. Then, I would propose a complete revised code, unless Daan wishes to do so. Here comes the changes: * remove the Scc module Seems to be redundant with SCC in Data.Graph * foldL, foldR, foldI -> foldl, foldr, foldi to match Prelude.fold[lr] * remove find, favouring (!) and lookup. avoids incoherency with Data.List.find * rename "single" to "singleton" more explicit, more usual. * remove (Queue, Seq).append (favouring "<>") Unneeded redundancy; addtitonally, the name doesn't suit well partial application. (implicit infix meaning) >map (append q1) listOfQueues means that q1 is "prepended" to each queue. * Rename subset & friends. Similarly, they have an implicit infix meaning. >filter (subset s1) listOfSets actually keeps sets that are supersets of s1. Is there an agreed-upon (operator) name for this? * Make maps look more like collections of couples. This involves changing some functions, as such: > filter f = Map.filterWithKey (curry f) > > partition f = Map.partitionWithKey (curry f) > > member (k,a) m = > case Map.lookup k m of > Nothing -> False > Just a' -> a == a' Similarly for fold, etc. Most debatable issues: * rename (<>) to (++) * Make some types instances of Ord and Functor. Additionnaly, a root name in the hierarchy must be chosen. A simple "DData" looks good to me. Or Data.Aux is better? Cheers, JP. __________________________________ Do you Yahoo!? Yahoo! Mail SpamGuard - Read only the mail you want. http://antispam.yahoo.com/tools

7 8

Data.PackedString.hGetPS
by Ian Lynagh 28 Feb '04

28 Feb '04

Hi all, If I have \begin{code} import Data.PackedString (hGetPS) import System.IO (openBinaryFile, hClose, IOMode(ReadMode)) main :: IO () main = do h <- openBinaryFile "1000000x" ReadMode p <- hGetPS h 1000000 hClose h return () \end{code} (1000000x is a file containing 1000000 'x' characters) then: and compile with ghc -O2 (either 6.2 or reasonably recent CVS) then I get: $ ./foo Stack space overflow: current size 1048576 bytes. Use `+RTS -Ksize' to increase it. $ The definition is: -- | Read a 'PackedString' directly from the specified 'Handle'. -- This is far more efficient than reading the characters into a 'String' -- and then using 'packString'. -- NOTE: as with 'hPutPS', the string representation in the file is -- assumed to be ISO-8859-1. hGetPS :: Handle -> Int -> IO PackedString hGetPS h i = do arr <- newArray_ (0, i-1) l <- hGetArray h arr i chars <- mapM (\i -> readArray arr i >>= return.chr.fromIntegral) [0..l-1] return (packString chars) (is the efficiency comment accurate? This definition is calling packString on a String it makes, so it's not intuitively obvious to me). I think the last line should be replaced with return (packNChars l chars) and that the packNChars definition: packNChars len str = PS (array (0,len-1) (zip [0..] str)) is equivalent to: packNChars len str = PS (listArray (0,len-1) str) which I assume is more efficient (due to not con/destructing tuples if nothing else). None of this solves the stack overflow, though. While I'm here, David Roundy has a PackedString which looks to have been forked from GHC or its libraries, now based around data PackedString = PS !(ForeignPtr Word8) !Int !Int He said it can't serve as a replacement as it doesn't support large unicode characters, though (would just changing to a Word32 or Char fix that?). If you're interested it's in the darcs (http://abridgegame.org/darcs/) darcs repo - selecting "FastPackedString.hs" on http://abridgegame.org/cgi-bin/darcs?darcs* should show you the latest version. It doesn't have this problem. Thanks Ian

1 0

RE: URI handling code; proposed Network.URI replacement
by Simon Marlow 25 Feb '04

25 Feb '04

> Thanks. (I've started trying to apply Haddock markup in my > own code, but > I've not yet got round to actually generating the docs from it.) > > >Since it's mostly backwards compatible, there's no issue > with upgrading > >it in place. (well, there's one small issue: it introduces > a dependency > >from the network package on the parsec package, but that's not a big > >deal). > > Hmmm... Parsec is pretty central to what I've done. It's not a problem: just add parsec to the PACKAGE_DEPS for the network package. > >We'll arrange for CVS access so you can commit your new > version to the > >repository. > > OK. > > Another detail. Do you have a preferred form of licence for > this. I've > been putting my own applications out as GPL to date, but I > imagine that the > viral effect would be problematic for the compiler library. > I'm happy to > use any reasonable (open) licence. BSD-style is preferred, because the network package is already BSD licensed. A package is the unit of licensing, so libraries with different licenses have to go in separate packages (to make it easier for users to be sure that they are complying with licenses). Cheers, Simon

1 0

RE: [Haskell] regular expression syntax - perl ain't got nothin on haskell
by Simon Marlow 25 Feb '04

25 Feb '04

[ moved to libraries(a)haskell.org ] > Inspired by an idea by Andrew Pang and an old project of > mine, I decided > to fill out a reusable regular expression library which is similar to > Perl's, but much more expressive. > > It provides regular and monadic versions, a very overloaded and useful > interface, as well as extensibility. although currently the only > instance is based on Text.Regex, it generalizes to matching lists of > arbitrary type, not just strings, and also leaves the door open for > compile-time checked and optimized regular expressions via template > Haskell. > > so, does this seem interesting? I am really enjoying the =~ syntax > already in projects, and the monadic version is great for > concise ad-hoc parsers. This is neat. We should provide (something like) this in Text.Regex itself, although perhaps the more generic regular expression functionality should go somewhere like Data.Regex? In any case, I'd be happy to add something to Text.Regex if you settle on something you're happy with and send it to the list for comments. Cheers, Simon

1 0

More on POpen for Windows
by Graham Klyne 24 Feb '04

24 Feb '04

Further to problems reported in [1]. I've made a small step forward on this. I'm nearly out of time for now, so this is a partial report. I've created a pair of simple C programs: + EchoShiftLetters.c Reads from standard input and writes to standard output, converting all letters to upper case + SpawnEchoShiftLetters.c Uses spawnProc (the function at the heart of POpen) to run EchoShiftLetters, write some data to its standard output and read back from the standard output and standard error streams. It displays the data read back. This program works fine. What this demonstrates is that the basic Win32 API calls are working as expected, and that the problems encountered previously are probably to do with the Haskell-to-C interface, or the Hugs library routines that create I/O streams from file descriptors. My next step, when I get more time, is to try a simple Haskell program spawning the EchoShiftLetters program -- I'm expecting that to fail. There's one other datum: I attempted to build these test programs using the MinGW gcc tools, but got an undefined symbol error (I think it was WinMain) when building the SpawnEchoShiftLetters program. So I reverted to using the old MS Developer Studio tools. I think it was probably just a library missing from the build, but can't be sure of that. The code for all this can be found at [2]. #g -- [1] http://www.haskell.org//pipermail/libraries/2004-February/001700.html [2] http://www.ninebynine.org/Software/HaskellUtils/Win32/ ------------ Graham Klyne For email: http://www.ninebynine.org/#Contact

1 0

RE: URI handling code; proposed Network.URI replacement
by Graham Klyne 24 Feb '04

24 Feb '04

At 13:07 24/02/04 +0000, Simon Marlow wrote: > > > I've completed the functionality of my proposed upgrade to > > Network.URI, > > though there are still a couple of issues that need to be > > resolved with the > > ongoing RFC2396bis work (none critical to the general > > functionality). This > > is still work-in-progress, but I'm airing it now to see if > > folks think it's > > an acceptable upgrade to the existing Network.URI module. > >Sorry for not replying on this topic sooner. > >I've taken a look at your replacement URI, and it looks great. The >Haddock markup needs a little cleanup, but the interface looks fine. Thanks. (I've started trying to apply Haddock markup in my own code, but I've not yet got round to actually generating the docs from it.) >Since it's mostly backwards compatible, there's no issue with upgrading >it in place. (well, there's one small issue: it introduces a dependency >from the network package on the parsec package, but that's not a big >deal). Hmmm... Parsec is pretty central to what I've done. >We'll arrange for CVS access so you can commit your new version to the >repository. OK. Another detail. Do you have a preferred form of licence for this. I've been putting my own applications out as GPL to date, but I imagine that the viral effect would be problematic for the compiler library. I'm happy to use any reasonable (open) licence. #g ------------ Graham Klyne For email: http://www.ninebynine.org/#Contact

1 0

RE: URI handling code; proposed Network.URI replacement
by Simon Marlow 24 Feb '04

24 Feb '04

> I've completed the functionality of my proposed upgrade to > Network.URI, > though there are still a couple of issues that need to be > resolved with the > ongoing RFC2396bis work (none critical to the general > functionality). This > is still work-in-progress, but I'm airing it now to see if > folks think it's > an acceptable upgrade to the existing Network.URI module. Sorry for not replying on this topic sooner. I've taken a look at your replacement URI, and it looks great. The Haddock markup needs a little cleanup, but the interface looks fine. Since it's mostly backwards compatible, there's no issue with upgrading it in place. (well, there's one small issue: it introduces a dependency from the network package on the parsec package, but that's not a big deal). We'll arrange for CVS access so you can commit your new version to the repository. Cheers, Simon

1 0

RE: new release of LIP prototype API
by Simon Marlow 24 Feb '04

24 Feb '04

> (snip) > > I've downloaded the latest, but I'm having a hard time > getting my head > > around the overall structure. It feels to me a bit like a tangled > > ball of string, whose free end I cannot find. (This a comment on my > > understanding, not your code/design!) > > :) Allow me to try to give a top-down overview of the structure, which should help you find the end(s) of the string. A Haskell library (or application?) comes with a Haskell program in the form of a file named Setup.hs. This program does the following: - it accepts certain standard command-line options (build all, build default etc.) - It knows how to build, and install, the library using sensible defaults. It does this by either - Invoking the LIP build/install system in Distribution.Build and Distribution.Install, or - Invoking a custom build system (standard configure; make; make install stuff). - Once installed, Setup.hs registers the package using the Distribution.Package interface. There are still quite a few questions to answer here, some of which Graham brought up. One is how do we tweak the installation locations for the libraries, and other build/install-time options? Normally this is done by passing arguments to configure: should we have more options to Setup.hs for these things? Looking at the proposal, Perhaps Setup.lhs should have a configure step too. Something like this: $ ./Setup.lhs configure prefix=/usr/local HC=/usr/bin/ghc $ ./Setup.lhs build $ ./Setup.lhs install $ ./Setup.lhs uninstall (registation is done as part of install). Note the uninstall step, which also unregisters the package. Does it matter which compiler is used to run Setup.lhs? I claim yes (see previous discussions). Cheers, Simon

1 0