- Libraries - Haskell.org

Data.PackedString.hGetPS
by Ian Lynagh 28 Feb '04

28 Feb '04

Hi all, If I have \begin{code} import Data.PackedString (hGetPS) import System.IO (openBinaryFile, hClose, IOMode(ReadMode)) main :: IO () main = do h <- openBinaryFile "1000000x" ReadMode p <- hGetPS h 1000000 hClose h return () \end{code} (1000000x is a file containing 1000000 'x' characters) then: and compile with ghc -O2 (either 6.2 or reasonably recent CVS) then I get: $ ./foo Stack space overflow: current size 1048576 bytes. Use `+RTS -Ksize' to increase it. $ The definition is: -- | Read a 'PackedString' directly from the specified 'Handle'. -- This is far more efficient than reading the characters into a 'String' -- and then using 'packString'. -- NOTE: as with 'hPutPS', the string representation in the file is -- assumed to be ISO-8859-1. hGetPS :: Handle -> Int -> IO PackedString hGetPS h i = do arr <- newArray_ (0, i-1) l <- hGetArray h arr i chars <- mapM (\i -> readArray arr i >>= return.chr.fromIntegral) [0..l-1] return (packString chars) (is the efficiency comment accurate? This definition is calling packString on a String it makes, so it's not intuitively obvious to me). I think the last line should be replaced with return (packNChars l chars) and that the packNChars definition: packNChars len str = PS (array (0,len-1) (zip [0..] str)) is equivalent to: packNChars len str = PS (listArray (0,len-1) str) which I assume is more efficient (due to not con/destructing tuples if nothing else). None of this solves the stack overflow, though. While I'm here, David Roundy has a PackedString which looks to have been forked from GHC or its libraries, now based around data PackedString = PS !(ForeignPtr Word8) !Int !Int He said it can't serve as a replacement as it doesn't support large unicode characters, though (would just changing to a Word32 or Char fix that?). If you're interested it's in the darcs (http://abridgegame.org/darcs/) darcs repo - selecting "FastPackedString.hs" on http://abridgegame.org/cgi-bin/darcs?darcs* should show you the latest version. It doesn't have this problem. Thanks Ian

1 0

RE: URI handling code; proposed Network.URI replacement
by Simon Marlow 25 Feb '04

25 Feb '04

> Thanks. (I've started trying to apply Haddock markup in my > own code, but > I've not yet got round to actually generating the docs from it.) > > >Since it's mostly backwards compatible, there's no issue > with upgrading > >it in place. (well, there's one small issue: it introduces > a dependency > >from the network package on the parsec package, but that's not a big > >deal). > > Hmmm... Parsec is pretty central to what I've done. It's not a problem: just add parsec to the PACKAGE_DEPS for the network package. > >We'll arrange for CVS access so you can commit your new > version to the > >repository. > > OK. > > Another detail. Do you have a preferred form of licence for > this. I've > been putting my own applications out as GPL to date, but I > imagine that the > viral effect would be problematic for the compiler library. > I'm happy to > use any reasonable (open) licence. BSD-style is preferred, because the network package is already BSD licensed. A package is the unit of licensing, so libraries with different licenses have to go in separate packages (to make it easier for users to be sure that they are complying with licenses). Cheers, Simon

1 0

RE: [Haskell] regular expression syntax - perl ain't got nothin on haskell
by Simon Marlow 25 Feb '04

25 Feb '04

[ moved to libraries(a)haskell.org ] > Inspired by an idea by Andrew Pang and an old project of > mine, I decided > to fill out a reusable regular expression library which is similar to > Perl's, but much more expressive. > > It provides regular and monadic versions, a very overloaded and useful > interface, as well as extensibility. although currently the only > instance is based on Text.Regex, it generalizes to matching lists of > arbitrary type, not just strings, and also leaves the door open for > compile-time checked and optimized regular expressions via template > Haskell. > > so, does this seem interesting? I am really enjoying the =~ syntax > already in projects, and the monadic version is great for > concise ad-hoc parsers. This is neat. We should provide (something like) this in Text.Regex itself, although perhaps the more generic regular expression functionality should go somewhere like Data.Regex? In any case, I'd be happy to add something to Text.Regex if you settle on something you're happy with and send it to the list for comments. Cheers, Simon

1 0

More on POpen for Windows
by Graham Klyne 24 Feb '04

24 Feb '04

Further to problems reported in [1]. I've made a small step forward on this. I'm nearly out of time for now, so this is a partial report. I've created a pair of simple C programs: + EchoShiftLetters.c Reads from standard input and writes to standard output, converting all letters to upper case + SpawnEchoShiftLetters.c Uses spawnProc (the function at the heart of POpen) to run EchoShiftLetters, write some data to its standard output and read back from the standard output and standard error streams. It displays the data read back. This program works fine. What this demonstrates is that the basic Win32 API calls are working as expected, and that the problems encountered previously are probably to do with the Haskell-to-C interface, or the Hugs library routines that create I/O streams from file descriptors. My next step, when I get more time, is to try a simple Haskell program spawning the EchoShiftLetters program -- I'm expecting that to fail. There's one other datum: I attempted to build these test programs using the MinGW gcc tools, but got an undefined symbol error (I think it was WinMain) when building the SpawnEchoShiftLetters program. So I reverted to using the old MS Developer Studio tools. I think it was probably just a library missing from the build, but can't be sure of that. The code for all this can be found at [2]. #g -- [1] http://www.haskell.org//pipermail/libraries/2004-February/001700.html [2] http://www.ninebynine.org/Software/HaskellUtils/Win32/ ------------ Graham Klyne For email: http://www.ninebynine.org/#Contact

1 0

RE: URI handling code; proposed Network.URI replacement
by Graham Klyne 24 Feb '04

24 Feb '04

At 13:07 24/02/04 +0000, Simon Marlow wrote: > > > I've completed the functionality of my proposed upgrade to > > Network.URI, > > though there are still a couple of issues that need to be > > resolved with the > > ongoing RFC2396bis work (none critical to the general > > functionality). This > > is still work-in-progress, but I'm airing it now to see if > > folks think it's > > an acceptable upgrade to the existing Network.URI module. > >Sorry for not replying on this topic sooner. > >I've taken a look at your replacement URI, and it looks great. The >Haddock markup needs a little cleanup, but the interface looks fine. Thanks. (I've started trying to apply Haddock markup in my own code, but I've not yet got round to actually generating the docs from it.) >Since it's mostly backwards compatible, there's no issue with upgrading >it in place. (well, there's one small issue: it introduces a dependency >from the network package on the parsec package, but that's not a big >deal). Hmmm... Parsec is pretty central to what I've done. >We'll arrange for CVS access so you can commit your new version to the >repository. OK. Another detail. Do you have a preferred form of licence for this. I've been putting my own applications out as GPL to date, but I imagine that the viral effect would be problematic for the compiler library. I'm happy to use any reasonable (open) licence. #g ------------ Graham Klyne For email: http://www.ninebynine.org/#Contact

1 0

RE: URI handling code; proposed Network.URI replacement
by Simon Marlow 24 Feb '04

24 Feb '04

> I've completed the functionality of my proposed upgrade to > Network.URI, > though there are still a couple of issues that need to be > resolved with the > ongoing RFC2396bis work (none critical to the general > functionality). This > is still work-in-progress, but I'm airing it now to see if > folks think it's > an acceptable upgrade to the existing Network.URI module. Sorry for not replying on this topic sooner. I've taken a look at your replacement URI, and it looks great. The Haddock markup needs a little cleanup, but the interface looks fine. Since it's mostly backwards compatible, there's no issue with upgrading it in place. (well, there's one small issue: it introduces a dependency from the network package on the parsec package, but that's not a big deal). We'll arrange for CVS access so you can commit your new version to the repository. Cheers, Simon

1 0

RE: new release of LIP prototype API
by Simon Marlow 24 Feb '04

24 Feb '04

> (snip) > > I've downloaded the latest, but I'm having a hard time > getting my head > > around the overall structure. It feels to me a bit like a tangled > > ball of string, whose free end I cannot find. (This a comment on my > > understanding, not your code/design!) > > :) Allow me to try to give a top-down overview of the structure, which should help you find the end(s) of the string. A Haskell library (or application?) comes with a Haskell program in the form of a file named Setup.hs. This program does the following: - it accepts certain standard command-line options (build all, build default etc.) - It knows how to build, and install, the library using sensible defaults. It does this by either - Invoking the LIP build/install system in Distribution.Build and Distribution.Install, or - Invoking a custom build system (standard configure; make; make install stuff). - Once installed, Setup.hs registers the package using the Distribution.Package interface. There are still quite a few questions to answer here, some of which Graham brought up. One is how do we tweak the installation locations for the libraries, and other build/install-time options? Normally this is done by passing arguments to configure: should we have more options to Setup.hs for these things? Looking at the proposal, Perhaps Setup.lhs should have a configure step too. Something like this: $ ./Setup.lhs configure prefix=/usr/local HC=/usr/bin/ghc $ ./Setup.lhs build $ ./Setup.lhs install $ ./Setup.lhs uninstall (registation is done as part of install). Note the uninstall step, which also unregisters the package. Does it matter which compiler is used to run Setup.lhs? I claim yes (see previous discussions). Cheers, Simon

1 0

RE: DData in hierarchical libraries
by Simon Peyton-Jones 23 Feb '04

23 Feb '04

JP Good for you! Do press ahead with getting a tidied-up interface agreed. (I doubt many people will have much to say, but don't be discouraged by that.) A good plan would be to Haddock-ise the source code, generate the HTML and send the link to the libraries list. That way people could easily browse your proposed interface. I wonder if any existing libraries could be removed (or, rather, deprecated) when you add DData? If so, say so. Simon | -----Original Message----- | From: libraries-bounces(a)haskell.org [mailto:libraries-bounces@haskell.org] On Behalf Of JP Bernardy | Sent: 20 February 2004 13:27 | To: libraries(a)haskell.org | Subject: DData in hierarchical libraries | | Dear Haskell Librairies Mailing List, | | Some time ago we discussed the inclusion of DData in | hierachical libraries. | | I remember Simon Marlow proposing someone "thrashes | out the details" and present the result for review. | | So, I've played with the code, to come up with a list | of proposed changes.

1 0

Re: new release of LIP prototype API
by Graham Klyne 23 Feb '04

23 Feb '04

At 15:55 22/02/04 -0500, Isaac Jones wrote: >Yep. It's unixy. See above. If you have any #defines I can add to >the code for windows, please pass them along. My preference would be to try and concentrate the system dependencies into a single location, (or to minimize their appearance). To that end, I'm minded to try and do an implementation of a basic parameter-grabbing module that can be easily re-implemented for different environments. Maybe easier said than done. But based on your responses this would be complementary to the work you're doing. >Let me know if I answered your questions well enough. That was helpful, thanks. I'll study the info more deeply later. #g ------------ Graham Klyne For email: http://www.ninebynine.org/#Contact

1 0

new release of LIP prototype API
by Isaac Jones 22 Feb '04

22 Feb '04

Greetings, I just uploaded a new release of the Library Infrastructure prototype API for Distribution.Package and the command-line utility "haskell-config". The API for package configuration is included below, FYI. I'm going to be focusing more on the Build stuff (rather than the packaging stuff) for a little while, but with luck, this can be near to finalization soon. The web page is here: http://www.haskell.org/libraryInfrastructure/ Feedback and volunteers for coding are always welcome :) peace, isaac ps. I'm using arch (tla) for version control, so if you've been wanting to check it out, now is a good time :) ------------------------------------------------------------ lookupPackage :: PackageDB -> PkgIdentifier -> Maybe PackageConfig addPackage :: PackageDB -> PackageConfig -> PackageDB delPackage :: PackageDB -> PkgIdentifier -> (PackageDB, Bool) allPackages :: PackageDB -> [PackageConfig] createPackageDB :: FilePath -> (PackageDB -> IO PackageDB) -> IO () withPackageDB :: FilePath -> (PackageDB -> IO (PackageDB, a)) -> IO a userPkgConfigLocation :: String -> FilePath userPkgConfigLocation home = home ++ "/.haskell/packages.conf" systemPkgConfigLocation :: FilePath systemPkgConfigLocation = "/etc/haskell/packages.conf" PackageConfig(..), PkgIdentifier(..), License(..), Dependency(..), PackageDB

2 3