- Libraries - Haskell.org

Data.ByteString: copyCStringLen not in the IO monad
by Tomasz Zielonka 18 May '06

18 May '06

Hello! Why copyCStringLen :: CStringLen -> ByteString? It seems pretty dangerous, because at the time of forcing (copyCStringLen csLen), csLen may no longer be valid. I think it would be better if this and other similar functions returned their results in the IO monad. Best regards Tomasz

2 1

Revised Numeric Prelude
by Ashley Yakeley 17 May '06

17 May '06

Aaron Denney wrote (on the Haskell' list): > Meh. I'd really like a revised numeric prelude to be able to use MPTCs > with FDs. Do you have some ideas for this? There's no reason why such an effort shouldn't start right now; I daresay even if we used FDs while Haskell eventually goes with ATs it might still be useful. I know there's the Basic Algebra proposal, but there's probably been some other work done here. -- Ashley Yakeley, Seattle WA WWEWDD? http://www.cs.utexas.edu/users/EWD/

2 1

make Text.Printf extensible, more flexible
by John Meacham 16 May '06

16 May '06

I was wondering if we could make Text.Printf extensible to user defined types. as far as I can tell, all that is needed is to export PrintfArg so that users can define new instances. it would also be nice if UPrintf could be switched from the current union type to a higher order function, since the only thing UPrintf is used for is formating strings. I am thinking something like newtype UPrintf = UPrintf (UFmt -> ShowS) data Adjustment = LeftAdjust | ZeroPad data UFmt = UFmt { fmtAdjust :: Maybe Adjustment, fmtFieldWidth :: Maybe Int, fmtPrecision :: Maybe Int, fmtCharacter :: Char } so, a UPrintf is just a function that takes a parsed formating string and prints the appropriate string. John -- John Meacham - ⑆repetae.net⑆john⑈

2 1

ByteStrings and the ram barrier
by dons＠cse.unsw.edu.au 16 May '06

16 May '06

Hey libraries@, Data.ByteStrings are strict, unboxed arrays of Word8s. They're great for fast processing of strings that fit in memory but no good for lazy processing, or dealing with data larger than your ram (i.e. they're infeasible above 1G on a 2G ram box). Last week Duncan Coutts had the bright idea to write Data.ByteString.Lazy, a newtyped [ByteString], which would allow constant space processing of chunks of strict ByteString arrays. The theory is that we'd be able to efficiently process large data, where both ByteString and [Char] fails, and open a new range of applications that could be handled successfully with Haskell. By using list operations over the spine of the structure, and fast ByteString operations over the leaves, we should be able to get some of the benefits of both lazy lists and strict bytestrings. For example, to map over a lazy bytestring you do a list map along the spine of the list, and a bytestring map over the nodes: lazy bytestring map = L.map (B.map f) xs So we hacked away at it for a few days, and here are some preliminary results: Simple task, filter the letter 'e' from a file. ------------------------------------------------------------------------ 4G file, 1G ram, Linux/x86, 3.20GHz P4, GHC 6.4.1 Data.ByteString.Lazy (128k chunks) constant heap, total time 2m 20s, 25% cpu Data.List constant heap, total time 8 minutes (quite good!), 98% cpu sed 's/e//g' constant heap, I gave up after 10 minutes ------------------------------------------------------------------------ 10G file, 1G ram, OpenBSD/x86, 2.4GHz P4, GHC 6.4.1 Data.ByteString.Lazy (5M chunks) constant heap, total time 11.44 minutes, 34% cpu Data.List constant heap, total time 18.21 minutes, 90% cpu ------------------------------------------------------------------------ A nice improvement for Haskell on mutli-gigabyte files, I think! Data.ByteString.Lazy is in fps head, here: http://www.cse.unsw.edu.au/~dons/code/fps/Data/ByteString/Lazy.hs Cheers, Don

3 5

Library Tests Location?
by Ashley Yakeley 16 May '06

16 May '06

I wrote a bunch of unit tests for my time library. What should I do with the unit tests I wrote? Should I convert them to all.T etc. format and put them in the testsuite repo? Or should I leave them in the time repo? -- Ashley Yakeley, Seattle WA WWEWDD? http://www.cs.utexas.edu/users/EWD/

2 1

xml in fptools?
by Lemmih 15 May '06

15 May '06

On 5/15/06, Isaac Jones <ijones(a)syntaxpolice.org> wrote: > Lemmih <lemmih(a)gmail.com> writes: > > > On 5/15/06, Isaac Jones <ijones(a)syntaxpolice.org> wrote: > >> Greetings, > >> > >> I'm wondering if we've ever discussed putting an XML parser into > >> fptools. I'm asking because cabal-install would like to use xml-rpc, > >> but it drags in a bunch of dependencies that aren't in fptools. > >> > >> What do other languages do: > >> * Python seems to have XML and XML-RPC as standard libraries > >> * Same with ruby > >> * Not java (shocking): http://java.sun.com/j2se/1.3/docs/api/ > >> > >> I guess that HaXml probably can't get into fptools because of license > >> incompatibilities, but hxml and HXmlToolbox both have MIT licenses, > >> and so could probably be put into fptools: > >> > >> http://www.flightlab.com/~joe/hxml/ > >> http://www.fh-wedel.de/~si/HXmlToolbox/ > >> > >> What do others think? > > > > XML and xml-rpc are the wrong tools for cabal-install, IMO. Picking a > > tool that isn't suited for the task will not result in anything good, > > no matter how common and popular that tool happens to be. I only chose > > xml-rpc because it was easily available, and it resulted in several > > dirty hacks. Please, lets avoid enterprise, XML and managers in suits; > > The /right/ tools are out there. > > End of way too emotional rant. > > Care to elaborate with some facts or details that can help the rest of > us make a decision? Well, XML/xml-rpc isn't very well suited for transferring binary data and that's pretty much the only thing cabal-install is interested in. Using bencoding would be more efficient and not drag in any dependencies. BEncoding is used by bittorrent and, hence, is implemented in almost every language. -- Friendly, Lemmih

1 0

ByteStrings and the Terabyte barrier
by dons＠cse.unsw.edu.au 15 May '06

15 May '06

Just for fun, I was able to filter a 1 Terabyte stream through a Haskell program, using Data.ByteString.Lazy, with a constant 2M heapsize, in $ time ( repeat 100 ; cat /home/dons/data/10G | ./lazy > /dev/null ) 12752.37s user 4069.35s system 58% cpu 8:03:11.42 total 8 hours. Just so we can say it is possible :) -- Don

1 0

Announce: Collection package pre-release.
by Jean-Philippe Bernardy 15 May '06

15 May '06

Hello folks, This is an announcement for an alpha release of the new collections package I (and others, see below) have been working on. It's still far from perfect, but I hope it's already a good choice for many use cases of collection data structures. Also, "release early, release often" : I hope releasing now will allow me to keep a on improving it at a steady rate. Darcs repo: http://darcs.haskell.org/packages/collections/ Documentation: http://users.skynet.be/jyp/html/ (documentation for base package can be found there as well, for the purpose of cross-referencing) It is originally designed to depend on the upcoming version of the base package; however I've arranged it to work with current version as well. (Meaning you can use it as a cabal package with GHC 6.4) Please try out and let me know what you think. 0.9rc1 release notes: * Efficient AVL-tree based implementations of Set and Map (by Adrian Hey) * efficient conversion routines between those Sets and Maps * Class-based Collection framework for abstracting over Collections, Sequences, Maps, Sets. * Nicely corresponds to the current API. In a lot of cases, just removing the module prefix does the trick. * QuickCheck-based validation of the instances. * Benchmark suite * Adaptation of Ross Paterson's Foldable class to MPTC+FD. * Trie data type, to map sequential types (eg. lists) to keys. * Efficient Set type for enumerations. (David F. Place) --- Cheers, JP.

1 0

Re: ByteStrings and the ram barrier
by dons＠cse.unsw.edu.au 14 May '06

14 May '06

kahl: > > > > After tuning the chunk size to the cache size, the filter 'e' test now > > runs in: > > 4.52 minutes, 53% cpu. > > With respect to the original comparison, > Ryan Lortie pointed out that ``sed'' is an unfair comparison, > and ``tr'' would be more appropriate. Hi! Good idea, I always forget about tr. Here's the result: manzano$ time tr -d e < /home/dons/data/10G > /dev/null tr -d e < /home/dons/data/10G > /dev/null 26% cpu 4:34.55 total I'm happy with this, within 6% of C, for a one liner. The Haskell program is: > import System.IO > import Data.Char > import qualified Data.ByteString.Lazy as L > > main = L.hGetContents stdin >>= L.hPut stdout . L.filterNotByte e > where e = fromIntegral . ord $ 'e' -- Don

1 0

[patch] Win32 enchaments
by Esa Ilari Vuokko 12 May '06

12 May '06

Hi Is this really the list for patches? http://darcs.haskell.org/packages/Win32/_darcs/prefs/email says it is. Here's few patches to Win32. First cabal file needed some poking to be usable and boring-control is also nice to have. Main patch brings in file mapping (mmap-style api in Windows) and most of the Time API. There's also some few more enchaments and fixes to warnings etc. I quickly tested that I can get Makefile work too, but I haven't tested it much yet. I also pushed version number so it's possible to use cabal versioning to fail versions without added stuff. Best regards, --Esa Tue May 9 15:35:25 FLE Standard Time 2006 Esa Ilari Vuokko <ei(a)vuokko.info> * Make repo buildable with cabal and add better boring-control Wed May 10 14:18:35 FLE Standard Time 2006 Esa Ilari Vuokko <ei(a)vuokko.info> * Add FileMapping and Time and other tweaks. Upped version to 1.1 Added System.Win32.Time - most of Time api Added System.Win32.FileMapping - file mapping api and helpers Added BY_HANDLE_FILE_INFORMATION and getFileInformationByHandle Added SYSTEM_INFO and getSystemInfo Cleaned up few warnings and bugs Few more cabal tweaks to options Wed May 10 14:25:41 FLE Standard Time 2006 Esa Ilari Vuokko <ei(a)vuokko.info> * Update Makefile and package.conf.in

3 5