- Haskell-Cafe - Haskell.org

ANN: regex-tdfa 0.56
by Chris Kuklewicz 27 Jan '07

27 Jan '07

Greetings, My new regular expression backend "regex-tdfa" is passing all the tests I have been throwing at it, so it is time to share the good news. First up: Where is it? Version 0.56 is in my "stable" location at > darcs get --partial darcs.haskell.org:/home/darcs/packages/regex-tdfa The version that will get updated and broken more often is "unstable" at > darcs get --partial darcs.haskell.org:/home/darcs/packages/regex-unstable/regex-tdfa It needs a slightly updated regex-base package (version >= 0.80) at > darcs get --partial darcs.haskell.org:/home/darcs/packages/regex-unstable/regex-base It has only been tested (on Max OS X) with GHC 6.6 and depends on base >= 2.0 for Data.ByteString, Data.ByteString.Lazy, and Data.Sequence. ** What does regex-tdfa do? It runs POSIX regular expression searches correctly. No other regular expression backend for Haskell does this correctly. The regular expression can be chosen to be from a String, ByteString, ByteString.Lazy, or (Data.Seq Char). Independent of that choice the searched text can also be any of those types. ** Properties and benefits of regex-tdfa: It follows a known standard: POSIX. (If you find that it disagrees with the standard then I will fix it.) It is written in Haskell, so there is no C library to compile and install. Though it is currently only written for GHC 6.6, it could be ported to any other compiler with only minor changes. (I expect that if you drop the polymorphic MPTC interface of regex-base it could be made Haskell98). It works with Unicode since it can be used on [Char] (and it works if there are '\NUL' characters around). It is lazy in the regular expression: it only builds the DFA if you use it, and only builds the state it needs, which are memoized in a lazy Trie. It is lazy in the searched text, and the pure function that searches the text only evaluates the text until the next match is done and does not hold onto the initial text. Thus you could efficiently search a [Char] up to the size of the Position type, which is an Int for now. It is free, the regex-tdfa library is under a BSD-3 license. ** Does it work? Yes. The regex-posix backend that come with GHC does not run all the searches correctly: it is buggy (example at end of announcement). And the regex-posix backend is also impossibly slow -- the slowest backend by far -- orders of magnitude slower. The regex-tre backend that wraps the TRE library has exposed this very fast library to the harsh gaze of QuickCheck. The lesson from QuickCheck is to never use regex-tre because the TRE library (version 0.7.4) is buggy. For regex-tdfa backend I took/stole ideas from Ville Laurikari's Master thesis which describes the algorithm that TRE uses. And I made a pure Haskell implementation that now passes all the tests at http://www.research.att.com/~gsf/testregex/testregex.html and follows the notes at http://www.research.att.com/~gsf/testregex/re-interpretation.html and passes those tests as well. And it "passes" QuickCheck tests which actually just compare the answers that regex-posix, regex-tre, and regex-tdfa since I do not have a bug free reference implementation to compare against. ** Why the name TDFA? Tagged Deterministic Finite Automata. The Tagged part is from Ville Laurikari's thesis work. ** Speed: If you do not care only about the match as a whole and not about subexpression matching then you can set the captureGroups ExecOption to False: makeMyRegex = makeRegex defaultCompOpt myExecOpt where myExecOpt = defaultExecOpt { captureGroups = False } And redefine (=~) and (=~~) to use makeMyRegex instead of makeRegex: (=~) :: (RegexMaker Regex CompOption ExecOption source,RegexContext Regex source1 target) => source1 -> source -> target (=~) s r = let re = makeMyRegex r :: Regex in match re s (=~~) :: (RegexMaker Regex CompOption ExecOption source,RegexContext Regex source1 target,Monad m)=> source1 -> source -> m target (=~~) s r = let re = makeMyRegex r :: Regex in match re s The benefit to this is speed, as it then proceeds to run all matches of "b(..?c)?d" against a large test file *faster* than the the TRE C library backend. My compliments to those who worked on ghc and -O2. It even runs nearly as fast as regex-dfa in this mode. Thus regex-dfa is (mostly) obsolete. If you use the default and capture the subgroups then the speed is reduced by a factor of about 4. This is because of the very simple data structures that keep track of the run time state. I intend to improve this aspect of the design now that I have a working algorithm. I have not benchmarked the speed of compiling the regular expression from the String. This compiling is done with very lazy data structures, so the DFA size is really only as big as you actually explore while running the match. ** Examples of bugs in regex-posix and regex-tre: An example for regex-posix: > *Main> "ab" Posix.=~ "((a)|(b)){2,}" :: MatchArray > array (0,3) [(0,(0,2)),(1,(1,1)),(2,(0,1)),(3,(1,1)) The array holds (group #, (match offset, match length)) The 0th group is the whole match "ab". The 2nd group is (a) and it matched the first "a" and the third group is (b) and matched the "b". But the second group is a child of the first group ((a)|(b)) which matched "b". The standard requires that captured groups are nested inside their parents, and here "a" is not not nested inside "b". So the regex-posix C backend that comes with GHC is buggy. The correct answer is, where the second group matched nothing: > *Main> "ab" TDFA.=~ "((a)|(b)){2,}" :: MatchArray > array (0,3) [(0,(0,2)),(1,(1,1)),(2,(-1,0)),(3,(1,1))] An example for regex-tre: "yyyyyy" TRE.=~ "(yyy|(x?)){2,4}" array (0,2) [(0,("yyyyyy",(0,6))),(1,("yyy",(3,3))),(2,("yyy",(3,3)))],"") Here TRE reports that the second group (x?) which should only match "" or "x" somehow matched "yyy". This is even more wrong than the regex-posix bug above. The correct answer, where the second group does not match: array (0,2) [(0,("yyyyyy",(0,6))),(1,("yyy",(3,3))),(2,("",(-1,0)))],"") -- Chris Kuklewicz

1 0

Trouble understanding records and existential types
by John Ky 27 Jan '07

27 Jan '07

Hi, A while back I asked about OO programming in Haskell and discovered existential types. I understood that existential types allowed me to write heterogeneous lists which seemed sufficient at the time. Now trying to combine those ideas with records: data AnyNode = forall a. Node a => AnyNode a class Node -- yadda yadda data Branch = Branch { name :: String, description :: String, children :: [AnyNode] } data Leaf = Leaf { name :: String, value :: String } The problem here is I can't use the same 'name' field for both Branch and Leaf. Ideally I'd like the name field in the Node class, but it doesn't seem that Haskell classes are for that sort of thing. -John

8 12

getting started with hsc (supercollider)
by alex 27 Jan '07

27 Jan '07

Hi all, I'm trying to get started on using Rohan Drape's Hsc library, excited about the prospect of parsing things straight into synthesis graphs, but progress is slow. There is good library documentation with examples, but no overview of how things fit together that I can find. Would anyone have some illustrative example code, and perhaps a couple of tips for how to get things working well in emacs? Once I get up to speed I would hope to contribute a beginner's guide back. Perhaps I should be addressing this email just to Rohan, but thought others might be able to help. alex

1 0

Type infer
by Marco Túlio Gontijo e Silva 27 Jan '07

27 Jan '07

Hello, I'm trying to define a partition__ function that is like Data.Set.partition, but use State Monad: > import Data.Set > import Control.Monad.State > partition__ f = > do > snapshot <- get > let > (firsts, rest) = Set.partition f snapshot > put rest > return firsts When I try to infer it's type in ghci I got: $ ghci ___ ___ _ / _ \ /\ /\/ __(_) / /_\// /_/ / / | | GHC Interactive, version 6.6, for Haskell 98. / /_\\/ __ / /___| | http://www.haskell.org/ghc/ \____/\/ /_/\____/|_| Type :? for help. Loading package base ... linking ... done. Prelude> :load partition.hs [1 of 1] Compiling Main ( partition.hs, interpreted ) Ok, modules loaded: Main. *Main> :type partition__ partition__ :: (MonadState (Set a) t, Ord a) => (a -> Bool) -> t (Set a) Ok, then I add > partition__ :: (MonadState (Set a) t, Ord a) => (a -> Bool) -> t (Set a) to the file and then: *Main> :reload [1 of 1] Compiling Main ( partition.hs, interpreted ) partition.hs:4:0: Non type-variable argument in the constraint: MonadState (Set a) t (Use -fglasgow-exts to permit this) In the type signature for `partition__': partition__ :: (MonadState (Set a) t, Ord a) => (a -> Bool) -> t (Set a) Failed, modules loaded: none. Why do I need glasgow-exts to specify a type infered by GHCi without -fglasgow-exts? Thanks. -- malebria Marco Túlio Gontijo e Silva Correio (MSN): malebria(a)riseup.net Jabber (GTalk): malebria(a)jabber.org Ekiga: malebria(a)ekiga.net IRC: malebria(a)irc.freenode.net malebria(a)irc.oftc.org Skype: marcotmarcot Telefone: 33346720 Celular: 98116720 Endereço: Rua Paula Cândido, 257/201 Gutierrez 30430-260 Belo Horizonte/MG Brasil

4 5

Concurrency in Haskell
by Alexy Khrabrov 27 Jan '07

27 Jan '07

What's the state of concurrency in Haskell? If Erlang's main strength is light-weight parallelism, can something like that be done in Haskell? Are there good examples of useful code employing GHC concurrency features one can play with? Cheers, Alexy

4 5

newbie timing question
by Sean McLaughlin 26 Jan '07

26 Jan '07

Hello, I'm trying to write a simple function to time an application. -- this doesn't work time f x = do n1 <- CPUTime.getCPUTime let res = f x in do n2 <- CPUTime.getCPUTime return (res,n2 - n1) On a function that takes 8 seconds to complete, returns (True,46000000) According to the documentation, this time is in picoseconds, making this 46 microseconds. I also tried -- neither does this time' f x = do n1 <- Time.getClockTime let res = f x in do n2 <- Time.getClockTime return (res,Time.diffClockTimes n2 n1) It returns something on that scale for picoseconds. How can I get a rough wall clock timing of functions? Thanks! Sean

4 3

newTArrayIO
by Chad Scherrer 26 Jan '07

26 Jan '07

This seems like a natural thing to have around, but it's not in GHC 6.6... newTArrayIO :: (Enum i, Ix i) => (i, i) -> a -> IO (TArray i a) newTArrayIO (a,b) = liftM (TArray . listArray (a,b)) . sequence . zipWith ignore [a..b] . repeat . newTVarIO where ignore = flip const I haven't done any testing with this beyond type checking, but it seems like it could be useful for similar cases to newTVarIO. Has anyone else played with anything similar? -- Chad Scherrer "Time flies like an arrow; fruit flies like a banana" -- Groucho Marx

2 2

Re: [Haskell] FW: A history of Haskell
by Justin Bailey 26 Jan '07

26 Jan '07

On 1/25/07, Simon Peyton-Jones <simonpj(a)microsoft.com> wrote: > Many of you will know that Paul Hudak, John Hughes, Phil Wadler and I have been working on a paper called > > A History of Haskell: being lazy with class > Just wanted to say this paper is excellent, and actually a great tool for learning Haskell. I read it about 2 months after I'd been introduced to Haskell, and it made quite a few things about the language clear to me. It's clearly not a tutorial, but somehow knowing parts of the history made the language clearer. Thanks for your (and all the other authors') effort! Justin

1 0

GHC concurrency runtime "breaks" every 497 (and a bit) days
by Neil Davies 26 Jan '07

26 Jan '07

In investigating ways of getting less jitter in the GHC concurrency runtime I've found the following issue: The GHC concurrency model does all of its time calculations in "ticks" - a tick being fixed at 1/50th of a second. It performs all of these calculations in terms the number of these ticks since the Unix epoch (Jan 1970). Which is stored as an Int - unfortunately this 2^31 bits overflows every 497 (and a bit) days. When it does overflow any threads on the timer queue will either a) run immediately or b) not be run for another 497 days - depending on which way the sign changes. Also any timer calculation that spans the wrapround will not be dealt with correctly. Although this doesn't happen often (the last one was around Sat Sep 30 18:32:51 UTC 2006 and the next one will not be until Sat Feb 9 21:00:44 UTC 2008) I don't think we can leave this sort of issue in the run-time system. Take a look at the definition of getTicksofDay (in base/include/HsBase.h (non-windows) / base/cbits/Win32Utils.c) and getDelay (in base/GHC/Conc.lhs) to understand the details of the issue. I don't mind having a go at the base system to remove this problem - but before I do I would like to canvas some opinions. I'll do that in a separate thread. Neil

3 3

No Derived Read for Unboxed Arrays
by Matthew Bromberg 26 Jan '07

26 Jan '07

I was trying to convert some code from ordinary boxed array to unboxed arrays for performance reasons. Presumably the conversion, according to the GHC documentation should only involve appropriate type signatures (e.g. using UArray instead of Array) and importing Data.Array.Unboxed. However my code ultimately failed because I load a large array saved as a text file using the derived Read, Show mechanism. For some reason there is no Read function for unboxed arrays. Here is a little GHCI session showing the problem: Prelude> :m Data.Array.Unboxed Prelude Data.Array.Unboxed> let e = listArray (0,3) [0 .. 3] :: UArray Int Int Prelude Data.Array.Unboxed> let se = show e Prelude Data.Array.Unboxed> se "array (0,3) [(0,0),(1,1),(2,2),(3,3)]" Prelude Data.Array.Unboxed> let e2 = read se :: UArray Int Int <interactive>:1:9: No instance for (Read (UArray Int Int)) arising from use of `read' at <interactive>:1:9-15 Possible fix: add an instance declaration for (Read (UArray Int Int)) In the expression: read se In the expression: read se :: UArray Int Int In the definition of `e2': e2 = read se :: UArray Int Int However if I try to read it in as boxed arrays, no problem: Prelude Data.Array.Unboxed> let e2 = read se :: Array Int Int Prelude Data.Array.Unboxed> e2 array (0,3) [(0,0),(1,1),(2,2),(3,3)] Is there a reason why this doesn't work? Is it a design choice or does it merely reflect the fact that no one has gotten around to writing a read function for unboxed arrays. I imagine it would be trivial to fix this however.

4 6