- Haskell-Cafe - Haskell.org

Windows console
by Andrew Coppin 13 Sep '08

13 Sep '08

When coding in a POSIX-compliant environment, you can usually write special escape codes to the console to change the text colour and so forth. However, this does not work on Windows. (Ignore all references you see to enabling ANSI.SYS in your config.sys file; this applies only to 16-bit MS-DOS programs, *not* to 32-bit Windows programs.) However - to my surprise - apparently the Win32 API provides a set of function calls that do in fact allow 32-bit applications to change the colours of the console on a character-by-character basis. However - wait for it - those *particular* functions aren't in Haskell's System.Win32.Console module. :-( Obviously, I know nothing about C. However, after much hunting around, it turns out that for some reason, GHC appears to ship with a complete set of C header files for the Win32 API. (In other words, the necessary header file for accessing the functions I want is there.) After staggering through the (very unhelpful) FFI language specification, I was able to make it so that I can apparently call these functions from Haskell. I was not, however, able to find any way at all to import the symbolic constants necessary, so I was forced to reading through the source code of the raw C header files to find out what the numeric values of these are (!!!) The long and short of it is, I now have a small Haskell library that enables me to print things in trippy colours on the screen from a normal Haskell CLI program. Yay for me! (Actually, you can also change the title on the window - so you can make your program say "My Fantastic Tool" instead of "C:\Documents and Settings\foo\Haskell\MyTool\MyTool.exe" in the titlebar. Again, this is for a normal Haskell console application with nothing done to it.) Would anybody else be interested in having this code? (Obviously, it's pretty tiny.) For that matter, would the maintainer of System.Win32.Console be interested in adding the necessary dozen lines of code to that module? Actually, now that I think about it, it would be kind of nice to have a magic package that writes out escape codes or calls the Win32 API depending on which platform your program is compiled on - in the style of System.FilePath. I don't know how to do that though... A nice idea, guys?

8 14

ANN: Twidge
by John Goerzen 13 Sep '08

13 Sep '08

Hi folks, I've released Twidge, a command-line Twitter and Identi.ca client, written -- of course -- in Haskell. I'm pretty excited about it, but then hey, I'm biased. Anyhow, it's available from Hackage or from the homepage at: http://software.complete.org/twidge I've also got links to its (11-page!) manual as well as some HOWTOs and FAQs on the wiki at the above URL. Comments, patches welcome. Feel free to reply here or send them to @jgoerzen. -- John

1 0

[Fwd: profiling in haskell]
by Vlad Skvortsov 13 Sep '08

13 Sep '08

Posting to cafe since I got just one reply on beginner@. I was suggested to include more SCC annotations, but that didn't help. The 'serialize' function is still reported to consume about 32% of running time, 29% inherited. However, functions called from it only account for about 3% of time. How do I get more insight into that? Thanks. -- Vlad Skvortsov, vss(a)73rus.com, http://vss.73rus.com

2 5

uuid package on OS X
by Jason Dusek 12 Sep '08

12 Sep '08

The UUID package builds and runs fine on OS X without any need for `e2fsprog` -- just remove the `extra-libs` line from the Cabal file. (Apple put the UUID stuff in `libc`.) Has anyone tried to build it on Windows? -- _jsn |...UUID package...| http://hackage.haskell.org/cgi-bin/hackage-scripts/package/uuid

1 0

Tons of retainers when inserting 611, 756 elements into a Trie
by Denis Bueno 12 Sep '08

12 Sep '08

Dear haskell-cafe, I've got an anagram-finder ("puzzler") that uses a "dictionary" datatype, which in turn uses a trie. In src/hs/Main.hs, I create a new dictionary from a word list (a file containing one word per line) and perform a query on it in order to force it to actually load something from disk. By problem is that though my word list is just shy of 7MB, puzzler occupies nearly 1GB of resident memory as soon as the dictionary is loaded. I'm using GHC 6.8.3 and Gtk2hs 0.9.13 on OS X 10.4.11. To see for yourself, check out the code with git and run it: $ git clone git://github.com/dbueno/puzzler.git $ git checkout -b origin/bsbench $ cabal configure # needs gtk2hs $ cabal build $ ./dist/build/puzzler/puzzler # look at the resident memory Retainer profiling (attachment "puzzler-hr.pdf") shows that `insertWith' on my trie is responsible for most of the memory. As you will see, in `makeDictionary' I try to strictly apply the combining function to combat these retainers. But the problem doesn't go away. More strangely: puzzler has a second executable, puzzler-test, which runs unit tests, including loading the exact same anagram query. However, puzzler-test never uses more than 250MB or so. The dictionary (in module Puzzler.Anagram) is an array of bytestrings paired with a Trie mapping bytestrings to sets of indices in the array. > data Dictionary = Dictionary > { dictWords :: Words > , sortWords :: Trie IntSet } > type Words = Array Int ByteString The Trie is a custom bytestring trie datatype. Each level is mapped to by the character it represents. If that character is a terminal character for some word (Right (t,x)), then that word maps to x. Otherwise no word ends at that character, but there may be longer words, so the trie continues with (Left t). > newtype Trie a = Trie { unTrie :: IntMap (Either (Trie a) (Trie a, a)) } > deriving (Eq, Ord, Show) The following is where _I think_ the error lies (src/hs/Puzzler/Anagram.hs): > -- | Creates an anagram dictionary from a file of words, one per line. The > -- words may be compound, as long as there is one per line. > createDictionary :: FilePath -> IO Dictionary > createDictionary path = (makeDictionary . lines) `liftM` readFile path > > -- | Makes an anagram dictionary from a list of words. > makeDictionary :: [ByteString] -> Dictionary > makeDictionary ws = Dictionary > { dictWords = dw > , sortWords = go end Trie.empty } > where > dw = listArray (0, length ws - 1) ws ; (begin, end) = bounds dw > go i t | seq t $ False = undefined > | i < begin = t > | otherwise = go (i-1) > -- $ Trie.insertWith Set.union (BS.sort (dw!i)) (Set.singleton i) t > $ insertWith' Set.union (BS.sort (dw!i)) (Set.singleton i) t > > insertWith' f k v m | seq k $ False = undefined > insertWith' f k v m = case Trie.lookup k m of > Nothing -> Trie.insert k v m > Just s -> (Trie.insert k $! f v s) m I've been scouring haskell-cafe for similar issues, and all seem to have been solved by a function similar to my `insertWith''. Uncommenting `Trie.insertWith' and commenting out the rest doesn't appear to differ noticeably in memory consumption. The implementation of `Trie.insertWith' is: > insertWith :: (a -> a -> a) -> ByteString -> a -> Trie a -> Trie a > insertWith f bs x t@(Trie m) = case uncons bs of > Nothing -> t > Just (c, cs) -> Trie (Map.alter myAlter (ord c) m) > where > myAlter Nothing = Just $ cons (insertWith f cs x empty, x) > myAlter (Just (Left t)) = Just $ cons (insertWith f cs x t, x) > myAlter (Just (Right (t,x'))) = > Just $ Right (insertWith f cs x t, if isLast then f x x' else x') > > -- If `cs' is empty then `c' is the last char of the word to insert; > -- that is, we're done. > cons (t, x) = if isLast then Right (t, x) else Left t > isLast = BS.null cs Any ideas? Denis

1 0

ANN: vxml (validating xml lib) - proof of concept - need some guidance - bad type level performance (ghc)
by Marc Weber 12 Sep '08

12 Sep '08

Hi @ll. This shouldn't have been an announce yet.. it's more crying for help to get to know how / wether to continue :-) I had an idea: a xml generating library validating the result against a given dtd (to be used in web frameworks etc .. ) After one week of coding I got the result git clone git://mawercer.de/vxml (1) (see README) git hash as of writing : 4dc53 A minimal example looks like this (taken from test.hs): import Text.XML.Validated.TH $( dtdToTypes "dtds/xhtml1-20020801/DTD/xhtml1-strict_onefile.dtd" (XmlIds (Just "-//W3C//DTD XHTML 1.0 Strict//EN") (Just "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd") ) ) main = putStrLn $ xml $ ((html << (head << (title <<< "vxml hello world"))) << (body << (div <<< "PCDATA text" )) ) Which takes about 7 secs to compile on my (fast) 2.5 Ghz machine. Before Oleg telling me about how to use the TypeEq implementation found in HList it took several hours (switch the cabal configure flag to see experience the incredible slow TypeToNat implementation) However type checking the simple small document (2) already takes 35 seconds, duplicating the tags within the body raises compilation time to 74 seconds. (Have a look at test/XmlToQ.hs to automatically create the code generating such a given xml file (its used in run-testcases.hs) The dtd representation used is very easy to understand: (3a) shows allowed subelements of the head tag Seq = sequence Star = zero or more ... (3b) shows the list of allowed attributes for a more commonly used tag such as div the element list (3a) is 324 lines long and there are approximately 15 attributes. Use a line such as putStrLn $ xml $ debugEl div to get this view. All the work is done by class Consume st el r | st el -> r -- result is on of C,CS,R,F class Retry elType st el st' | st el -> st' st = state as given in (3a) el = child to be added (Elem Html_T) or PCDATA C = element consumed, end CS a = element consumed, continue with state a R a = retry with given state (can happen after removing a Star = ()* on a no match ) F a = no match, show failure a instance Retry elType C el C instance Retry elType (CS st) el st instance ( -- retry Consume st el r , Retry elType r el st' ) => Retry elType (R st) el st' Maybe you knowing much more about ghc internals have some more ideas how to optimize? I only came up with a) implement xml HTrue $ xmldoc -- type checking variant xml HFalse $ xmldoc -- unchecked but faster variant so you can run xml HTrue once a day only while having lunch.. ( doubt Maybe you have to take lunch two or more times on intermediate web projects .. ) b) change (A attrType) to attrType only and (Elem e) to e which does not work for the same reason as class TypeEq a b c | a b -> c instance TypeEq a a HTrue instance TypeEq a b HFalse doesn't. Of course a a will match a b as well, but the result is totally different.. So would it be possible to either tell ghc to ignore this happily taking the result of the better matching instance? Or to explicitely tell ghc when an instance matches the way it does in current implementations make it a no match if some constraints can or cannot be satisfied such as this: class TypeEq a b c | a b -> c instance TypeEq a a HTrue instance [ NoMatchOn a b -- if this can be satisfied ignore this instance ] => TypeEq a b HFalse class NoMatchOn a b instance NoMatchOn a a c) Having a reduction rule such as this: If the left side of (Or a b) returnns C (consumed) tell ghc to not evaluate the result of (Consume b element result) which must be done to see wether there is closer instance match (Is this correct ?) Of course not everything is implemented yet.. but except speed I consider it beeing quite usable. He, thanks for reading till the end. Any feedback and suggestions are appreciated. If you have any trouble contact me on irc. I've been using $ ghc --version The Glorious Glasgow Haskell Compilation System, version 6.8.2 Sincerly Marc Weber (1) If you are interested and either don't have git or are not familiar using it I can send you a tarball. (2) <!DOCTYPE html SYSTEM "/tmp/dtd.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" lang="en-US" xml:lang="en-US"> <head> <title>Example 6 - XHTML 1.0 Strict as application/xhtml+xml</title> <meta http-equiv="Content-Type" content="application/xhtml+xml; charset=utf-8" /> <link rel="stylesheet" type="text/css" href="style.css" /> </head> <body> <h1>Example 6 - XHTML 1.0 Strict as application/xhtml+xml</h1> <p> This document is valid XHTML 1.0 Strict served as <code>application/xhtml+xml</code>. </p> <p> This document references CSS rules contained in an external stylesheet via <code>link</code>. </p> <p> Note how the CSS rules for the background are applied in Netscape 7.x, Mozilla, Opera 7 but that Internet Explorer can not display the page at all. </p> <p> <a href="http://validator.w3.org/check/referer"><img src="http://www.w3.org/Icons/valid-xhtml10" alt="Valid XHTML 1.0!" height="31" width="88" /></a> </p> </body> </html> (3a) (Seq (Star (Or (Elem Script_T) (Or (Elem Style_T) (Or (Elem Meta_T) (Or (Elem Link_T) (Elem Object_T)))))) (Or (Seq (Elem Title_T) (Seq (Star (Or (Elem Script_T) (Or (Elem Style_T) (Or (Elem Meta_T) (Or (Elem Link_T) (Elem Object_T)))))) (Query (Seq (Elem Base_T) (Star (Or (Elem Script_T) (Or (Elem Style_T) (Or (Elem Meta_T) (Or (Elem Link_T) (Elem Object_T)))))))))) (Seq (Elem Base_T) (Seq (Star (Or (Elem Script_T) (Or (Elem Style_T) (Or (Elem Meta_T) (Or (Elem Link_T) (Elem Object_T)))))) (Seq (Elem Title_T) (Star (Or (Elem Script_T) (Or (Elem Style_T) (Or (Elem Meta_T) (Or (Elem Link_T) (Elem Object_T))))))))))) (3b) (HCons (A Lang_A) (HCons (A Xml:lang_A) (HCons (A Dir_A) (HCons (A Id_A) (HCons (A Profile_A) HNil)))))

2 7

Text.JSON idiomatic use
by ntupel 12 Sep '08

12 Sep '08

As a follow up to my previous JSON serialization post I came up with a first draft of some simple record type serialization/deserialization. What I would like to know is, whether this is the right approach or what better ways there are to make a custom data type an instance of class JSON. Any chance to reduce the amount of boilerplate required to do this? I would be grateful for any feedback (also general style comments are much appreciated). Many thanks! module Test where import Text.JSON data Message = Error { event :: String, channel :: String, id :: String, cause :: String, message :: String} | Join { event :: String, channel :: String, id :: String, name :: String} | Leave { event :: String, channel :: String, id :: String, really :: Bool} deriving (Eq, Show, Read) asJSString :: String -> JSValue asJSString = JSString . toJSString asString :: JSValue -> String asString (JSString s) = fromJSString s asBool :: JSValue -> Bool asBool (JSBool b) = b showErrorJSON, showJoinJSON, showLeaveJSON :: Message -> JSValue showErrorJSON (Test.Error evt cha id cau msg) = showJSON $ toJSObject [("event", evt), ("channel", cha), ("id", id), ("cause", cau), ("message", msg)] showJoinJSON (Join evt cha id nme) = showJSON $ toJSObject [("event", evt), ("channel", cha), ("id", id), ("name", nme)] showLeaveJSON (Leave evt cha id rly) = showJSON $ toJSObject [("event", asJSString evt), ("channel", asJSString cha), ("id", asJSString id), ("really", JSBool rly)] createMessage, readErrorJSON, readJoinJSON, readLeaveJSON :: [(String, JSValue)] -> Maybe Message readErrorJSON xs = do evt <- lookup "event" xs cha <- lookup "channel" xs id <- lookup "id" xs cau <- lookup "cause" xs msg <- lookup "message" xs Just (Test.Error (asString evt) (asString cha) (asString id) (asString cau) (asString msg)) readJoinJSON xs = do evt <- lookup "event" xs cha <- lookup "channel" xs id <- lookup "id" xs nme <- lookup "name" xs Just (Join (asString evt) (asString cha) (asString id) (asString nme)) readLeaveJSON xs = do evt <- lookup "event" xs cha <- lookup "channel" xs id <- lookup "id" xs rly <- lookup "really" xs Just (Leave (asString evt) (asString cha) (asString id) (asBool rly)) createMessage obj = do evt <- lookup "event" obj case asString evt of "/error" -> readErrorJSON obj "/me/add" -> readJoinJSON obj "/me/remove" -> readLeaveJSON obj _ -> Nothing instance JSON Message where showJSON x@(Test.Join _ _ _ _) = showJoinJSON x showJSON x@(Test.Leave _ _ _ _) = showLeaveJSON x showJSON x@(Test.Error _ _ _ _ _) = showErrorJSON x readJSON (JSObject o) = case createMessage . fromJSObject $ o of Just m -> Ok m Nothing -> Text.JSON.Error "Parsing failed." readJSON _ = Text.JSON.Error "Records must be JSObjects"

1 0

template haskell -- include a file?
by Jason Dusek 12 Sep '08

12 Sep '08

I'd like to use template Haskell to include as a string in a Haskell file. How do I do it? Is there any lengthy documentation with examples for Template Haskell? The pages linked to from `haskell.org` are a little sparse. -- _jsn

3 4

Haskell and Java
by Maurício 12 Sep '08

12 Sep '08

Hi, I use Haskell, and my friends at work use Java. Do you think it could be a good idea to use Haskell with Java, so I could understand and cooperate with them? Is there a a Haskell to Java compiler that's already ready to use? Thanks, Maurício

14 18

JSON serialization/deserialization
by ntupel＠googlemail.com 12 Sep '08

12 Sep '08

Dear list members, I try to use Text.JSON (http://hackage.haskell.org/cgi-bin/hackage-scripts/package/json) to serialize and deserialize record types. As I understand it, the data types need to be instances of class JSON. However I have difficulties to come up with a nice implementation of showJSON and readJSON. For example: module Test where import Text.JSON data Record = One { field1 :: String, field2 :: String } | Two { field :: String } deriving (Eq, Ord, Show, Read) showJSON (One x y) = toJSObject [("field1", x), ("field2", y)] showJSON (Two x) = toJSObject [("field", x)] readJSON x = -- ??? This lacks the instance declaration itself but my problem is more fundamental. How can I write the readJSON method? In general how would one approach the problem of serialization/deserialization? I am puzzled that the type classes Show and Read can be derived automatically. I can not think of any way to do this. Thanks for your help!

2 1