- Haskell-Cafe - Haskell.org

Text.JSON and utf8
by Martin Hilbig 16 Feb '13

16 Feb '13

hi, tl;dr: i propose this patch to Text/JSON/String.hs and would like to know why it is needed: @@ -375,7 +375,7 @@ where go s1 = case s1 of - (x :xs) | x < '\x20' || x > '\x7e' -> '\\' : encControl x (go xs) + (x :xs) | x < '\x20' -> '\\' : encControl x (go xs) ('"' :xs) -> '\\' : '"' : go xs ('\\':xs) -> '\\' : '\\' : go xs (x :xs) -> x : go xs i recently stumbled upon CouchDB telling me i'm sending invalid json. i basically read lines from a utf8 file with german umlauts and send them to CouchDB using Text.JSON and Database.CouchDB. $ file lines.txt lines.txt: UTF-8 Unicode text lets take 'ö' as an example. i use LANG=de_DE.utf8 ghci tells > 'ö' '\246' > putChar '\246' ö > putChar 'ö' ö > :m + Text.JSON Database.CouchDB > runCouchDB' $ newNamedDoc (db "foo") (doc "bar") (showJSON $ toJSObject [("test","ö")]) *** Exception: HTTP/1.1 400 Bad Request Server: CouchDB/1.2.1 (Erlang OTP/R15B03) Date: Mon, 11 Feb 2013 13:24:49 GMT Content-Type: text/plain; charset=utf-8 Content-Length: 48 Cache-Control: must-revalidate couchdb log says: Invalid JSON: {{error,{10,"lexical error: invalid bytes in UTF8 string.\n"}},<<"{\"test\":\"<F6>\"}">>} this is indeed hex ö: > :m + Numeric > putChar $ toEnum $ fst $ head $ readHex "f6" ö if i apply the above patch and reinstall JSON and CouchDB the doc creation works: > runCouchDB' $ newNamedDoc (db "db") (doc "foo") (showJSON $ toJSObject [("test", "ö")]) Right someRev but i dont get back the ö i expected: > Just (_,_,x) <-runCouchDB' $ getDoc (db "foo") (doc "bar") :: IO (Maybe (Doc,Rev,JSObject String)) > let Ok y = valFromObj "test" =<< readJSON x :: Result String > y "\195\188" > putStrLn y Ã¼ apperently with curl everything works fine: $ curl localhost:5984/db/foo -XPUT -d '{"test": "ö"}' {"ok":true,"id":"foo","rev":"someOtherRev"} $ curl localhost:5984/db/foo {"_id":"bars","_rev":"someOtherRev","test":"ö"} so how can i get my precious ö back? what am i doing wrong or does Text.JSON need another patch? another question: why does encControl in Text/JSON/String.hs handle the cases x < '\x100' and x < '\x1000' even though they can never be reached with the old predicate in encJSString (x < '\x20') finally: is '\x7e' the right literal for the job? thanks for reading have fun martin

3 2

What magic has the new parallel io manager done to improve performance?
by yi huang 16 Feb '13

16 Feb '13

I'm curious about the design and trade offs in IO manager, I can point out two changes that should boost performance: 1. Run an IO manager thread on each capability. 2. Use ONESHOT flag to save a system call. I wonder is there anything else? It must be interesting. -- http://yi-programmer.com/

1 1

how to release (on hackage) a package that needs foreign libs?
by Johannes Waldmann 16 Feb '13

16 Feb '13

What is the recommended approach for publishing (on hackage) a package that depends on external libraries? In my case, the current version of http://hackage.haskell.org/package/satchmo (sources at https://github.com/jwaldmann/satchmo) depends on minisat-haskell-bindings (in Haskell), minisat-c-bindings (in C), and minisat (C++), all from https://github.com/niklasso/ My guess is that this would require that the external libraries are packaged (deb, rpm?), then hackage gods install them, and only then, my package will be built (and docs will be shown)? - J.W.

1 0

optimization of recursive functions
by Andrew Polonsky 16 Feb '13

16 Feb '13

Hello, Is there any difference in efficiency between these two functions, when compiled with all optimizations? map f [] = [] map f (a:as) = f a : map f as and map f x = map' x where map' [] = [] map' (a:as) = f a : map' as Thanks, Andrew

3 3

ANN: rss2irc-1.0, announces RSS/Atom feed updates to IRC
by Simon Michael 15 Feb '13

15 Feb '13

I'm pleased to announce a new release of rss2irc, the software behind hackagebot on #haskell. rss2irc is an IRC bot that polls an RSS or Atom feed and announces updates to an IRC channel, with options for customizing output and behavior. It aims to be an easy-to-use, reliable, well-behaved bot. Release notes: 1.0 (2013/2/15) New: * more robust item detection and duplicate announcement protection, with simpler options * easier irc address syntax, drop -p/--port option * can poll urls with semicolon parameter separator (eg darcsweb's) * can poll https feeds * can poll from stdin (-) * can poll a file containing multiple copies of a feed (eg for testing) * can announce item urls containing percent * `--cache-control` option sets a HTTP Cache-Control header * `--use-actions` announces with CTCP ACTIONs (like the /me command) Fixed: * updated for GHC 7.6 & current libs * initialises http properly on microsoft windows * builds threaded and optimised by default * thread and error handling is more robust, eg don't ignore exceptions in the irc writer thread * no longer adds stray "upload:" to IRC messages * renamed --dupe-descriptions to `--allow-duplicates` * dropped --debug flag * new item detection and announcing is more robust * announcements on console are clearer * a simulated irc connection is not logged unless --debug-irc is used Best, -Simon

1 0

(+) on two lists ?
by sheng chen 15 Feb '13

15 Feb '13

Hi, I was puzzled by the following little program. sum' [] = [] sum' (x:xs) = x + sum' xs I thought the GHC type checker will report a type error. However, the type checker accepts this program and gives the type Num [a] => [[a]] -> [a] When I add type annotation to the program sum' :: Num [a] => [[a]] -> [a] sum' [] = [] sum' (x:xs) = x + sum' xs The GHC asks me to add FlexibleContexts language extension. I would appreciate explanation on this issue. Thanks, Sheng

5 4

How to quantify the overhead?
by Daryoush Mehrtash 15 Feb '13

15 Feb '13

Before I write the code I like to be able to quantify my expected result. But I am having hard time quantifying my expected result for alternative approaches in Haskell. I would appreciate any comments from experienced Haskell-ers on this problem: Suppose I have a big list of integers and I like to find the first two that add up to a number, say 10. One approach is to put the numbers in the map as I read them and each step look for the 10-x of the number before going on to the next value in the list. Alternatively, I am trying to see if I it make sense to covert the list to a series of computations that each is looking for the 10-x in the rest of the list (running them in breath first search). I like to find out sanity of using the Applicative/Alternative class's (<|>) operator to set up the computation. So each element of the list (say x) is recursively converted to a computation to SearchFor (10 - x) that is (<|>)-ed with all previous computation until one of the computations returns the pair that add up to 10 or all computation reach end of the list. Does the approach make any sense? What kind of overhead should I expect if I convert the numbers to a computation on the list? In one case I would have a number in a map and in the other I would have a number in a computation over a list. How do I quantify the overheads? Thanks, Daryoush

2 1

python under attack. ??Haskell??
by Rustom Mody 15 Feb '13

15 Feb '13

Please see http://pyfound.blogspot.in/2013/02/python-trademark-at-risk-in-europe-we.ht… I cannot say that I understand whats really going on The one thing that I get is that there was some minor neglect a decade or more ago. <http://blog.languager.org>Maybe there are things that Haskell needs to beware of?

1 0

performance question
by Nicolas Bock 15 Feb '13

15 Feb '13

Hi list, I wrote a script that reads matrix elements from standard input, parses the input using a regular expression, and then bins the matrix elements by magnitude. I wrote the same script in python (just to be sure :) ) and find that the python version vastly outperforms the Haskell script. To be concrete: $ time ./createMatrixDump.py -N 128 | ./printMatrixDecay real 0m2.655s user 0m2.677s sys 0m0.095s $ time ./createMatrixDump.py -N 128 | ./printMatrixDecay.py - real 0m0.445s user 0m0.615s sys 0m0.032s The Haskell script was compiled with "ghc --make printMatrixDecay.hs". Could you have a look at the script and give me some pointers as to where I could improve it, both in terms of performance and also generally, as I am very new to Haskell. Thanks already, nick

14 34

When the gauge depends on what it measures
by Alexander Bernauer 14 Feb '13

14 Feb '13

Hi, I am currently working on benchmarking the pretty library. Pretty itself has no non-trivial dependencies. But criterion and language-c, which I use to perform the benchmarks, directly and/or indirectly depend on pretty. Everything is configured in one Cabal file via library and benchmark sections. Now, whenever I modify pretty and want to rebuild the bench executable, things become messy. Either I rebuild criterion et al. as well, or the bench executable ends up with two different pretty version built in - if this is even possible. Until now I could not find a robust workflow for this scenario. At some point pretty "shadows" itself and the build fails (c.f. my previous caffee post). What is the right way to handle this scenario? Confused greetings, Alex

4 3