June 2007 - Haskell-Cafe

Sneaking haskell in the workplace -- cleaning csv files
by Jim Burton 17 Jun '07

17 Jun '07

I need to remove newlines from csv files (within columns, not at the end of entire lines). This is prior to importing into a database and was being done at my workplace by a java class for quite a while until the files processed got bigger and it proved to be too slow. (The files are up to ~250MB at the moment) It was rewritten in PL/SQL, to run after the import, which was an improvement, but it still has our creaky db server thrashing away. (You may have lots of helpful suggestions in mind, but we can't clean the data at source and AFAIK we can't do it incrementally because there is no timestamp or anything on the last change to a row from the legacy db.) We don't need a general solution - if a line ends with a delimiter we can be sure it's the end of the entire line because that's the way the csv files are generated. I had a quick go with ByteString (with no attempt at robustness etc) and although I haven't compared it properly it seems faster than what we have now. But you can easily make it faster, surely! Hints for improvement please (e.g. can I unbox anything, make anything strict, or is that handled by ByteString, is there a more efficient library function to replace the fold...?). module Main where import System.Environment (getArgs) import qualified Data.ByteString.Char8 as B --remove newlines in the middle of 'columns' clean :: Char -> [B.ByteString] -> [B.ByteString] clean d = foldr (\x ys -> if B.null x || B.last x == d then x:ys else (B.append x $ head ys):(tail ys)) [] main = do args <- getArgs if length args < 2 then putStrLn "Usage: crunchFile INFILE OUTFILE [DELIM]" else do bs <- B.readFile (args!!0) let d = if length args == 3 then head (args!!2) else '"' B.writeFile (args!!1) $ (B.unlines . clean d . B.lines) bs Thanks, Jim -- View this message in context: http://www.nabble.com/Sneaking-haskell-in-the-workplace----cleaning-csv-fil… Sent from the Haskell - Haskell-Cafe mailing list archive at Nabble.com.

7 19

dangerous inlinePerformIO in Data.Binary(?)
by Udo Stenzel 17 Jun '07

17 Jun '07

Greetings, I was trying to understand the magic inside Data.Binary, and found two somewhat suspicious uses of inlinePerformIO, which imho has a far too innocuous name: | toLazyByteString :: Builder -> L.ByteString | toLazyByteString m = S.LPS $ inlinePerformIO $ do | buf <- newBuffer defaultSize | return (runBuilder (m `append` flush) (const []) buf) Why is this safe? Considering the GHC implementation of IO, isn't there a real danger that 'newBuffer defaultSize' is floated out and therefore every invocation of 'toLazyByteString' starts out with the same buffer? Isn't that exactly the reason why unsafePerformIO and runST are declared NOINLINE? The other occurence is: | unsafeLiftIO :: (Buffer -> IO Buffer) -> Builder | unsafeLiftIO f = Builder $ \ k buf -> inlinePerformIO $ do | buf' <- f buf | return (k buf') which might be safe, since 'f buf' cannot float out of the lambda which binds 'buf', but still, all this stuff is inlined, something constant might get plugged in the place of buf and the result might be floated out to give us an even harder to find Heisenbug. Am I missing something and this is actually safe? If not, what can be done to avoid such errors? I'd really hate to find building blocks that crumble under pressure in standard libraries... -Udo

4 5

How to read -ddump-hi output (and other phases) from GHC
by Alistair Bayley 17 Jun '07

17 Jun '07

I'm collecting links to documents describing some of GHC's optimisation output flags, like -ddump-hi and -ddump-simpl. I'll add stuff to this wiki page: http://www.haskell.org/haskellwiki/Performance/GHC So far I've found little bits in the GHC manual, like: 4.16.3. How to read Core syntax http://www.haskell.org/ghc/docs/latest/html/users_guide/options-debugging.h… (and the Encoding module has the actual rules for the Unique names) http://darcs.haskell.org/ghc/compiler/utils/Encoding.hs 6.2. Faster: producing a program that runs quicker http://www.haskell.org/ghc/docs/latest/html/users_guide/faster.html (see "How do I find out a function's strictness?") Any other hints? Alistair

5 4

Re: Yaham - Yet Another HAskell Mode for GNU Emacs
by Pete Kazmier 17 Jun '07

17 Jun '07

Ian Zimmerman <itz(a)madbat.mine.nu> writes: > It has a different focus than the Moss haskell-mode that's normally > used nowadays. Yaham strives, first and foremost, to integrate well > with the rest of Emacs and respect the Emacs ecosystem. Could you elaborate on this? Perhaps an itemized list of things that are different? Thanks, Pete

1 0

lists with mixed types
by Anatoly Yakovenko 17 Jun '07

17 Jun '07

If I define something like this: data Bar = Bi Int | Bf String deriving Show data Foo = Fi Int | Fs Float deriving Show func::Foo -> Bar func (Fi xx) = Bi xx func (Fs ff) = Bf (show ff) I can do: > map func [(Fi 1), (Fs 2.0)] [Bi 1,Bf "2.0"] but what i really want to do is just do map func [1, 2.0] [1, "2.0"] I understand that this is impossible in haskell, but why cant the compiler generate the Foo and Bar data types for me and just require that i have a func defined for Int -> Int and Float -> String?

2 1

Xlib: problem creating a window with override_redirect set
by Andrea Rossato 16 Jun '07

16 Jun '07

Hi, I'm trying, without success, to create a window with the attribute override_redirect set to True (this way the window manager should not take care of it). Obviously with Xlib (X11-1.2.2). No meter how I try I seem not to be able to get there. In test1 I try with the correct method (createWindow), but I get an error message: X Error of failed request: BadMatch (invalid parameter attributes) Major opcode of failed request: 1 (X_CreateWindow) Serial number of failed request: 7 Current serial number in output stream: 9 With test2 I can open the window, but I'm not able to set that attribute. Any help would be greatly appreciated. TIA. Andrea the example code: module Main where import Graphics.X11.Xlib import Graphics.X11.Xlib.Misc import Graphics.X11.Xlib.Extras import Control.Concurrent import Data.Bits main = test2 test1 = do dpy <- openDisplay "" let dflt = defaultScreen dpy rootw <- rootWindow dpy dflt win <- mkWindow dpy (defaultScreenOfDisplay dpy) rootw 0 0 100 100 mapWindow dpy win sync dpy True threadDelay $ 2 * 1000000 mkWindow dpy scr rw x y h w = do let attrmask = cWOverrideRedirect visual = defaultVisualOfScreen scr attributes <- allocaSetWindowAttributes (\s -> return $ s ) set_override_redirect attributes True window <- createWindow dpy rw x y -- x, y w h -- width, height 1 -- border_width 1 inputOutput visual attrmask attributes putStrLn "Done!" return window test2 = do dpy <- openDisplay "" let dflt = defaultScreen dpy rootw <- rootWindow dpy dflt win <- createSimpleWindow dpy rootw 0 0 100 100 1 0x000000 0xFFFFFF mapWindow dpy win sync dpy True threadDelay $ 2 * 1000000 wa <- getWindowAttributes dpy win allocaXEvent $ \ev -> do setEventType ev configureNotify setConfigureEvent ev win win 100 100 20 20 1 win True sendEvent dpy win False cWOverrideRedirect ev --nextEvent dpy ev getEvent ev sync dpy True a <- getWindowAttributes dpy win putStrLn $ show (wa_override_redirect a)

1 2

Darcs on Solaris x86
by Rich Collins 16 Jun '07

16 Jun '07

I was able to install the ghc binary for solaris x86, but darcs fails on configure: configure:2718: ghc -o conftest conftest.hs ld: fatal: symbol `GHC_ZCCReturnable_static_info' in file /opt/local/ lib/ghc-6.6.1/libHSrts.a(PrimOps.o): section [1] .text: size 8212: symbol (address 0x2014, size 4) lies o utside of containing section ld: fatal: library -lgmp: not found ld: fatal: File processing errors. No output written to conftest collect2: ld returned 1 exit status

3 3

embedded build language?
by Greg Fitzgerald 16 Jun '07

16 Jun '07

Has anyone embedded a build language in Haskell? Something like Rake<http://rake.rubyforge.org/>is to Ruby, but in Haskell or any statically-typed functional language. Thanks, Greg

3 4

Literate Priority Queue, plus question
by Michael T. Richter 16 Jun '07

16 Jun '07

I'm trying my hand at making an improved, more efficient, Sieve of Eratosthenes implementation based on Melissa O'Neil's paper (http://www.cs.hmc.edu/~oneill/papers/Sieve-JFP.pdf) to augment the inefficient not-Sieve I've documented at http://en.literateprograms.org/Sieve_of_Eratosthenes_(Haskell). To get the most efficient version that she outlines in her paper, I've had to implement a priority queue. Being a lazy^Wsmart programmer, I hunted for an implementation of a priority queue and found one at http://www.haskell.org/haskellwiki/Haskell_Quiz/Astar/Solution_Dolio which I promptly nicked and modified to suit the interface Melissa's code requires. I then documented it exhaustively. The output of this can be found at http://en.literateprograms.org/Priority_Queue_(Haskell). (I'd appreciate any vetting people could make -- I did some minimal testing of my own, but some more experienced eyes will likely spot edge cases I'm not able to at this point.) The vetting, however, isn't my main point. My main point is the priority queue as found on the Haskell Wiki. While I was explaining the code of my priority queue, I spotted something odd: although the queue is structured, essentially, as a binary tree (Branch key value leftNode rightNode), I could see no operations at all that ever put anything in the right node. Ever. At all. Thinking this was because of the weird interface I put on the priority queue, I went back to look at the place I stole it from. There, too, I could see no operations at all that put anything into the right node. This bothers me a bit (to put it mildly) because it seems I'm carrying around this one data element that's perpetually set to "Nil" and that I'm basically now having my "priority queue" devolve, in effect, into a sorted list. Am I missing something obvious? If so, what is it? If not, should I just bite the bullet and change my priority queue implementation into a sorted list or is there a way to actually use a binary tree there? -- Michael T. Richter <ttmrichter(a)gmail.com> (GoogleTalk: ttmrichter(a)gmail.com) Theory is knowledge that doesn't work. Practice is when everything works and you don't know why. (Hermann Hesse)

4 3

Haskell record extension?
by bf3＠telenet.be 16 Jun '07

16 Jun '07

I'm learning Haskell. I was surprised that the following example did not compile: data Vector2 = Vector2 { x :: Float, y :: Float } data Vector3 = Vector3 { x :: Float, y :: Float, z :: Float } error: "Multiple declarations of `Main.x'" This seems to be a known issue and if I understand it correctly it has to do with the fact that Haskell does not support function general overloading ala C++. One either has to use type classes, or put each record in a separate module and fiddle with the qualified namespaces. I read multiple papers with proposals to fix this, but does GHC implement any of these? Thanks, Peter

5 6