October 2007 - Haskell-Cafe

Distributed haskell using Hadoop
by Brad Clow 16 Oct '07

16 Oct '07

I was looking around for a haskell solution for distributing a computation across multiple machines and couldn't find anything that looked current and alive. I found out that the Hadoop project (java based) can interact with binary executables via stdin and stdout. So I have set up a Hadoop cluster of 5 machines, wrapped my Haskell code to accept data via stdin and write results to stdout and successfully executed it on the cluster. I would prefer a more Haskell orientated solution and welcome any suggestions. If not maybe this will be of use to others. Regards brad

3 2

Re: [Haskell] Re: Trying to install binary-0.4
by Simon Marlow 16 Oct '07

16 Oct '07

Claus Reinke wrote: > but calling "split-base" "base" goes directly against all basic > assumptions of all packages depending on "base". The new base will have a new version number. There is no expectation of compatibility when the major version is bumped; but we do have an informal convention that minor version bumps only add functionality, and sub-minor version bumps don't change the API at all. So a package that depends on 'base' (with no upper version bound) *might* be broken in GHC 6.8.1, depending on which modules from base it actually uses. Let's look at the other options: - if we rename base, the package will *definitely* be broken - if the package specified an upper bound on its base dependency, it will *definitely* be broken In the design we've chosen, some packages continue to work without change. Specifying a dependency on a package without giving an explicit version range is a bet: sometimes it wins, sometimes it doesn't. The nice thing is that we have most of our packages in one place, so we can easily test which ones are broken and notify the maintainers and/or fix them. Another reason not to change the name of 'base' is that there would be a significant cost to doing so: the name is everywhere, not just in the source code of GHC and its tools, but wiki pages, documentation, and so on. Yes I know we've changed other names - very little in packaging is clear-cut. Cheers, Simon

8 17

Re: [Haskell-cafe] ANNOUNCE: metaplug
by Axel Mannhardt 16 Oct '07

16 Oct '07

Hello, There is one limitation to this, however. compileCall expects to compile a > dynamic via GHC.dynCompileExpr; > what this means is your resource must be monomorphic (for Typeable to > work.) As of right now, the easiest > way I can see to get around this is to simply define a datatype like such: > > data Plugin { > rsrc :: ... -- your type here > } deriving Typeable I have the same problem, although from a different direction. I am only interested in an eval-myDynamicFunction functionality not general modules, so I did it on a per function basis, using hs-plugins eval. A typeable data wrapper would be fine for me, although I have not yet found a non-ugly way to import needed modules for the plugin, as the plugin source would not be necessarily in the same place as the API. But thats rather an implementation detail. The real problem is, that I do not know, if there is an appropriate way to represent functions with a dynamic parameter count. As of now, I think it would need a bigger amount of hacking than it is worth to get rid of the workarounds I already have, but it would be interesting anyway. > (unsafeCoerce# is an option but there's not a version > of compileCall to support this as of right now. I might add it if it seems > needed.) > As I am relatively new, and since I have not found any introduction what unsafeCoerce is actually capable of (and how), I have ignored this possibility. Could someone give me some pointers? (the most useful thing I found is http://osdir.com/ml/lang.haskell.glasgow.bugs/2005-03/msg00048.html ) The approach I stopped working at was: type TestSpecs = (String, String, String) data FunctionTest = forall a b . (Show a, Show b, Eq b) => FunctionTest (String , [a], (a ->b), [b]) deriving Typeable readFunctionTest :: TestSpecs -> IO (Either [String] (Maybe FunctionTest)) readFunctionTest ( params,fn,predicted) = eval_ str ["FunctionTest"] ["-i"++libDir] [] [libDir] where str = "FunctionTest("++show fn++", "++params++", " ++fn++", "++predicted++")" applyFunctionTest :: FunctionTest -> [Bool ] applyFunctionTest (FunctionTest (_ ,parameters,function,predicted) ) = zipWith (==) (map function parameters) predicted ..what works for general Functions with one Parameter. The wrapper ensures that the function takes one value to produce another value comparable to the other given value. So far, this looks ok to me, I can 'read' a function once and apply it to multiple tests. However, if the function has more than one curried parameter, it has to be extended. I see the following approaches: -Do everything inside the eval (returning a [Bool] for example). That works, and I use it right now, but apart from being ugly, the function has to be read more than once. This might be a non-practical issue though. -Introduce constructors for each parameter count (up to a limit). FunctionTest2 (String, [(a,a')], (a->a'-> b), [b]) applyFunctionTest (FunctionTest2 ...{-this might be generalizeable-} That is a lot more redundant and not as general as the inside-eval approach, but the function has to be read only once. -Convert any functions into ones that take one list/tuple parameter within the eval (assume there is no type ambiguity for simplicity). Partial application is not the goal here anyway and information about the parameter count could be obtained from the "parameter"-parameter or from additional information in the function String itself. But for that a wrapper has to be added to the code inside eval that turns a function with unknown parameter count into the one actually exported. The type of this transformer function could be distinguished at eval compile time, but that means the implementation would have to be included in each eval... uncurry3 fn (a, b, c) = fn a b c uncurryN = --would be defined using template Haskell or recursive data structures or something else I do not know yet --simpler as a list uncurryL fn [] = fn uncurryL fn (x:xs) = uncurryL (fn (fromValue x)) xs -- but then the values would have to be wrapped for type ambiguity: data Value = IntV Int | ... I had a look on some techniques to deal with dynamic parameter count, but they are either decided at runtime (as my understanding of QuickCheck and its recursive data types is) or they do not represent functions with unknown parameter count but only process one at a time (as Printf does). Tell me if I am wrong. So, I end up with three possibilities how to do the job (although I have not tried to implement all of them), but I am unhappy with each. How would one express a type "a->...->b" in Haskell (GHC)? Are you able to express the second approach without the redundant FunctionTestN definitions? I am willing to do ugly things, as long as I can restrict the effects to that particular place. Thanks, Axel

1 0

Of phantom types and type extentions
by Thomas M. DuBuisson 16 Oct '07

16 Oct '07

All, I've been casually developing a PacketBB (i.e. Generalized Manet Packet Format) library in Haskell. I think I have a need to pass state information as a phantom type - I'll step through the issue now. With the 'AddressBlock' (S5.2.1 packetBB draft 8), network addresses are abbreviated as sets of bytes (potentially just one byte each, with a head or tail identical with other addresses). How many bytes are in the set is determined, in part, by the type of address stored (ex: IPv4 or IPv6). Thus, when serializing, I need to provide this information. Saying this again, but in (simplified) code: data NetworkAddress a => AddressBlock a = AddrBlkWire { lenHd :: Word8, hd :: [Word8], lenTl :: Word8, tl :: [Word8], nrAddrs :: Word8, addrs :: [Word8] } | AddrBlkAbstract [a] data (NetworkAddress a) => SomeHigherLevelStruct a = SHLS (AddressBlock a) Word32 Word8 -- length (addrs x) == ("TotalAddressLength" - lenHd - lenTl) * nrAddrs I can think of several ways to convert between AddrBlkWire and ByteStrings: 1) Make separate instance of 'Binary' for each data type element of NetworkAddress. instance Binary (AddressBlock IPv4) where get = ... put = ... instance Binary (AddressBlock IPv6) where get = ... put = ... This solution immediately causes problems with every higher level structure you wish to serialize. For example, now you have to have individual instance for SHLS, you can't do: instance (NetworkAddress a) => Binary (SomeHigherLevelStruct a) where ... 2) You can pass another argument into a custom 'get' routine. I see this as a hack that makes me break a good naming convention. getNetworkAddress :: Int -- bytes per address -> Get NetworkAddress 3) If you don't worry about decoding, only encoding, then an extra field in the data structure can fill the void of an extra argument. Also a hack. I'm hoping someone here has a better solution. Perhaps I am making a mountain out of a mole hill, or perhaps this requires one of those type system extensions I have yet to look hard at. The solution I would want looks like this: class NetworkAddress a where addressByteSize :: a -> Int instance (NetworkAddress a) => Binary (AddressBlock a) where get = do lenH <- get h <- replicateM get (fromIntegral lenH) lenT <- get t <- replicateM get (fromIntegral lenT) nr <- get let addrSize = addressByteSize (undefined :: a) bytes = (addrSize - lenH - lenT) * nr addrs <- replicateM get (fromIntegral bytes) return ... The line 'addrSize = ' is what I don't know how to write. How does one call an instance of a type class without knowing the type at compile time? Thanks, Tom -- "The philosophy behind your actions should never change, on the other hand, the practicality of them is never constant." - Thomas Main DuBuisson

2 1

Re: [Haskell-cafe] do
by Peter Verswyvelen 15 Oct '07

15 Oct '07

jerzy.karczmarczuk(a)info.unicaen.fr wrote: >> Yes, *different approach*. So, there *are* differences. Compilers, >> anyway, >> are special applications. I wanted to see - responding to Brandon - a >> "normal" Haskell program, which does IO without monads, that't all. >> The problem is then when you hide something, you hide. It is possible to >> superpose a kind of monadic framework on unique worlds, files, etc. in >> Clean, but the reverse operation goes beyond my horizons. >> Some examples, anybody? >> Jerzy Karczmarczuk Ah yes, I see what you mean now. I have no idea, I guess only unsafePerformIO will allow you to do something like that... But that will completely break referential transparency. But I really don't know. What do the experts have to say?

4 7

Pixel plotter
by Andrew Coppin 15 Oct '07

15 Oct '07

I write a lot of programs that build bitmapped images. (Some of you may remember the Chaos Pendulum Simulator...) Currently, all of these programs work by writing the final image data to a PPM file on disk. (Because that's the only way I can figure out to do something *useful* with the data!) What I'd *like* to be able to do, in priority order: 1. Display images on the screen. 2. Save images as PNG files. (PPM eats too much space!) 3. Animate images on the screen. 4. With double-buffering. 5. Not break horribly when rendering in multiple threads. As far as I know, all of this is *possible* with Gtk2hs right now - it's just vastly more complex than making a single function call. So what I'd like to do is write a small library which will enable me to do each of the above tasks in 1 function call. However, I'm getting nowhere fast with this. Anybody have any suggestions? PS. I've investigated existing APIs and I'm not seeing anything that looks useful. Lots of support for drawing lines and arcs and so on, but not for plotting individual pixels.

8 9

Re: [Haskell-cafe] do
by jerzy.karczmarczuk＠info.unicaen.fr 15 Oct '07

15 Oct '07

Peter Verswyvelen writes about non-monadic IO, unique "external worlds": > But... isn't this what the Haskell compiler & runtime do internally when > IO monads are executed? Passing the RealWorld "singleton" from "action" to > "action"? I never looked into any Haskell compiler. Chalmers, or York, don't remember, used continuations, this seems a bit different from the Clean approach, but I don't really know the gory details. > To me, no real > difference exists between IO monads and Clean's uniques types; it's just a > different approach to tackle the same problem. Yes, *different approach*. So, there *are* differences. Compilers, anyway, are special applications. I wanted to see - responding to Brandon - a "normal" Haskell program, which does IO without monads, that't all. The problem is then when you hide something, you hide. It is possible to superpose a kind of monadic framework on unique worlds, files, etc. in Clean, but the reverse operation goes beyond my horizons. Some examples, anybody? Jerzy Karczmarczuk

2 1

ANNOUNCE: Haskell Selenium bindings, version 0.2.1
by Aaron Tomb 15 Oct '07

15 Oct '07

Hi, I'm pleased to announce the first external release of a set of Haskell bindings for the Selenium automated web testing system (specifically, the Selenium Remote Control). These bindings make it possible to use Haskell to write test scripts that automatically exercise arbitrary web sites by controlling a web browser. More information on Selenium is available here: http://www.openqa.org/ You can get the tagged release from Hackage: http://hackage.haskell.org/cgi-bin/hackage-scripts/package/ selenium-0.2.1 Or you can keep up with the development version from darcs (which currently has no changes from 0.2.1): darcs get http://code.haskell.org/selenium Many thanks to Don Stewart for code and design contributions. Aaron

1 0

Re: [Haskell-cafe] do
by PR Stanley 15 Oct '07

15 Oct '07

Thanks for the very clear explanation. More questions: What is the role of ">>"? How is ">>" different to ">>="? I am aware that ">>=" is used for sequencing parsers but that's all I know about it. Thanks, Paul At 22:28 13/10/2007, you wrote: >On 10/13/07, PR Stanley <prstanley(a)ntlworld.com> wrote: > > Hi > > "do", what's its role? > > I know a few uses for it but can't quite understand the semantics - > > e.g. do putStrLn "bla bla" > > So, what does do, do? > >In this example, do doesn't do anything. do doesn't do anything to a >single expression (well, I think it enforces that its return value is >a monad...). It's only when you give it multiple expressions that it >"rewrites" them into more formal notation. For example: > > do putStrLn "bla" > putStrLn "blah" > >Will be rewritten into: > > putStrLn "bla" >> putStrLn "blah" > >It introduces a block of "sequential actions" (in a monad), to do each >action one after another. Both of these (since they're equivalent) >mean print "bla" *and then* print "blah". > >do also allows a more imperative-feeling variable binding: > > do line <- getLine > putStr "You said: " > putStrLn line > >Will be rewritten into: > > getLine >>= (\line -> putStr "You said: " >> putStrLn line) > >Looking at the do notation again: execute getLine and bind the return >value to the (newly introduced) variable 'line', then print "You said: >", then print the value in the variable line. > >You can think of the last line in the block as the return value of the >block. So you can do something like: > > do line <- do putStr "Say something: " > getLine > putStr "You said: " > putStrLn line > >In this example it's kind of silly, but there are cases where this is useful. > >Luke

9 17

Hosting of Haskell project
by Magnus Therning 15 Oct '07

15 Oct '07

I recently had reason to do some encoding-related coding and noticed that Haskell was somewhat lacking (I could only find code for base64, on the other hand there are two implementations of it :-). I've almost reached a state where I wouldn't be ashamed of sharing the code so I looked into my options of free hosting. It seems I only have one option for publishing the code: - Request a project on code.haskell.org. I could only find one option for a "homepage" for the project: - Create a page on the Wiki. There seems to be no option when it comes to tracking bugs. :-( I could also not locate any option for publishing haddock pages. :-( Have I missed anything? /M -- Magnus Therning (OpenPGP: 0xAB4DFBA4) magnus＠therning．org Jabber: magnus．therning＠gmail．com http://therning.org/magnus The British have "the perfect temperament to be hackers--technically skilled, slightly disrespectful of authority, and just a touch of criminal behavior". -- Mary Ann Davidson, Oracle's Security Chief

8 12