- Haskell-Cafe - Haskell.org

Re: Patch for library (new feature) - how to deal with it
by Maciej Piechotka 26 Mar '10

26 Mar '10

On Fri, 2010-03-26 at 17:06 +0000, Duncan Coutts wrote: > On Fri, 2010-03-26 at 16:50 +0000, Maciej Piechotka wrote: > > On Fri, 2010-03-26 at 13:29 +0000, Duncan Coutts wrote: > > > On Fri, 2010-03-26 at 02:11 +0000, Maciej Piechotka wrote: > > > > Hello, > > > > > > > > I have written patch (attached) which introduce new API for zlib - > > > > personally I wrote it to have easy implementation of > > > > compression/decompression in iteratee (example file - no yet working > > > > -attached). > > > > > > > > Earlier version of patch I tried to sent to maintainer of zlib. However > > > > I received no response (possibly I get into spam folder by helpful > > > > spamassassin or something like that). I definitely don't want to fork > > > > this library - especially that it is in haskell platform. > > > > > > Ivan is right :-) > > > > > > > I understand. However given the last update of zlib it could be that the > > address is no longer valid. > > > > > I've discussed iterator approaches with zlib before. Tibbe and I > > > discussed it in the context of web servers that want precise control > > > over outputting blocks. > > > > > > I'm not happy however that something that is essentially pure > > > (compression/decompression) should have to have be in IO just to fit the > > > iterator pattern. That seems wrong. I could tolerate ST however. > > > > > > > Well. Iteration can be made pure again iff: > > 1. It will be possible to copy the z_stream. Currently it is not > > possible with zlib. > > It is in principle possible, however it is rather expensive. You would > not want to copy it in practise. > I know. It is a list why I put it in IO monad. (The same for point 3) > > 2. It is deal with great care in iteration. It is possible to use safely > > unsafePerformIO and seq. Iteratee can be pure as it have no copy of the > > context. Existsing compress/decompress functions as well are 'safe' for > > the same reason. > > I'm not quite sure I follow what you're referring to. > While the iterating API is in IO monad you can have something like. > > > I think that it is possible for you to construct the impure interface > > > you want on top of the existing pure interface, using threads and MVars. > > > It is a general pattern, to transform a lazy [a] -> [a] function into an > > > imperative "push block / pull block" interface. You connect up the lazy > > > function to an input channel and an output MVar (running in a new > > > thread). Then externally you push blocks into the input channel and wait > > > for blocks on the output channel. If benchmarks prove that to have too > > > high an overhead then we might consider doing something lower level. > > > > > > > 1. That was my first approach. It was so horribly complicated that I > > gave up. Especially if you wanted any level of error handling. > > On the plus side you would only have to do it once and it would let you > wrap any lazy [a] -> [a] function into the iterator stuff. > If you consider the error handling and/or trailing data it is not so nice and for sure not general. > > 3. If anyone is going to write such function - I'm fine. I just chosen > > what I believe it is a simpler and more correct approach. > > I understand it was easier to write it that way, however it is not a > nice general API to expose from the zlib package I tried to be as close as possible to Data.Digest.Pure.MD5[1]. However due to mentioned above problems with z_stream it cannot. > and it messes with the > layering of the library. > Could you explain? > > > Another alternative that I would be satisfied with would be a lower > > > level interface using the ST monad. That should allow the existing pure > > > interface to be implemented safely on top, and hopefully would allow the > > > kind of impure interface you're after. > > > > > > > As the function have side effects I have chosen IO. > > Right but they only have local side effects. As such IO is overkill and > it forces users to use unsafePerformIO/unsafeIOToST which means they > need to know about the library internals to know if that is safe. > If they would be prefixed with unsafePerformIO/unsafeIOToST user still would have to know something about internals of library but he would have no visual clue. > > The patch was intended as a low-level interface for specific > > applications. > > Sure, but I'm convinced that it is possible to make a low level ST > interface that is not ugly and that the rest of the library can be based > upon too. > The rest of library can be based onto this (if you want I can prove it by providing patch). Also I'm not quite sure what you mean by ugly. > Duncan > Regards [1] http://hackage.haskell.org/packages/archive/pureMD5/1.1.0.0/doc/html/Data-D…

1 0

Re: Patch for library (new feature) - how to deal with it
by Maciej Piechotka 26 Mar '10

26 Mar '10

On Fri, 2010-03-26 at 13:29 +0000, Duncan Coutts wrote: > On Fri, 2010-03-26 at 02:11 +0000, Maciej Piechotka wrote: > > Hello, > > > > I have written patch (attached) which introduce new API for zlib - > > personally I wrote it to have easy implementation of > > compression/decompression in iteratee (example file - no yet working > > -attached). > > > > Earlier version of patch I tried to sent to maintainer of zlib. However > > I received no response (possibly I get into spam folder by helpful > > spamassassin or something like that). I definitely don't want to fork > > this library - especially that it is in haskell platform. > > Ivan is right :-) > I understand. However given the last update of zlib it could be that the address is no longer valid. > I've discussed iterator approaches with zlib before. Tibbe and I > discussed it in the context of web servers that want precise control > over outputting blocks. > > I'm not happy however that something that is essentially pure > (compression/decompression) should have to have be in IO just to fit the > iterator pattern. That seems wrong. I could tolerate ST however. > Well. Iteration can be made pure again iff: 1. It will be possible to copy the z_stream. Currently it is not possible with zlib. 2. It is deal with great care in iteration. It is possible to use safely unsafePerformIO and seq. Iteratee can be pure as it have no copy of the context. Existsing compress/decompress functions as well are 'safe' for the same reason. 3. It is implemented in pure haskell. > Secondly we cannot have the zlib package depend on the iteratee package. > Though we can expose interfaces that allow you to construct the iterator > stuff. > Hmm. Have you seen the patch? It does not change any dependencies. It just adds 6 functions and 3 data structures for API (+ helper functions). It DOES NOT remove any currently existing function (or even change them). it is just additional interface defined. It could be a separate package if it hasn't use Stream. I have never imagined to add dependency of iteratee. > I think that it is possible for you to construct the impure interface > you want on top of the existing pure interface, using threads and MVars. > It is a general pattern, to transform a lazy [a] -> [a] function into an > imperative "push block / pull block" interface. You connect up the lazy > function to an input channel and an output MVar (running in a new > thread). Then externally you push blocks into the input channel and wait > for blocks on the output channel. If benchmarks prove that to have too > high an overhead then we might consider doing something lower level. > 1. That was my first approach. It was so horribly complicated that I gave up. Especially if you wanted any level of error handling. 2. To fully handle the iterator API the decompress would have to be: ... -> (L.ByteString, L.ByteString) Where first is decompressed string and second a rest of input. 3. If anyone is going to write such function - I'm fine. I just chosen what I believe it is a simpler and more correct approach. > Another alternative that I would be satisfied with would be a lower > level interface using the ST monad. That should allow the existing pure > interface to be implemented safely on top, and hopefully would allow the > kind of impure interface you're after. > As the function have side effects I have chosen IO. The patch was intended as a low-level interface for specific applications. If application ensure that computation is done in proper way it may convert it into ST/pure easily. I think that adding unsafePerformIO/unsafeIOToST misses the point. If computation is pure it can probably use compress/decompress. If not then IO/ST monad does not impose additional restrictions > Duncan > Regards PS. I would like to underline that it DOES NOT add dependency on iteratee or any other package and it DOES NOT intend to replace/change an existing API. It is a simply new even-lower-level API for special situation like iteratee (web servers etc.).

1 0

Patch for library (new feature) - how to deal with it
by Maciej Piechotka 26 Mar '10

26 Mar '10

Hello, I have written patch (attached) which introduce new API for zlib - personally I wrote it to have easy implementation of compression/decompression in iteratee (example file - no yet working -attached). Earlier version of patch I tried to sent to maintainer of zlib. However I received no response (possibly I get into spam folder by helpful spamassassin or something like that). I definitely don't want to fork this library - especially that it is in haskell platform. What should I do with the patch? Regards

3 2

Takusen sqlite3 insert is very slow
by Vasyl Pasternak 26 Mar '10

26 Mar '10

Hi Cafe, I have another problem, please look at code: storeInDb = withSession (connect "test.db") (do execDDL (sql "create table x (y int)") forM_ ([1..10000] :: [Int]) (\x -> do execDML (cmdbind ("insert into x (y) values (?);") [bindP x]) return ())) This code runs 16 seconds which is very slow for this simple task. RTS output is below. After profiling this program I found that 85% of its time it spends in 'Database.Sqlite.SqliteFunctions.stmtFetch'. Currently I don't know how to make it faster, maybe anyone had this problem later? HDBC inserts very fast, so this is not sqlite error. RTS output: 156,195,264 bytes allocated in the heap 591,280 bytes copied during GC 3,952 bytes maximum residency (1 sample(s)) 10,968 bytes maximum slop 1 MB total memory in use (0 MB lost due to fragmentation) Generation 0: 295 collections, 0 parallel, 0.00s, 0.01s elapsed Generation 1: 1 collections, 0 parallel, 0.00s, 0.02s elapsed INIT time 0.00s ( 0.03s elapsed) MUT time 0.87s ( 16.09s elapsed) GC time 0.00s ( 0.02s elapsed) RP time 0.00s ( 0.00s elapsed) PROF time 0.00s ( 0.00s elapsed) EXIT time 0.00s ( 0.00s elapsed) Total time 0.88s ( 16.14s elapsed) %GC time 0.5% (0.1% elapsed) Alloc rate 179,111,917 bytes per MUT second Productivity 99.5% of total user, 5.4% of total elapsed Regards, Vasyl

4 4

Windows version of Haskell Platform containing ghc 6.12
by Han Joosten 26 Mar '10

26 Mar '10

Last weekend a new version of the haskell platform has been released. I was expecting that also for windows the ghc 6.12 would be in it. However, when I follow the link: http://hackage.haskell.org/platform/windows.html I still get the 'old' version (2009.2.0.2) which contains ghc 6.10 Is this intentional? If so, is there anything to say when a windows version of the platform will become available that contains ghc 6.12? Thanks! Han -- View this message in context: http://old.nabble.com/Windows-version-of-Haskell-Platform-containing-ghc-6.… Sent from the Haskell - Haskell-Cafe mailing list archive at Nabble.com.

2 1

sendfile leaking descriptors on Linux?
by Bardur Arantsson 26 Mar '10

26 Mar '10

Hi all, I've been using the sendfile package off Hackage, but it seems to be leaking file descriptors when using the Linux-native build. What's happening in my specific case is the following: 1) client requests a range of a file 2) server starts sending the range 3) client disconnects before receiving the whole file This happens over and over with the client requesting different ranges of the file (so the client does make progress). If I use the portable build of the sendfile package, everything works fine for hours and hours of this happening. If I use the Linux-native build of the sendfile package, the server will eventually run out of file descriptors. According to "lsof" the files that are being kept open are the data files being sent in 2) above. This is on GHC 6.10.x (Ubuntu). Is anyone else seeing this? Anyone got any idea what's going on? Cheers,

10 62

Is hGetLine lazy like hGetContents? And what about this todo item?
by Jason Dagit 25 Mar '10

25 Mar '10

Hello, I was trying to figure out if hGetLine is safe to use inside of withFile. Specifically, I want to return the line I read and use it later, without inspecting it before withFile calls hClose. If you want to understand the concern I have, look here: http://www.haskell.org/haskellwiki/Maintaining_laziness#Input_and_Output There is a bit of explanation showing that hGetContents can be problematic with withFile. I can tell from reading the source of hGetContents that it uses unsafeInterleaveIO so this make sense to me why that wiki page talks about hGetContents: http://www.haskell.org/ghc/docs/latest/html/libraries/base/src/GHC-IO-Handl… When I read the source of hGetLine, it is less clear to me if I need to be concerned. I believe it is not lazy in the sense of lazy IO above. Could someone else please comment? http://www.haskell.org/ghc/docs/latest/html/libraries/base/src/GHC-IO-Handl… Then I notice this 'todo' item in the description: -- ToDo: the unbuffered case is wrong: it doesn't lock the handle for -- the duration. The code itself looks to me like it only handles the buffered case. Perhaps this todo is obsolete and needs to be removed? If it's not obsolete, do we need to create a ticket for this? Thanks, Jason

3 5

HLIst - how to use TIPTransform with functions that run in the IO monad?
by Andrew U. Frank 25 Mar '10

25 Mar '10

i use the TIPTransform module that Oleg has provided - very nice and easy to use! Thank you Oleg!! now i run into a problem, i seem not to be able to resolve: how to use TIPTransform with functions that use the IO monad? considering the example tip5 in Olegs communiation, where he uses a function with three arguments from the TIP (HList) and producing a result in the HList. i would like to use a similar functionality, getting the arguments from the HList and producing a result which is again stored in the HList - but the function has a signature like f :: Int -> String -> IO String. the resulting string should go int the HList. i assume i need a function like ttip: ttipM :: op -> db -> IO db? can somebody help me with this? i appreciate any help! andrew

1 0

ANNOUNCE: cuboid 0.12: 3D Yampa/GLUT Puzzle Game
by Pedro Martins 25 Mar '10

25 Mar '10

Hi all, Recently, I have been playing around a bit with Yampa and GLUT, integrating them and trying to develop a simple 3D game within the framework. As it involved quite a bit of source code reading from Yampa and Frag (thanks to both!), I have decided to upload the game to Hackage, to help other people interested in using AFRP for game development. Overall, the experience of using Yampa was very pleasing, as it allowed me to describe the logic of the game in a more declarative fashion than usual. However, because there aren't many tutorials/examples on using Yampa for games, it involved a fair amount of experimentation and so my code might not be very idiomatic for Yampa. Please tell me if so and I'll fix it so this can be a good example. For playing instructions, see the Hackage description [1]. It currently only has 3 hardcoded easy levels, but the level definition is a simple Haskell ADT definition (Game.hs), and I plan on placing it in a separate data file in the future. This is my first upload to Hackage, so any comments or suggestions are welcome. Thanks, Pedro [1] http://hackage.haskell.org/package/cuboid

1 0

existentially quantified data types - restrictions
by Ozgur Akgun 25 Mar '10

25 Mar '10

Dear Cafe, I need to use a language feature which is explicitly documented to be a restriction, and -even worse- I think I reasonably need to use it. f2 (Baz1 a b) (Baz1 p q) = a==q > It's ok to say a==b or p==q, but a==q is wrong because it equates the two > distinct types arising from the two Baz1 constructors. > [from *7.4.4.4. Restrictions* at > http://www.haskell.org/ghc/docs/latest/html/users_guide/data-type-extension… > ] > To simplify, let's say Baz is the only constructor of a data type, data Baz = forall a. Eq a => Baz a -- | this cannot be done: instance Eq (Baz a) where (Baz x) == (Baz y) = x == y I am quite tempted to use show functions for this equality comparison, but after trying to have a nicely type framework I really don't want to do that. What I simply want is, haskell to be able to compare them if they belong to the same type, and return False otherwise. (not that haskelly way of doing things, I know.) Any suggestions better than the following are very welcome: (==) = (==) `on` show Regards, -- Ozgur Akgun

3 4