- Haskell-Cafe - Haskell.org

RE: Database interface - would like advice on oracle library bind ing
by Bayley, Alistair 25 Sep '03

25 Sep '03

> From: oleg(a)pobox.com [mailto:oleg@pobox.com] > > I'd like to remark first that many real databases let us avoid opening > many cursors at once. It seems to be more efficient to do as much as > possible within the engine ... I agree (and Tom Pledger makes the same comment). It's rare that I want to have more than one cursor open at once. When I've seen other developers do it, they're often doing something that would be better accomplished as a join. And that last time I remember doing it, it was a nested loops situation (which, on reflection, probably could have been done as a join... oh well). OTOH, I have seen interleaved fetches used to simulate a join where the specific DBMS product struggled to perform the join efficiently itself. In this case the developer is making up for a deficiency in the DBMS, and I don't want to exclude the option from the interface (if you agree with what you read on http://www.dbdebunk.com , then most DBMS products are deficient in a number of ways). > Still, opening several cursors may be unavoidable. The left fold > approach may still help -- we _can_ mechanically invert a left fold > combinator (to be precise, a half-baked version of it) into a lazy > list. Please see a separate message "how to invert the left fold" Found it. I'll have a look... > I believe the extract functions can be constructed automatically -- > similar to the way Quickcheck constructs test cases. I believe that > instead of I though this might be possible, but I had no idea how to do it. I'll have a look at the Quickcheck source to see how it's done, unless you can suggest a better example. > buffers. It's hard for me to write the corresponding code because I > don't have Oracle handy (besides, I like Informix better). Is it > possible to come up with a "stub" that uses flat files? We are only > interested in fetching rows. It doesn't matter if these rows come from > a file or from a database. That would make prototyping the interface > quite easier. I wasn't interested so much in prototyping the interface, as trying to write an implementation that supported the interface(s) discussed previously. I intended to provide the left-fold interface, and was wondering if that was all that was needed (for selects). Still, this would be a good exercise for me, at least so I can figure out how to generate extraction functions. > From: Tom Pledger [mailto:Tom.Pledger@peace.com] > Here's one approach: find the OCI equivalent of JDBC's > ResultSetMetaData, and use it to drive the allocation and freeing of > buffers. I've considered this, and I think it's the next route I'll take (the OCI supports it). At the least it'll give me the ability to perform arbitrary queries (at present I have to know the types of the result set columns and construct the extraction function manually). I've also considered stuffing more information into the Cursor type (which I've introduced), and using this in a modal fashion to decide what to do at each point. > Here's another: ... > Make getInt (and friends) behave differently depending on the mode > of the cursor they're passed: either allocate a buffer and return > _|_, decode and return the current column of the current row, or > free a buffer and return _|_. Not wanting to sound too dumb, but how do you return _|_ ? Thanks, Alistair. ***************************************************************** The information in this email and in any attachments is confidential and intended solely for the attention and use of the named addressee(s). This information may be subject to legal professional or other privilege or may otherwise be protected by work product immunity or other legal rules. It must not be disclosed to any person without our authority. If you are not the intended recipient, or a person responsible for delivering it to the intended recipient, you are not authorised to and must not disclose, copy, distribute, or retain this message or any part of it. *****************************************************************

3 2

Database interface - would like advice on oracle library binding
by Bayley, Alistair 24 Sep '03

24 Sep '03

(2nd attempt; mailman thinks I'm not a list member, but it still keeps sending me mail.) Still making slow progress on an Oracle database binding... now I'm trying to fit the API I have into some sort of abstract interface (like the one(s) discussed previously: http://haskell.org/pipermail/haskell-cafe/2003-August/004957.html ). 1. Is the left-fold the best/only interface to expose? I think yes, but that doesn't allow fine-grained control over cursors i.e. being able to open many cursors at once and interleave fetches from them. Or does it? 2. I'm finding it hard to write a doQuery function that takes an extraction function that isn't a pig to write. Some advice would be useful here... (and a long-ish explanation follows): The Oracle Call Interface (OCI) requires that I allocate buffers for the result of a single row fetch, before the first row is fetched. So a query involves: - prepare statement etc - allocate buffers (one for each column - call OCI C function "DefineByPos") - fetch row - extract/marshal data from buffer into Haskell types (which are then processed by fold function) - fetch, marshal (repeat until no more rows) - free buffers i.e. the same buffers are re-used for each row. The problem for me is how to specify the left-fold function in terms of the low-level API. If I want to supply extraction functions (to extract Haskell values from result buffer), how do I manage the buffer allocation in the doQuery function? The buffer allocate/free code also needs to know the column positions and types, in the same manner as the extract functions. I want to be able to write code like this: results <- doQuery dbconn sqltext [] \row results -> do name <- stringv row 1 address <- stringv row 2 return (name,address):results .. where the stringv function extracts/marshals a Haskell String from the result buffer. The intermediate approach I currently have means I have to pass an IO action into the doQuery function that, when evaluated, allocates the buffer and returns two more actions: - an action that extracts the row as a tuple - another action that frees the buffer The doQuery function evaluates the initial action (to allocate the buffer), uses the extract action to build the result (at present a list), and when there are no more rows, uses the free action to free the buffer. This approach is quite awkward (especially w.r.t. writing extract functions), and it's hard for me to see how to build a better interface. Hard, because of the memory management requirements. -------------------- Here's a chunk of the code. A lot of it is OCI plumbing, but I hope you can see how awkward it is to create an extract function (see ex3 at the bottom). Given pointers to the buffer, extract a string of variable length (you have to terminate it yourself). > fetchStringVal :: OCIColInfo -> IO String > fetchStringVal (_, bufptr, nullindptr, retsizeptr) = do > retsize <- liftM cShort2Int (peek retsizeptr) > nullind <- liftM cShort2Int (peek nullindptr) -- unused > pokeByteOff (castPtr bufptr) retsize nullByte > val <- peekCString (castPtr bufptr) > return val Free a single column's buffer. > freeColBuffer :: OCIColInfo -> IO () > freeColBuffer (_, bufptr, nullindptr, retsizeptr) = do > free bufptr > free retsizeptr > free nullindptr Create a buffer for a string column, and return the extract and free IO actions. > getExtractFnString :: Int -> ErrorHandle -> StmtHandle -> IO (IO String, IO ()) > getExtractFnString posn err stmt = do > c <- defineCol err stmt posn 2000 oci_SQLT_CHR > return ((fetchStringVal c), (freeColBuffer c)) doQuery uses the extractFns action to create the result buffer, and the two actions (extract and free) which are passed to doQuery2. > doQuery2 :: ErrorHandle -> StmtHandle -> IO a -> IO () -> [a] -> IO [a] > doQuery2 err stmt extractData freeMem results = do > rc <- fetch err stmt > if rc == oci_NO_DATA > then do > freeMem > return results > else do > v <- extractData > doQuery2 err stmt extractData freeMem (v:results) > doQuery :: Session -> String -> (ErrorHandle -> StmtHandle -> IO (IO a, IO ())) -> IO [a] > doQuery (Sess env err con) qry extractFns = do > stmt <- getStmt env err > prepare err stmt qry > execute err con stmt > (extractData, freeMem) <- extractFns err stmt > doQuery2 err stmt extractData freeMem [] The interface provided by doQuery means I have to write extract functions like this. Here's one for a select that returns three String columns. It's quite awkward... > ex3 :: ErrorHandle -> StmtHandle -> IO (IO (String, String, String), IO ()) > ex3 err stmt = do > (fetchcol1, freecol1) <- getExtractFnString 1 err stmt > (fetchcol2, freecol2) <- getExtractFnString 2 err stmt > (fetchcol3, freecol3) <- getExtractFnString 3 err stmt > return > ( do { s1 <- fetchcol1; s2 <- fetchcol2; s3 <- fetchcol3; return (s1, s2, s3) } > , do { freecol1; freecol2; freecol3 } > ) ***************************************************************** The information in this email and in any attachments is confidential and intended solely for the attention and use of the named addressee(s). This information may be subject to legal professional or other privilege or may otherwise be protected by work product immunity or other legal rules. It must not be disclosed to any person without our authority. If you are not the intended recipient, or a person responsible for delivering it to the intended recipient, you are not authorised to and must not disclose, copy, distribute, or retain this message or any part of it. *****************************************************************

3 2

Re: Generating "setMember" functions for record structures
by Peter Simons 23 Sep '03

23 Sep '03

Peter Gammie writes: > Haskell Report, Sec 3.15.3 Great, that's exactly what I need. Thanks a lot to all who replied! Peter

1 0

Generating "setMember" functions for record structures
by Peter Simons 23 Sep '03

23 Sep '03

Hi, in one of my programs, I have to modify individual fields of a pretty large record structure quite frequently. Assume the structure is: > data State = State { foo :: [Int] > , bar :: [String] > } Then I'd like to have functions setFoo and setBar, which copy a State but replace the field in question with a given value. This is implemented as: > setFoo :: State -> [Int] -> State > setFoo st x = State { foo = x > , bar = bar st > } > setBar :: State -> [String] -> State > setBar st x = State { foo = foo st > , bar = x > } Now, defining these helper functions for large records is not what I call an interesting job. Thus I wonder, is there any way to automate this task? I thought about using Template Haskell, but I haven't used it at all yet and am uncertain about how difficult accomplishing this would be. I also thought about using "Data.Generics", which I know better, but the generic approach feels like over-kill for this task. In order to use it, I'd have to re-design my data structure as well. So this isn't too attractive ... Any ideas? Comments? Source Code? :-) Peter

3 2

garbage collection and Ptr/ForeignPtr/StablePtr
by Bayley, Alistair 23 Sep '03

23 Sep '03

Should I expect that Ptr memory allocated with malloc is garbage collected when no longer used by the Haskell runtime? The FFI spec doesn't say so explicitly (that I could see); AFAICT this is how the pointer types behave: Ptr : freed when no longer used/referenced ForeignPtr : like Ptr, but will also run a finaliser when freed StablePtr : not freed or collected (I haven't tested this; I've only read the FFI spec.) ***************************************************************** The information in this email and in any attachments is confidential and intended solely for the attention and use of the named addressee(s). This information may be subject to legal professional or other privilege or may otherwise be protected by work product immunity or other legal rules. It must not be disclosed to any person without our authority. If you are not the intended recipient, or a person responsible for delivering it to the intended recipient, you are not authorised to and must not disclose, copy, distribute, or retain this message or any part of it. *****************************************************************

2 1

Re: Thanks: (was Question about categories)
by Bill Halchin 23 Sep '03

23 Sep '03

2 1

Thanks: (was Question about categories)
by Graham Klyne 22 Sep '03

22 Sep '03

Thanks to everyone who responded to my question about category theory. The responses turned up some interesting and useful material on the web, which I'm still digesting. Notably [1] [2]. There's something in the introduction of [2] that I think will help me get a handle on this: [[ ... objects are not collections of "elements," and morphisms do not need to be functions between sets (thus morphisms cannot be applied to "elements" but only composed with other morphisms). Any immediate access to the internal structure of objects is prevented: all properties of objects must be specified by properties of morphisms ... ]] My earlier supposition was that an "object" was roughly in correspondence with a member of a set, but I'm coming to perceive that "object" corresponds more with the notion of a set or collection of some (unspecified) things. #g -- [1] http://www.cs.toronto.edu/~sme/presentations/cat101.pdf [2] http://www.di.ens.fr/users/longo/download.html which has a link to content of the (apparently) out-of-print book: [[ CATEGORIES TYPES AND STRUCTURES An Introduction to Category Theory for the working computer scientist Andrea Asperti Giuseppe Longo FOUNDATIONS OF COMPUTING SERIES, M.I.T. PRESS, 1991 ]] _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe(a)haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ------------ Graham Klyne GK(a)NineByNine.org

3 2

RE: garbage collection and Ptr/ForeignPtr/StablePtr
by Simon Marlow 22 Sep '03

22 Sep '03

> Should I expect that Ptr memory allocated with malloc is > garbage collected > when no longer used by the Haskell runtime? The FFI spec > doesn't say so > explicitly (that I could see); AFAICT this is how the pointer > types behave: > Ptr : freed when no longer used/referenced > ForeignPtr : like Ptr, but will also run a finaliser when freed > StablePtr : not freed or collected No, all memory that you allocate with the FFI must be freed explicitly. You can arrange for memory to be deallocated when no longer referenced by using a ForeignPtr, and using the finalizer to free the memory. Bear in mind that if your application has large, or strict, memory requirements, then relying on the ForeignPtr method may not be the best solution, because finalizers can be unpredictable. Cheers, Simon

1 0

Re: An IO Question from a Newbie
by Jon Cast 22 Sep '03

22 Sep '03

> On Fri, 12 Sep 2003 16:27:59 -0600 > Matt O'Connor <angagon(a)earthlink.net> wrote: > > > Hello all. I'm new to functional programming and Haskell, but have > > been programming in C and Java for a while. I've been going through > > the tutorials and whatnot on haskell.org. I've read from the Gentle > > Introduction to Haskell about IO and some of the other stuff and I > > have a question about it. > > > > main = do putStr "Type Something: " > > str <- getLine > > putStrLn ("You typed: " ++ str) > > > > When compile and run this code the "Type Something: " isn't displayed > > until the putStrLn. So there is no prompt. The output looks like > > this. > > > > s > > Type Something: You typed: s > > > > But if I change the putStr "Type Something: " to a putStrLn or put a > > \n at the end of the string it displays the text immediately (ie, when > > I want it to). Is there a good reason for this? Am I doing something > > wrong? Or do I need to call some kind of standard output flush? > > Yes, the problem is that the first line is not being flushed. Importing > hFlush and stdout from IO (or System.IO) and adding hFlush stdout after > you putStr will fix it. Alternatively, you could change the buffering > on stdout to NoBuffering. I don't know if there is a standard > putStrWithFlush function or just a better answer than this. A > putStrWithFlush would be trivial to write or more generally to a > HOF that flushed after performing the action passed to it. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Just thought I'd point out (in case it isn't clear) that this function is flushAfter h a = liftM2 const a (hFlush h). Not much of an improvement... Jon Cast

1 0

(Off-topic) Question about categories
by Graham Klyne 19 Sep '03

19 Sep '03

(I'm asking here because I believe there is some overlap between category theory and functional programming cognoscenti...) It has been suggested to me that categories may be a useful framework (more useful than set theory) for conceptualizing things for which there is no well-defined equality relationship. (The discussion is about resources in WWW.) I've done a little digging about category theory, and find some relatively approachable material at: http://www.wikipedia.org/wiki/Category_theory http://www.wikipedia.org/wiki/Class_(set_theory) and nearby. But I'm hitting a mental block with this (and other places I've look don't add anything I can grok). The definition of a category depends on the definition of a morphism, and in particular the existence of an identity morphism for every object in a category. The definition of an morphism is in terms of equality of compositions of morphisms: for f : A -> B we have Id[B]. f = f = f . Id[A] My problem is this: how does it make sense to define an equality of morphisms without some well-defined concept of equality on the underlying objects to which they apply? That is, given object X and an object Y, it is possible to examine them and determine whether or not they are the same object. And if the underlying objects have such a concept of equality, what prevents them from being sets or members of sets? But categories are presented as being more general than sets. Does anyone see the cause of my non-comprehension here? #g ------------ Graham Klyne GK(a)NineByNine.org

6 5