I'm at wits end with respect to GHC's garbage collector and would very much appreciate a code review of my MySQL driver for HDBC, which is here: <http://www.maubi.net/~waterson/REPO/HDBC-mysql/Database/HDBC/MySQL/Connectio...

In particular, the problem that I'm having is that my "statements" (really, just iterators over a SQL query result set) are getting garbage collected prematurely. I've included the source for the relevant functions below. The first is newStatement, which creates a new HDBC Statement record (an elided version, below):

newStatement :: ForeignPtr MYSQL -> String -> IO Types.Statement newStatement mysql__ query = withForeignPtr mysql__ $ \mysql_ -> do stmt_ <- mysql_stmt_init mysql_ ... stmt__ <- newForeignPtr mysql_stmt_close stmt_ ... return $ Types.Statement { , Types.fetchRow = fetchRow mysql__ stmt__ results }

The mysql_stmt_init function is the MySQL C API that creates a new statement record. Its result must be freed by mysql_stmt_close, and therefore is wrapped with a ForeignPtr appropriately. The fetchRow function is part of the HDBC Statement API, and its signature is:

fetchRow :: IO (Maybe [SqlValue])

I.e., it retrieves the next row from the result set and returns the column values as a list. The "results" parameter is not relevant to this discussion: it maintains the buffers used for storage of the column values. The next function of interest is my implementation of fetchRow, copied in its entirety, below:

fetchRow :: ForeignPtr MYSQL -> ForeignPtr MYSQL_STMT -> [MYSQL_BIND] -> IO (Maybe [Types.SqlValue]) fetchRow mysql__ stmt__ results = withForeignPtr mysql__ $ \_ -> withForeignPtr stmt__ $ \stmt_ -> do rv <- mysql_stmt_fetch stmt_ case rv of 0 -> row #{const MYSQL_DATA_TRUNCATED} -> row #{const MYSQL_NO_DATA} -> return Nothing _ -> statementError stmt_ where row = mapM cellValue results >>= \cells -> return $ Just cells

It unwraps the ForeignPtrs and invokes the mysql_stmt_fetch C API with the native MYSQL_STMT structure, causing the column values for the next row to get bound into storage areas that we can convert to Haskell values. Typically, this code gets invoked in the generic HDBC layer by the fetchAllRows routine, which looks like this:

fetchAllRows :: Statement -> IO [[SqlValue]] fetchAllRows sth = unsafeInterleaveIO $ do row <- fetchRow sth case row of Nothing -> return [] Just x -> do remainder <- fetchAllRows sth return (x : remainder)

The sth argument is simply an instance of the Statement record that I've returned from low-level MySQL driver. This all works, except... The behavior that I'm seeing is that the ForeignPtr to the native MYSQL_STMT struct has become unreachable after some number of iterations of the above loop. This causes mysql_stmt_close to be invoked, and subsequent calls to mysql_stmt_fetch fail with an error. Curiously, if I replace

stmt__ <- newForeignPtr mysql_stmt_close stmt_

with

stmt__ <- newForeignPtr_ stmt_

in my newStatement routine (i.e., so that mysql_stmt_close is never called), then fetchAllRows will merrily fetch all the rows from my statement handle. The down side, of course, being that I'll leak statement handles. So, I'm completely puzzled as to how I can continue to extract values from the apparently unreachable pointer! Using System.Mem.Weak, I've verified that: * The Statement record that I return from newStatement becomes unreachable. * The applied value of fetchRow becomes unreachable; e.g.: let r = fetchRow mysql__ stmt__ results addFinalizer r $ putStrLn "r is unreachable!" I'm not sure how this can be, given that fetchAllRows appears to be explicitly maintaining a reference to sth, which in turn ought to reference my fetchRow application. I've created a test program that directly invokes my driver code the same way that the generic HDBC layer does, and have verified that the lack of "-fno-cse" with unsafeInterleaveIO in the generic HDBC module is not relevant. I was similarly able to verify that this problem occurs with "-O0", so it's not due to any particular GHC optimization. Any thoughts on how to debug this would be greatly appreciated! Thanks in advance, chris