I mean literally use the function `lift`. Proxy (it's the underlying type of all pipes, Pipe is a type synonym that expands to Proxy, filling some type variables for you) implements the MonadTrans class (from http://hackage.haskell.org/package/transformers-0.5.1.0/docs/Control-Monad-T...): class MonadTrans t where lift :: Monad m => m a -> t m a So, if `t` is `Pipe a b` and `m` is IO, then `lift` becomes: lift :: IO r -> Pipe a b IO r Thus, for example, `lift (c_hkl_engine_list_init engines geometry detector sample)` will return `Pipe a b IO <something>` (for any `a` and `b`, this is going to be a trivial pipe that doesn't yield or await anything, and just executes the effect) rather than `IO <something>`. You don't need to import the Monad.Trans.Class module, Pipe reexports it for you. Best regards, Marcin Mrotek

Yes, yield does return a Pipe. But "do" notation expands to a chain of ">>" and ">>=", and these functions only connect monadic values from the same monad. Pipes are monad transformers, so the conversion from IO to `Pipe a b IO` doesn't do anything interesting (converting back to IO would require running the pipe, which is less trivial), but it still has to be done for the types to agree. Also, now I've noticed that you may have some trouble with the `withSomething` functions. Do they take functions of type `a -> m b` and return the result wrapped again in `m` for any monad `m`, or do they only work with IO? If they are limited to IO, then you can't use them here, as you would need to convert a Pipe back to IO, and this conversion isn't a no-op and probably not what you want to do here. If this is the case, you'd need the raw functions that open and close these resources, and use Pipes.Safe to wrap them (https://hackage.haskell.org/package/pipes-safe): bracket :: MonadSafe m => Base m a -> (a -> Base m b) -> (a -> m c) -> m c This might look a bit convoluted, but for the sake of working with Pipes and IO, you can think of it as having type: bracket :: IO a -> (a -> IO b) -> (a -> Pipe x y (SafeT IO) c) -> Pipe x y (SafeT IO) c The first argument opens some resource, the senond closes it, and the third is the main working function. Instead of having a `Pipe x y IO c` you have `Pipe x y (SafeT IO) c` but this is almost the same, you'd only have to use `liftIO` instead of just `lift` to lift from IO to this monad (or use `lift` twice, once to get `SafeT IO`, and yet again to lift to `Pipe`), and after running the effect, use `runSafeT` to unwrap the value to get IO (this action ensures that the finalizer you provided to `bracket` is always executed, even if some exceptions have been raised meanwhile). But again, if the `withSomething` functions are polymorphic with respect to the monad used, then you leave them as they are and don't bother with Pipes.Safe, just wanted to warn you about a potential problem. Best regards, Marcin Mrotek

Well, if pipelining only the outermost layer is enough to improve performance of your code, perhaps you could just move `yield` outside all the `with...` functions, so everything stays in IO until it's ready to yield the result: solveTraj :: Factory -> Geometry -> Detector -> Sample -> Pipe Engine Geometry IO () solveTraj f g d s = do e <- await let name = engineName e solution <- withSample s $ \sample -> withDetector d $ \detector -> withGeometry f g $ \geometry -> withEngineList f $ \engines -> withCString name $ \cname -> do c_hkl_engine_list_init engines geometry detector sample engine <- c_hkl_engine_list_engine_get_by_name engines cname nullPtr n <- c_hkl_engine_pseudo_axis_names_get engine >>= darrayStringLen return $ solve' engine n e >>= getSolution0 yield solution This can be further simplified to use the version of `mapM` from Pipes.Prelude (https://hackage.haskell.org/package/pipes-4.1.8/docs/Pipes-Prelude.html) mapM :: Monad m => (a -> m b) -> Pipe a b m r This way your code would look more or less like it looked without pipes, something like: import qualified Pipes.Prelude as Pipes runEffect $ for (each engines) >-> Pipes.mapM (solve' engine n e >>= getSolution0) >-> P.print Best regards, Marcin Mrotek

except that if I use this runEffect $ each engines >-> solveTrajPipe factory geometry detector sample >-> P.drain instead of -- >-> P.print I get a segfault when I do a big number of coputation ??? It seems to me that the dif object is released as I am still using the underlying foreign ptr...

In fact I got the segfault also with the P.print... So I rewrote the solve function like this withDiffractometer :: Diffractometer -> (Ptr HklEngineList -> IO b) -> IO b withDiffractometer d fun = do let f_engines = difEngineList d withForeignPtr f_engines fun solveTrajPipe' :: Diffractometer -> Pipe Engine Geometry IO () solveTrajPipe' dif = flip evalStateT dif $ forever $ do -- Inside here we are using `StateT Int (Consumer a IO) r` e <- lift await dif <- get let name = engineName e solutions <- lift . lift $ withDiffractometer dif $ \engines -> withCString name $ \cname -> do engine <- c_hkl_engine_list_engine_get_by_name engines cname nullPtr n <- c_hkl_engine_pseudo_axis_names_get engine >>= darrayStringLen solve' engine n e >>= getSolution0 put dif lift $ yield solutions using the evalStateT. It works with million of evaluations :) but now if I remove the get and put lines, I get the segfault. This segfault is located in the solve' method solve' :: Ptr HklEngine -> CSize -> Engine -> IO (ForeignPtr HklGeometryList) solve' engine n (Engine _ ps _) = do let positions = [v | (Parameter _ v _) <- ps] withArray positions $ \values -> c_hkl_engine_pseudo_axis_values_set engine values n unit nullPtr >>= newForeignPtr c_hkl_geometry_list_free so to my opinion I do not manage correctly the life time of my dif object, but I do not understand how I can ensure this life time in until the solve' method proceed. Cheers Frederic

First of all, (StateT Diffractometer (Pipe Engine Geometry IO) ()) isn't the same as (Pipe Engine Geometry (StateT Diffractometer IO) ()), although I'm not sure what exactly the difference will be, as I've never used it the former way. This might be again a question for the Pipes mailing list.

ok, I will investigate this part :)

...

but now if I remove the get and put lines, I get the segfault. Okay, I have no idea. As I see it, this shouldn't happen, as you're getting and putting the same pointer all the time. What if you remove

Secondly: the StateT altogether and just use the `dif` from the function argument, are you still getting segfaults? Also what about writting the function without using pipes, and using Pipes.mapM to make it a pipe like I mentioned? (if the only Pipes operation you're doing are an `await` in the beginning and a `yield` at the end, Pipes.mapM covers it)

Yes I get the segfault if I remove all the State Part and use directly the dif. I think that I have something like this , I need to connect the life of one foreignPtr to another one. This is why I put them all into the Diffractometer data. But it seems that this is not the magic bullet in order to garanty the same life time for all of these foreignPtr. I am wonderig if the fact that I use only one of the ForeignPtr in the solveTrajPipe function does not give a clu to haskell that he just need to keep a reference to the used one and get ride of the other one stored in dif. withDiffractometer :: Diffractometer -> (Ptr HklEngineList -> IO b) -> IO b withDiffractometer d fun = do let f_engines = difEngineList d withForeignPtr f_engines fun then I extract a pointer from the engines one but as I am using the withForeingPtr I think that it is ok. I would say that I am not yet a specialist of FFI and haskell, but I love this language a lot when I will manage this problem :)) solveTrajPipe' :: Diffractometer -> Pipe Engine Geometry IO () solveTrajPipe' dif = flip evalStateT dif $ forever $ do -- Inside here we are using `StateT Int (Consumer a IO) r` e <- lift await -- dif <- get let name = engineName e solutions <- lift . lift $ withDiffractometer dif $ \engines -> withCString name $ \cname -> do engine <- c_hkl_engine_list_engine_get_by_name engines cname nullPtr n <- c_hkl_engine_pseudo_axis_names_get engine >>= darrayStringLen solve' engine n e >>= getSolution0 -- put dif lift $ yield solutions Cheers Frederic Ps: just for fun I got this adding just a lift print :) :~/hkl/contrib/haskell$ cabal run Preprocessing executable 'ghkl' for hkl-0.1.0.0... [3 of 5] Compiling Hkl.C ( dist/build/ghkl/ghkl-tmp/Hkl/C.hs, dist/build/ghkl/ghkl-tmp/Hkl/C.p_o ) src/Hkl/C.hsc:125:3: Couldn't match kind `* -> *' with `*' Expected type: Diffractometer -> Proxy () Engine () Geometry IO () Actual type: Diffractometer -> Proxy () Engine () Geometry IO () Kind incompatibility when matching types: Diffractometer :: * -> * Diffractometer :: * The function `lift'ghc: panic! (the 'impossible' happened) (GHC version 7.6.3 for x86_64-unknown-linux): kindFunResult ghc-prim:GHC.Prim.*{(w) tc 34d} Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug