On Thu, Apr 30, 2026 at 12:07:08PM +0200, Henning Thielemann wrote:
'Zip.getEntrySource selector' generates a conduit for reading a file from the ZIP archive. It opens the archive anew for every accessed file, but it calls Conduit.Binary.sourceIOHandle, which promises to close a file promptly after reading. [2]
You have to be a bit careful about what "immediately after reading" means. Actually, I think it means "immediately after the file handle indicates it has reached end of file". `getEntrySource`[1] calls `sourceEntry`[2] which runs source .| CB.isolate (fromIntegral edCompressedSize) .| decompress Now, source is source = CB.sourceIOHandle $ do ... But `sourceIOHandle` will only close its handle after it determines that it has reached EOF. If you look at the argument to `sourceIOHandle` in [2] you'll see that it just seeks to the start position of the target file in the archive. It's CB.isolate which is used to limit reading to the extent of the target file only. It brings down the pipeline once it is done but because EOF is never reached in the handle (i.e. the final bytes of the entire .zip file are never read) the cleanup action doesn't trigger at this point. The cleanup action will nonetheless trigger at the end of the runConduitRes block, but that's too late. You've already opened too many files. I have a simpler reproducer for this below. Maybe this is salvagable by putting a ResourceT block within the definition of `sourceEntry`, maybe not. I don't have the finer details of ResourceT in working memory. However, there are numerous problems with Conduit and Pipes bracketing that make it very difficult to use them safely. My effect system Bluefin solves this problem, as I elaborated in my article "Bluefin streams finalize promptly". https://h2.jaguarpaw.co.uk/posts/bluefin-streams-finalize-promptly/ However, there is sadly not a large ecosystem around Bluefin, and no Zip library. I'm happy to help you implement what you need if you're interested! Tom [1] https://hackage-content.haskell.org/package/zip-2.2.1/docs/src/Codec.Archive... [2] https://hackage-content.haskell.org/package/zip-2.2.1/docs/src/Codec.Archive... {- cabal: build-depends: conduit, base -} {-# LANGUAGE GHC2021 #-} import Conduit (liftIO, mapM_C, runConduitRes, (.|)) import Control.Monad (forever, when) import Data.Conduit.Combinators (sourceIOHandle) import Data.Conduit.List qualified as CB import Data.IORef (modifyIORef', newIORef, readIORef, writeIORef) import System.IO (IOMode (ReadMode), openFile) -- cabal-script-test27.hs: /dev/zero: openFile: resource exhausted (Too -- many open files) main :: IO () main = do ref <- newIORef @Int 0 runConduitRes $ forever $ do sourceIOHandle (openFile "/dev/zero" ReadMode) .| CB.isolate 1 .| mapM_C ( \_ -> liftIO $ do modifyIORef' ref (+ 1) n <- readIORef ref when (n `mod` 20 == 0) $ do Prelude.print n )