cannot catch exception
Hi all, The program reads lots of small Text files. readCDFile handles the encoding. Below is the simplest version of readCDFile. If I call readCDFile "/home/kees/freeDB/inputError/" "blah" (the file blah does not exist) I get: Left "MyError: /home/kees/freeDB/inputError/blah: openBinaryFile: does not exist (No such file or directory)". The exception is caught by exceptionHandler If I call readCDFile "/home/kees/freeDB/inputError/" "67129209" I get freeDB: Cannot decode byte '\xa0': Data.Text.Internal.Encoding.decodeUtf8: Invalid UTF-8 stream. The exception is not caught by exceptionHandler (No "MyError: " in front). The file 67129209 is indeed bad encoded. I'am using SomeException. Still, this 'bad encoding exception' is not caught. Why? Kees import qualified Data.Text as T import System.FilePath.Posix import qualified Data.Text.Encoding as TE import qualified Data.ByteString.Lazy as B import Prelude hiding (catch) import Control.Exception main :: IO () main = do res <- readCDFile "/home/kees/freeDB/inputError/" "67129209" print res readCDFile :: FilePath -> FilePath -> IO (Either String T.Text) readCDFile baseDir fn = do catch ( do buffer <- B.readFile (combine baseDir fn) let bufferStrict = B.toStrict buffer return $ Right $ TE.decodeUtf8 bufferStrict ) exceptionHandler exceptionHandler :: SomeException -> IO (Either String T.Text) exceptionHandler e = do let err = show e return $ Left $ "MyError: " ++ err --- Dit e-mailbericht is gecontroleerd op virussen met Avast antivirussoftware. https://www.avast.com/antivirus
This fixes it by forcing the evaluation of the decode where it can be caught: return $ Right $! TE.decodeUtf8 bufferStrict or Right <$> evaluate (TE.decodeUtf8 bufferStrict) On Mon, Mar 4, 2019 at 3:50 AM Kees Bleijenberg <K.Bleijenberg@lijbrandt.nl> wrote:
Hi all,
The program reads lots of small Text files. readCDFile handles the encoding. Below is the simplest version of readCDFile.
If I call readCDFile "/home/kees/freeDB/inputError/" "blah" (the file blah does not exist) I get:
Left "MyError: /home/kees/freeDB/inputError/blah: openBinaryFile: does not exist (No such file or directory)". The exception is caught by exceptionHandler
If I call readCDFile "/home/kees/freeDB/inputError/" "67129209" I get freeDB: Cannot decode byte '\xa0': Data.Text.Internal.Encoding.decodeUtf8: Invalid UTF-8 stream. The exception is not caught by exceptionHandler (No “MyError: ” in front). The file 67129209 is indeed bad encoded.
I’am using SomeException. Still, this ‘bad encoding exception’ is not caught. Why?
Kees
import qualified Data.Text as T
import System.FilePath.Posix
import qualified Data.Text.Encoding as TE
import qualified Data.ByteString.Lazy as B
import Prelude hiding (catch)
import Control.Exception
main :: IO ()
main = do
res <- readCDFile "/home/kees/freeDB/inputError/" "67129209"
print res
readCDFile :: FilePath -> FilePath -> IO (Either String T.Text)
readCDFile baseDir fn = do
catch ( do
buffer <- B.readFile (combine baseDir fn)
let bufferStrict = B.toStrict buffer
return $ Right $ TE.decodeUtf8 bufferStrict
) exceptionHandler
exceptionHandler :: SomeException -> IO (Either String T.Text)
exceptionHandler e = do let err = show e
return $ Left $ "MyError: " ++ err
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient> Virusvrij. www.avast.com <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient> <#m_4240515626836095125_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.
Another approach would be to use the Data.Text.IO.hGetContents function on a file handle that explicitly sets its character encoding to UTF-8. This is what we do in rio: https://www.stackage.org/haddock/lts-13.10/rio-0.1.8.0/src/RIO.Prelude.IO.ht... On Mon, Mar 4, 2019 at 3:36 PM David Fox <dsf@seereason.com> wrote:
This fixes it by forcing the evaluation of the decode where it can be caught:
return $ Right $! TE.decodeUtf8 bufferStrict
or
Right <$> evaluate (TE.decodeUtf8 bufferStrict)
On Mon, Mar 4, 2019 at 3:50 AM Kees Bleijenberg < K.Bleijenberg@lijbrandt.nl> wrote:
Hi all,
The program reads lots of small Text files. readCDFile handles the encoding. Below is the simplest version of readCDFile.
If I call readCDFile "/home/kees/freeDB/inputError/" "blah" (the file blah does not exist) I get:
Left "MyError: /home/kees/freeDB/inputError/blah: openBinaryFile: does not exist (No such file or directory)". The exception is caught by exceptionHandler
If I call readCDFile "/home/kees/freeDB/inputError/" "67129209" I get freeDB: Cannot decode byte '\xa0': Data.Text.Internal.Encoding.decodeUtf8: Invalid UTF-8 stream. The exception is not caught by exceptionHandler (No “MyError: ” in front). The file 67129209 is indeed bad encoded.
I’am using SomeException. Still, this ‘bad encoding exception’ is not caught. Why?
Kees
import qualified Data.Text as T
import System.FilePath.Posix
import qualified Data.Text.Encoding as TE
import qualified Data.ByteString.Lazy as B
import Prelude hiding (catch)
import Control.Exception
main :: IO ()
main = do
res <- readCDFile "/home/kees/freeDB/inputError/" "67129209"
print res
readCDFile :: FilePath -> FilePath -> IO (Either String T.Text)
readCDFile baseDir fn = do
catch ( do
buffer <- B.readFile (combine baseDir fn)
let bufferStrict = B.toStrict buffer
return $ Right $ TE.decodeUtf8 bufferStrict
) exceptionHandler
exceptionHandler :: SomeException -> IO (Either String T.Text)
exceptionHandler e = do let err = show e
return $ Left $ "MyError: " ++ err
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient> Virusvrij. www.avast.com <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient> <#m_152412307694788344_m_4240515626836095125_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.
_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.
David, I tried your first suggestion $!. Nothing changed. When I tried ‘Right <$> evaluate (TE.decodeUtf8 bufferStrict)’ success. handleException catches the exception. I don’t understand why. Maybe the documentation for the evaluate function below has to do with it: There is a subtle difference between <http://hackage.haskell.org/package/base-4.12.0.0/docs/Control-Exception-Base.html#v:evaluate> evaluate x and <http://hackage.haskell.org/package/base-4.12.0.0/docs/Control-Monad.html#v:return> return <http://hackage.haskell.org/package/base-4.12.0.0/docs/Prelude.html#v:-36--33-> $! x, analogous to the difference between <http://hackage.haskell.org/package/base-4.12.0.0/docs/Control-Exception-Base.html#v:throwIO> throwIO and <http://hackage.haskell.org/package/base-4.12.0.0/docs/Control-Exception-Base.html#v:throw> throw. If the lazy value x throws an exception, <http://hackage.haskell.org/package/base-4.12.0.0/docs/Control-Monad.html#v:return> return <http://hackage.haskell.org/package/base-4.12.0.0/docs/Prelude.html#v:-36--33-> $! x will fail to return an <http://hackage.haskell.org/package/base-4.12.0.0/docs/System-IO.html#t:IO> IO action and will throw an exception instead. <http://hackage.haskell.org/package/base-4.12.0.0/docs/Control-Exception-Base.html#v:evaluate> evaluate x, on the other hand, always produces an <http://hackage.haskell.org/package/base-4.12.0.0/docs/System-IO.html#t:IO> IO action; that action will throw an exception upon execution iff x throws an exception upon evaluation. I don’t fully understand this, but evaluate works. Thanks! Kees readCDFile :: FilePath -> FilePath -> IO (Either String T.Text) readCDFile baseDir fn = do catch ( do buffer <- B.readFile (combine baseDir fn) --reads strict the whole file let bufferStrict = B.toStrict buffer return $ Right $! TE.decodeUtf8 bufferStrict -- this doesn’t work Right <$> evaluate (TE.decodeUtf8 bufferStrict) –- this does liftM Right $ evaluate (TE.decodeUtf8 bufferStrict) – this works too ) exceptionHandler Van: David Fox [mailto:dsf@seereason.com] This fixes it by forcing the evaluation of the decode where it can be caught: return $ Right $! TE.decodeUtf8 bufferStrict or Right <$> evaluate (TE.decodeUtf8 bufferStrict) <snip> --- Dit e-mailbericht is gecontroleerd op virussen met Avast antivirussoftware. https://www.avast.com/antivirus
The problem is non-strict evaluation aka "laziness". "return" doesn't force evaluation of the pure expression you give to it (actual I/O involving its value generally would, but return just puts a wrapper around it), so its evaluation is forced later outside of your code to catch it, when the value is actually examined. In extreme cases you end up using deepseq's rnf to force full evaluation, but here it's enough to force evaluation to WHNF with ($!) before handing the result off to return. With a lazy ByteString you might need rnf, since ($!) would only force part of the structure of the list of chunks but you would need to force all of the chunks' contents to trigger the decode exception. On Mon, Mar 4, 2019 at 3:27 PM Kees Bleijenberg <K.Bleijenberg@lijbrandt.nl> wrote:
David,
I tried your first suggestion $!. Nothing changed.
When I tried ‘Right <$> evaluate (TE.decodeUtf8 bufferStrict)’ success. handleException catches the exception.
I don’t understand why. Maybe the documentation for the evaluate function below has to do with it:
There is a subtle difference between evaluate <http://hackage.haskell.org/package/base-4.12.0.0/docs/Control-Exception-Base.html#v:evaluate> x and return <http://hackage.haskell.org/package/base-4.12.0.0/docs/Control-Monad.html#v:return> $! <http://hackage.haskell.org/package/base-4.12.0.0/docs/Prelude.html#v:-36--33-> x, analogous to the difference between throwIO <http://hackage.haskell.org/package/base-4.12.0.0/docs/Control-Exception-Base.html#v:throwIO> and throw <http://hackage.haskell.org/package/base-4.12.0.0/docs/Control-Exception-Base.html#v:throw>. If the lazy value x throws an exception, return <http://hackage.haskell.org/package/base-4.12.0.0/docs/Control-Monad.html#v:return> $! <http://hackage.haskell.org/package/base-4.12.0.0/docs/Prelude.html#v:-36--33-> x will fail to return an IO <http://hackage.haskell.org/package/base-4.12.0.0/docs/System-IO.html#t:IO> action and will throw an exception instead. evaluate <http://hackage.haskell.org/package/base-4.12.0.0/docs/Control-Exception-Base.html#v:evaluate> x, on the other hand, always produces an IO <http://hackage.haskell.org/package/base-4.12.0.0/docs/System-IO.html#t:IO> action; that action will throw an exception upon *execution* iff x throws an exception upon *evaluation*.
I don’t fully understand this, but evaluate works. Thanks!
Kees
readCDFile :: FilePath -> FilePath -> IO (Either String T.Text)
readCDFile baseDir fn = do
catch ( do
buffer <- B.readFile (combine baseDir fn) --reads strict the whole file
let bufferStrict = B.toStrict buffer
return $ Right $! TE.decodeUtf8 bufferStrict -- this doesn’t work
Right <$> evaluate (TE.decodeUtf8 bufferStrict) –- this does
liftM Right $ evaluate (TE.decodeUtf8 bufferStrict) – this works too
) exceptionHandler
*Van:* David Fox [mailto:dsf@seereason.com]
This fixes it by forcing the evaluation of the decode where it can be caught:
return $ Right $! TE.decodeUtf8 bufferStrict
or
Right <$> evaluate (TE.decodeUtf8 bufferStrict)
<snip>
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient> Virusvrij. www.avast.com <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient> <#m_1428190607071173671_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.
-- brandon s allbery kf8nh allbery.b@gmail.com
On Mon, Mar 04, 2019 at 09:26:41PM +0100, Kees Bleijenberg wrote:
readCDFile :: FilePath -> FilePath -> IO (Either String T.Text) readCDFile baseDir fn = do catch ( do buffer <- B.readFile (combine baseDir fn) --reads strict the whole file let bufferStrict = B.toStrict buffer return $ Right $! TE.decodeUtf8 bufferStrict -- this doesn’t work Right <$> evaluate (TE.decodeUtf8 bufferStrict) –- this does liftM Right $ evaluate (TE.decodeUtf8 bufferStrict) -- this works too ) exceptionHandler
Take a close look at the three increasingly strict examples below: ghci> do { x <- return $ Right $ undefined ; putStrLn "delayed..."; case x of { Right _ -> return 1; _ -> return 0 } } delayed... 1 ghci> do { x <- return $ Right $! undefined; putStrLn "delayed..."; case x of { Right _ -> return 1; _ -> return 0 } } delayed... *** Exception: Prelude.undefined CallStack (from HasCallStack): error, called at libraries/base/GHC/Err.hs:78:14 in base:GHC.Err undefined, called at <interactive>:8:29 in interactive:Ghci3 ghci> do { x <- return $! Right $! undefined ; putStrLn "delayed..."; case x of { Right _ -> return 1; _ -> return 0 } } *** Exception: Prelude.undefined CallStack (from HasCallStack): error, called at libraries/base/GHC/Err.hs:78:14 in base:GHC.Err undefined, called at <interactive>:9:30 in interactive:Ghci3 While "Right $! x" is strict in x, "return $ Right $! x" is not! You need "return $! Right $! x" to make that happen. -- Viktor.
participants (5)
-
Brandon Allbery -
David Fox -
Kees Bleijenberg -
Michael Snoyman -
Viktor Dukhovni