On Wed, Jun 29, 2016 at 11:20 AM, Dennis Raddle <dennis.raddle@gmail.com> wrote:

Hello, I wrote a program that reads filenames using System.Directory "listDirectory", does some tests on them, and writes certain file names (as read into a FilePath variable by listDirectory) to a text file, which I later parse with Parsec. I am getting an error on writing to the text file:

commitBuffer: invalid argument (invalid character)

What can I do about this? If I am reading some kind of unusual characters from the FilePath, I want to choose some way of writing them so that the resulting file can still be parsed by Text.Parsec.ByteString. I hope that I can still rely on the "space" parser to find CR and LF, and don't want any other surprises.

D

I'd recommend being explicit about the character encoding you're using, and using the bytestring API for the I/O itself. As an example:

import qualified Data.Text.Lazy as TL

import qualified Data.Text.Lazy.Encoding as TL

import qualified Data.ByteString.Lazy as L

writeFileUtf8 :: FilePath -> String -> IO ()

writeFileUtf8 fp str = L.writeFile fp (TL.encodeUtf8 (TL.pack str))

The default handling of character encodings is reliant on environment variables, which IME makes the textual file writing functions notoriously fragile.

Michael