error writing FilePath to text file
Hello, I wrote a program that reads filenames using System.Directory "listDirectory", does some tests on them, and writes certain file names (as read into a FilePath variable by listDirectory) to a text file, which I later parse with Parsec. I am getting an error on writing to the text file: commitBuffer: invalid argument (invalid character) What can I do about this? If I am reading some kind of unusual characters from the FilePath, I want to choose some way of writing them so that the resulting file can still be parsed by Text.Parsec.ByteString. I hope that I can still rely on the "space" parser to find CR and LF, and don't want any other surprises. D
On Wed, Jun 29, 2016 at 11:20 AM, Dennis Raddle
Hello, I wrote a program that reads filenames using System.Directory "listDirectory", does some tests on them, and writes certain file names (as read into a FilePath variable by listDirectory) to a text file, which I later parse with Parsec. I am getting an error on writing to the text file:
commitBuffer: invalid argument (invalid character)
What can I do about this? If I am reading some kind of unusual characters from the FilePath, I want to choose some way of writing them so that the resulting file can still be parsed by Text.Parsec.ByteString. I hope that I can still rely on the "space" parser to find CR and LF, and don't want any other surprises.
D
I'd recommend being explicit about the character encoding you're using, and using the bytestring API for the I/O itself. As an example: import qualified Data.Text.Lazy as TL import qualified Data.Text.Lazy.Encoding as TL import qualified Data.ByteString.Lazy as L writeFileUtf8 :: FilePath -> String -> IO () writeFileUtf8 fp str = L.writeFile fp (TL.encodeUtf8 (TL.pack str)) The default handling of character encodings is reliant on environment variables, which IME makes the textual file writing functions notoriously fragile. Michael
Thanks. I don't know if I was clear that I'm writing the characters of the FilePath to a file. I could use something like: writeFilePaths :: FilePath -> [FilePath] -> IO () writeFilePaths fileToWrite fps = ... The ByteStrings are for reading the files generated by the above and parsing with Text.Parsec.ByteString, for efficiency (these are many megabytes of text) D
On Wed, Jun 29, 2016 at 11:31 AM, Dennis Raddle
Thanks. I don't know if I was clear that I'm writing the characters of the FilePath to a file. I could use something like:
writeFilePaths :: FilePath -> [FilePath] -> IO () writeFilePaths fileToWrite fps = ...
The ByteStrings are for reading the files generated by the above and parsing with Text.Parsec.ByteString, for efficiency (these are many megabytes of text)
D
A FilePath is just a type synonym for a String, so the same caveats will apply to both. I'm guessing that you're dealing with some file paths that have non-ASCII characters in them, and you have some locale environment variables set that do not support UTF8. You can try rerunning your program as-is with: export LANG=en_US.UTF-8 in the shell (assuming you're on a POSIX system, Windows will behave differently). Michael
participants (2)
-
Dennis Raddle -
Michael Snoyman