In a previous thread on this mailing list I proposed a way to use gettext. Since Haskell uses Unicode to represent characters, and gettext has support for converting strings into UTF-8, I made that the default mode of operation in my I18N module. I found a UTF8 library, which I modified to pass illegal UTF-8 sequences through unchanged. Since Haskell uses Unicode, and UTF-8 is one of the most common encodings for unicode, it would be good to have a UTF-8 library like this. You can find the files at http://www.dtek.chalmers.se/~d95mback/gettext/ where you also can find a printf-like implementation. I would very much like to hear some comments, also on the use of unsafePerformIO with locale-dependent functions (gettext is of course locale-dependent). Regards, Martin -- Martin Norbäck d95mback@dtek.chalmers.se Kapplandsgatan 40 +46 (0)708 26 33 60 S-414 78 GÖTEBORG http://www.dtek.chalmers.se/~d95mback/ SWEDEN OpenPGP ID: 3FA8580B
I would very much like to hear some comments, also on the use of unsafePerformIO with locale-dependent functions (gettext is of course locale-dependent).
As long as nothing calls setlocale, there's not much problem: things will work consistently within a program run though they may vary between runs. If you wanted to be able to change the locale at runtime or you want to follow the lead of System.getArgs, you can either: 1) Put locale dependent functions in the IO monad. or 2) Make the locale a parameter (either explicit or implicit) of the functions and have the locale dependent functions change locale as needed. Only call setlocale if the locale has changed since the last time you called a locale-dependent function. Nice clean design but you'd better be sure to wrap all locale-dependent functions (including other C functions which invoke locale-dependent functions) - it'd be murder to debug it if you mess up. -- Alastair Reid alastair@reid-consulting-uk.ltd.uk Reid Consulting (UK) Limited http://www.reid-consulting-uk.ltd.uk/alastair/
One major nit I have with this is the type signature of decodeUTF8 and encodeUTF8 a String should always represent a string of characters, not a byte stream, the signatures should be decodeUTF8 :: String -> [Word8] encodeUTF8 :: [Word8] -> String this problem occurs all over the place in the haskell libraries, now that the FFI spec gives us Word8 we should make use of it. Just a pet peeve of mine. good work otherwise, i like it. you might want to check out my Format.hs (similar to your printf module but somewhat more powerful) and my modified utf8 code to use byte streams properly... http://repetae.net/john/computer/haskell/Format.hs http://repetae.net/john/computer/haskell/UTF8.hs John On Mon, Aug 05, 2002 at 12:12:02PM +0200, Martin Norbäck wrote:
In a previous thread on this mailing list I proposed a way to use gettext. Since Haskell uses Unicode to represent characters, and gettext has support for converting strings into UTF-8, I made that the default mode of operation in my I18N module.
I found a UTF8 library, which I modified to pass illegal UTF-8 sequences through unchanged.
Since Haskell uses Unicode, and UTF-8 is one of the most common encodings for unicode, it would be good to have a UTF-8 library like this. You can find the files at http://www.dtek.chalmers.se/~d95mback/gettext/ where you also can find a printf-like implementation.
I would very much like to hear some comments, also on the use of unsafePerformIO with locale-dependent functions (gettext is of course locale-dependent).
-- --------------------------------------------------------------------------- John Meacham - California Institute of Technology, Alum. - john@foo.net ---------------------------------------------------------------------------
On Tue, Aug 06, 2002 at 05:38:13AM -0700, John Meacham wrote:
One major nit I have with this is the type signature of decodeUTF8 and encodeUTF8 a String should always represent a string of characters, not a byte stream, the signatures should be
decodeUTF8 :: String -> [Word8] encodeUTF8 :: [Word8] -> String I guess that is a good point, but due to backwards compatibility this is propably not acceptable: The C interface of the FFI has the string functions: peekCString :: CString -> IO String newCString :: String -> IO CString
which should really be peekCString :: CString -> IO [Word8] newCString :: [Word8] -> IO CString Unless that changes, there is really no point to give the encode and decode functions that type. Axel.
I guess that is a good point, but due to backwards compatibility this is propably not acceptable:
These libraries are fairly new so it's easy to change them. (AFAIK, only GHC has released a compiler with these libraries and they only appeared in the 5.04 release.) If a change is appropriate, please tell us. I, personally, haven't given much thought to UTF8 encoding but people on the ffi mailing list (ffi@haskell.org) have, so send us a brief explanation of what change is needed and why.
The C interface of the FFI has the string functions:
peekCString :: CString -> IO String newCString :: String -> IO CString
which should really be
peekCString :: CString -> IO [Word8] newCString :: [Word8] -> IO CString
Unless that changes, there is really no point to give the encode and decode functions that type.
One of the great things about the FFI is that it is really easy to use. In particular, it's easy to write your own functions which do the right thing so if lack of functions with the right type is holding you back, it's easy to add the right functions yourself. -- Alastair Reid alastair@reid-consulting-uk.ltd.uk Reid Consulting (UK) Limited http://www.reid-consulting-uk.ltd.uk/alastair/
participants (4)
-
Alastair Reid -
Axel Simon -
John Meacham -
Martin Norbäck