- Glasgow-haskell-users

Re: H98 Text IO
by Ben Franksen 28 Feb '08

28 Feb '08

David Leuschner wrote: > If I write simple program just printing a non-ASCII string to the terminal > or to a file I'd expect that I can read it on the screen or using my > favorite text editor without having change anything -- neither in my > terminal nor in my program. When I run the program on my platform don't > mind if somebody else might get differently encoded output from the same > program as long as I get what I expect. I think this is sensible advice. Furthermore I want to be able to chose an encoding (e.g. by setting my environment), even for file/socket IO, instead of being locked into utf-8. Thus, for Linux, I'd say always use the environment encoding. More advanced stuff like sending in a certain encoding e.g. over network or to a foreign file system should not use H98 IO but newer libraries that allow to set the encoding programmatically, or encode yourself and read/write Word8. Cheers Ben

1 0

Exports of STM constructors.
by Sterling Clover 27 Feb '08

27 Feb '08

I was recently bitten by the urge to hack around with STM, and discovered to my dismay that there's no way to get at the primitive constructors for TVar and STM, both of which are defined in GHC.Conc. Given that, when we so desire, we can get at the primitives to break IO into pieces, manually unbox integers and such, and that the primitives for working with MVars are similarly exposed through appropriate imports, it would be really nice (or, well, at least fun) to be able to do the same for STM. --S

1 0

Re: H98 Text IO
by Matthew Bentham 27 Feb '08

27 Feb '08

----- Original Message ---- > From: Johan Tibell <johan.tibell(a)gmail.com> > To: Duncan Coutts <duncan.coutts(a)worc.ox.ac.uk> > Cc: Haskell Libraries <libraries(a)haskell.org>; GHC-users list <Glasgow-haskell-users(a)haskell.org>; Simon Marlow <simonmarhaskell(a)gmail.com> > Sent: Wednesday, February 27, 2008 9:25:39 AM > Subject: Re: H98 Text IO > > On Wed, Feb 27, 2008 at 1:06 AM, Duncan Coutts > wrote: > > As a data point, Java and python use "always locale" as default if you > > don't specify an encoding when opening a text stream. > > > > I think personally I'm coming round to the "always locale" point of > > view. We already have no cross-platform consistency for text files > > because of the lf vs cr/lf issue and we have no cross-implementation > > consistency. > > I think following Java and Python in this matter is a good idea and > leads to fewer surprises for developers. If you want files created on > one machine to work on another you have to be explicit about encoding. I'm a newcomer to haskell so I don't want to goof by offering any naive opinions, but I would like to inform the discussion by linking to a couple of useful things. This blog post on encodings in python (mostly related to the behaviour of the standard input/output streams): http://drj11.wordpress.com/2007/05/14/python-how-is-sysstdoutencoding-chose… Particularly: "It’s not unreasonable to have a program that wants to encode its outputin a particular encoding. The example I gave earlier still seemsreasonable to be, a program that takes input in one encoding andrecodes to a different encoding on its output, with both the input andoutput encoding specified on the command line. Clearly such a programshould be able to use stdin and stdout so that it can form part of apipe. So how in Python do I change sys.stdout to use a particular encoding? It’s a right pain." And the reference to the POSIX documentation related to locale, mentioned earlier in this thread: http://www.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap07.html The problem that I see is generally that because stdin/out/err are _already_open_ by the time they reach the haskell program, it is pretty hard for the developer "to be explicit about encoding" if no library is provided to effectively change the encoding used by an already-open stream.

2 1

Re: H98 Text IO
by Chris Kuklewicz 27 Feb '08

27 Feb '08

Small correction: I think "./prog in vs ./prog < in " and "utf8" should be "ok". (and I thought this was switched to Glasgow-haskell-users(a)haskell.org) David Leuschner wrote: >> Let me try and summarise: > > Thanks for the great summary! And thanks to Emacs' table mode here're the > results displayed as a table: > > +--------------------------------+-----+--------+------+-------+-------+ > | | now | locale | utf8 | mix-A | mix-B | > +--------------------------------+-----+--------+------+-------+-------+ > | putStrLn "..." | - | ok | - | ok | ok | > +--------------------------------+-----+--------+------+-------+-------+ > | ./prog vs ./prog | cat | ok | ok | ok | - | ok | > +--------------------------------+-----+--------+------+-------+-------+ > | ./prog in vs ./prog < in | - | ok | - | ok | - | > +--------------------------------+-----+--------+------+-------+-------+ > | ./prog vs ./prog | hexdump -C | ok | ok | ok | - | ok | > +--------------------------------+-----+--------+------+-------+-------+ > > > The mixtures are good ideas but can give inconsistent and suriprising > results (especially when debugging encoding issues). And if our CEO would > have known that ... putStrLn <his-name> ... doesn't work he'd have > probably ruled out Haskell right from the start. Even "utf8" gives > surprising results: I'd be very surprised if my Mac-written Haskell > program outputs junk on Windows or Linux even if the byte sequence is > exactly the same UTF-8 text. > > Personally I think consistency on a single platform is more important than > trying to achieve cross-platform consistency which involves a lot more > than just encoding. If you've reached that point with your program you're > probably anyway using "advanced functions" to exactly specify what will be > output. Following "the principle of least surprise" is also a good idea. > > Cheers, > > David > > > > > ------------------------------------------------------------------------ > > _______________________________________________ > Libraries mailing list > Libraries(a)haskell.org > http://www.haskell.org/mailman/listinfo/libraries

2 1

H98 Text IO
by Duncan Coutts 27 Feb '08

27 Feb '08

>From the H98 report: All I/O functions defined here are character oriented. [...] These functions cannot be used portably for binary I/O. In the following, recall that String is a synonym for [Char] (Section 6.1.2). So ordinary text Handles are for text, not binary. Char is of course a Unicode code point. The crucial question of course is what encoding of text to use. For the H98 IO functions we cannot set it as a parameter, we have to pick a sensible default. Currently different implementations disagree on that default. Hugs has for some time used the current locale on posix systems (and I'm guessing the current code page on windows). GHC has always used the Latin-1 encoding. These days, most operating systems use a locale/codepage encoding that covers full the Unicode range. So on hugs we get the benefit of that but on GHC we do not. This is endlessly surprising for beginners. They do putStrLn "αβγδεζηθικλ" and it comes out on their terminal as junk. It also causes problems for serious programs, see for example the recent hand-wringing on cabal-devel. So here is a concrete proposal: * Haskell98 file IO should always use UTF-8. * Haskell98 IO to terminals should use the current locale encoding. The main controversial point I think is whether to always use UTF-8 or always use the current locale or some split as I've suggested. C chose to always go with the current locale. Some people think that was a mistake because the interpretation changes from user to user. For terminals it is more clear cut that the locale is the right choice because that is what the terminal is capable of displaying. Using anything else will produce junk. We can detect if a handle is a terminal when we open it using hIsTerminalDevice. This should be done automatically (and ghc would ghc get it for free because it already does that check to determine default buffering modes). Sockets and pipes would be treated the same as files when opened in the default text mode. The only special case is terminals. The major problem is with code that assumes GHC's Handles are essentially Word8 and layer their own UTF8 or other decoding over the top. The utf8-string package has this problem for example. Such code should be using openBinaryFile because they are reading/writing binary data, not String text. Note that many programs that really need to work with binary file already use openBinaryFile, those that do not are already broken on Windows which does cr/lf conversion on text files which breaks many binary formats (though not utf8). So we have decide which is more painful, keeping a limited text IO system in GHC or breaking some existing programs which assume GHC's current behaviour. Opinions? Please can we keep this discussion to the interpretation of the H98 IO functions and not get into the separate discussion of how we could extend or redesign the whole IO system. This is a questions of what are the right defaults. Duncan

7 15

Re: H98 Text IO
by Reinier Lamers 26 Feb '08

26 Feb '08

Op 26-feb-2008, om 18:42 heeft Chris Kuklewicz het volgende geschreven: > The goal is that more complicated situations are reflected in > more complicated "ghc" or "main" invocations. The least complicated > usage defaults to being identical cross-platform and regardless of > terminal I/O. > > I think the best default would be UTF8 for all text handles. This can > be easily documented, it can be easily understood, and will produce > the fewest suprises. > (...) > ** Unless influenced by command-line switches, these default to UTF8. I think that making the behavior of programs change, depending on compiler options, will produce a lot of surprises. I think that being only able to set the default encoding from within the program is a better idea, because it keeps the specification of the behavior of the program inside the source. Reinier

2 1

how to use ghci-debugger with packages
by Frederik Eaton 26 Feb '08

26 Feb '08

Hello, I have a program which uses some code in a package, and I would like to be able to find out the source of an error which is occuring inside that package. Can I use the ghci-debugger to do this? If I try to set a breakpoint inside the package, it says: "cannot set breakpoint on Vector.Sparse.Wrappers.vmergeOp: module Vector.Sparse.Wrappers is not interpreted" But this page: http://www.haskell.org/ghc/dist/current/docs/users_guide/ghci-debugger.html has no use of the word "package", so I assume that reading that (long) document won't ansnwer my question... Thank you, Frederik -- http://ofb.net/~frederik/

3 5

Visual Haskell sources
by Andrei Formiga 26 Feb '08

26 Feb '08

Hello, The Visual Haskell 0.2 release notes [1] say that sources are available, but the download page only has binaries available. Where are the sources? Also, does it use the Visual Studio SDK, and is it compatible with VS 2008? Thanks. [1] http://haskell.org/visualhaskell/doc/index.html#release-notes -- []s, Andrei Formiga

2 1

Re: Problem with Windows ghc 6.8.2
by Matthew Bentham 24 Feb '08

24 Feb '08

----- Original Message ---- > From: Daniel Fischer <daniel.is.fischer(a)web.de> > To: Matthew Bentham <matthew_bentham(a)yahoo.com>; glasgow-haskell-users(a)haskell.org > Sent: Sunday, February 24, 2008 1:13:48 AM > Subject: Re: Problem with Windows ghc 6.8.2 > > Am Sonntag, 24. Februar 2008 01:54 schrieb Matthew Bentham: > > Hi all, > > > > I'm not too experienced with ghc or Haskell, so I wanted to check here > > rather than submitting a bug: > > It's probably the base-split. In 6.6.1, Text.PrettyPrint was in the base > package, in 6.8.2 it's in the pretty package. > Try compiling with -package pretty, or better still, with --make: > > /path/to/ghc --make test > > or > > /path/to/ghc -o test.exe test.hs -package pretty > > If that's not it, you might have a real problem. Thanks, they both work great. Matthew

1 0

Problem with Windows ghc 6.8.2
by Matthew Bentham 24 Feb '08

24 Feb '08

Hi all, I'm not too experienced with ghc or Haskell, so I wanted to check here rather than submitting a bug: $ cat test.hs import Text.PrettyPrint main = print "foo" $ /c/ghc/ghc-6.6.1/bin/ghc test.hs $ ./main.exe "foo" $ /c/ghc/ghc-6.8.2/bin/ghc test.hs test.o(.text+0x217):fake: undefined reference to `__stginit_prettyzm1zi0zi0zi0_TextziPrettyPrint_' $ I'm using Vista64. The session above was copied from a cygwin shell, but it does the same in an msys shell or cmd.exe. Both versions of ghc were installed using the official binary distribution. I'm having trouble compiling ghc from source at the moment (I think there's another issue I'm hitting with the 6.6.1 build, haven't tracked it down yet) so I can't debug what is going wrong myself. I'd appreciate people's advice about what to do. Should my test program work or am I doing something wrong? Maybe there are some optional libraries to install that I have missed, that used to be core? Output from ghc -v test.hs follows: $ rm test.hi $ $ $ /c/ghc/ghc-6.8.2/bin/ghc -v test.hs *** Tidy Core: Result size = 16 writeBinIface: 1 Names writeBinIface: 7 dict entries *** CorePrep: Result size = 18 *** Stg2Stg: *** CodeGen: *** CodeOutput: *** Assembler: c:\ghc\ghc-6.8.2\gcc -Bc:\ghc\ghc-6.8.2\gcc-lib/ -I. -c c:\Users\mjb67\AppData\Local\Temp\ghc4332_0\ghc4332_0.s -o test.o *** Windres: c:\ghc\ghc-6.8.2\bin/windres --preprocessor="c:\ghc\ghc-6.8.2\gcc" "-Bc:\ghc\ghc-6.8.2\gcc-lib/" "-E" "-xc" "-DRC_INVOKED" --use-temp-file --input=c:/Users/mjb67/AppData/Local/Temp/ghc4332_0/ghc4332_0.rc --output=c:/Users/mjb67/AppData/Local/Temp/ghc4332_0/ghc4332_0.o --output-format=coff *** Linker: c:\ghc\ghc-6.8.2\gcc -Bc:\ghc\ghc-6.8.2\gcc-lib/ -v -o main.exe -DDONT_WANT_WIN32_DLL_SUPPORT test.o c:/Users/mjb67/AppData/Local/Temp/ghc4332_0/ghc4332_0.o -Lc:/ghc/ghc-6.8.2/lib\haskell98-1.0.1.0 -Lc:/ghc/ghc-6.8.2/lib\array-0.1.0.0 -Lc:/ghc/ghc-6.8.2/lib\process-1.0.0.0 -Lc:/ghc/ghc-6.8.2/lib\random-1.0.0.0 -Lc:/ghc/ghc-6.8.2/lib\directory-1.0.0.0 -Lc:/ghc/ghc-6.8.2/lib\filepath-1.1.0.0 -Lc:/ghc/ghc-6.8.2/lib\old-time-1.0.0.0 -Lc:/ghc/ghc-6.8.2/lib\old-locale-1.0.0.0 -Lc:/ghc/ghc-6.8.2/lib\base-3.0.1.0 -Lc:/ghc/ghc-6.8.2 -Lc:/ghc/ghc-6.8.2/gcc-lib -lHShaskell98-1.0.1.0 -lHSarray-0.1.0.0 -lHSprocess-1.0.0.0 -lHSrandom-1.0.0.0 -lHSdirectory-1.0.0.0 -lHSfilepath-1.1.0.0 -lHSold-time-1.0.0.0 -lHSold-locale-1.0.0.0 -lHSbase-3.0.1.0 -lwsock32 -lmsvcrt -lkernel32 -luser32 -lshell32 -lHSrts -lm -lgmp -lwsock32 -u _base_GHCziBase_Izh_static_info -u _base_GHCziBase_Czh_static_info -u _base_GHCziFloat_Fzh_static_info -u _base_GHCziFloat_Dzh_static_info -u _base_GHCziPtr_Ptr_static_info -u _base_GHCziWord_Wzh_static_info -u _base_GHCziInt_I8zh_static_info -u _base_GHCziInt_I16zh_static_info -u _base_GHCziInt_I32zh_static_info -u _base_GHCziInt_I64zh_static_info -u _base_GHCziWord_W8zh_static_info -u _base_GHCziWord_W16zh_static_info -u _base_GHCziWord_W32zh_static_info -u _base_GHCziWord_W64zh_static_info -u _base_GHCziStable_StablePtr_static_info -u _base_GHCziBase_Izh_con_info -u _base_GHCziBase_Czh_con_info -u _base_GHCziFloat_Fzh_con_info -u _base_GHCziFloat_Dzh_con_info -u _base_GHCziPtr_Ptr_con_info -u _base_GHCziPtr_FunPtr_con_info -u _base_GHCziStable_StablePtr_con_info -u _base_GHCziBase_False_closure -u _base_GHCziBase_True_closure -u _base_GHCziPack_unpackCString_closure -u _base_GHCziIOBase_stackOverflow_closure -u _base_GHCziIOBase_heapOverflow_closure -u _base_GHCziIOBase_NonTermination_closure -u _base_GHCziIOBase_BlockedOnDeadMVar_closure -u _base_GHCziIOBase_BlockedIndefinitely_closure -u _base_GHCziIOBase_Deadlock_closure -u _base_GHCziIOBase_NestedAtomically_closure -u _base_GHCziWeak_runFinalizzerBatch_closure -u _base_GHCziConc_ensureIOManagerIsRunning_closure Reading specs from c:/ghc/ghc-6.8.2/gcc-lib/specs Configured with: ../gcc/configure --with-gcc --with-gnu-ld --with-gnu-as --host=mingw32 --target=mingw32 --prefix=/mingw --enable-threads --disable-nls --enable-languages=c,c++,f77,ada,objc,java --disable-win32-registry --disable-shared --enable-sjlj-exceptions --enable-libgcj --disable-java-awt --without-x --enable-java-gc=boehm --disable-libgcj-debug --enable-interpreter --enable-hash-synchronization --enable-libstdcxx-debug Thread model: win32 gcc version 3.4.2 (mingw-special) ld -Bdynamic -o main.exe -u _base_GHCziBase_Izh_static_info -u _base_GHCziBase_Czh_static_info -u _base_GHCziFloat_Fzh_static_info -u _base_GHCziFloat_Dzh_static_info -u _base_GHCziPtr_Ptr_static_info -u _base_GHCziWord_Wzh_static_info -u _base_GHCziInt_I8zh_static_info -u _base_GHCziInt_I16zh_static_info -u _base_GHCziInt_I32zh_static_info -u _base_GHCziInt_I64zh_static_info -u _base_GHCziWord_W8zh_static_info -u _base_GHCziWord_W16zh_static_info -u _base_GHCziWord_W32zh_static_info -u _base_GHCziWord_W64zh_static_info -u _base_GHCziStable_StablePtr_static_info -u _base_GHCziBase_Izh_con_info -u _base_GHCziBase_Czh_con_info -u _base_GHCziFloat_Fzh_con_info -u _base_GHCziFloat_Dzh_con_info -u _base_GHCziPtr_Ptr_con_info -u _base_GHCziPtr_FunPtr_con_info -u _base_GHCziStable_StablePtr_con_info -u _base_GHCziBase_False_closure -u _base_GHCziBase_True_closure -u _base_GHCziPack_unpackCString_closure -u _base_GHCziIOBase_stackOverflow_closure -u _base_GHCziIOBase_heapOverflow_closure -u _base_GHCziIOBase_NonTermination_closure -u _base_GHCziIOBase_BlockedOnDeadMVar_closure -u _base_GHCziIOBase_BlockedIndefinitely_closure -u _base_GHCziIOBase_Deadlock_closure -u _base_GHCziIOBase_NestedAtomically_closure -u _base_GHCziWeak_runFinalizzerBatch_closure -u _base_GHCziConc_ensureIOManagerIsRunning_closure c:/ghc/ghc-6.8.2/gcc-lib/crt2.o c:/ghc/ghc-6.8.2/gcc-lib/crtbegin.o -Lc:/ghc/ghc-6.8.2/lib\haskell98-1.0.1.0 -Lc:/ghc/ghc-6.8.2/lib\array-0.1.0.0 -Lc:/ghc/ghc-6.8.2/lib\process-1.0.0.0 -Lc:/ghc/ghc-6.8.2/lib\random-1.0.0.0 -Lc:/ghc/ghc-6.8.2/lib\directory-1.0.0.0 -Lc:/ghc/ghc-6.8.2/lib\filepath-1.1.0.0 -Lc:/ghc/ghc-6.8.2/lib\old-time-1.0.0.0 -Lc:/ghc/ghc-6.8.2/lib\old-locale-1.0.0.0 -Lc:/ghc/ghc-6.8.2/lib\base-3.0.1.0 -Lc:/ghc/ghc-6.8.2 -Lc:/ghc/ghc-6.8.2/gcc-lib -Lc:/ghc/ghc-6.8.2/gcc-lib test.o c:/Users/mjb67/AppData/Local/Temp/ghc4332_0/ghc4332_0.o -lHShaskell98-1.0.1.0 -lHSarray-0.1.0.0 -lHSprocess-1.0.0.0 -lHSrandom-1.0.0.0 -lHSdirectory-1.0.0.0 -lHSfilepath-1.1.0.0 -lHSold-time-1.0.0.0 -lHSold-locale-1.0.0.0 -lHSbase-3.0.1.0 -lwsock32 -lmsvcrt -lkernel32 -luser32 -lshell32 -lHSrts -lm -lgmp -lwsock32 -lmingw32 -lgcc -lmoldname -lmingwex -lmsvcrt -luser32 -lkernel32 -ladvapi32 -lshell32 -lmingw32 -lgcc -lmoldname -lmingwex -lmsvcrt c:/ghc/ghc-6.8.2/gcc-lib/crtend.o test.o(.text+0x217):fake: undefined reference to `__stginit_prettyzm1zi0zi0zi0_TextziPrettyPrint_' *** Deleting temp files: Deleting: c:/Users/mjb67/AppData/Local/Temp/ghc4332_0/ghc4332_0.o c:/Users/mjb67/AppData/Local/Temp/ghc4332_0/ghc4332_0.rc c:/Users/mjb67/AppData/Local/Temp/ghc4332_0/ghc4332_0.s *** Deleting temp dirs: Deleting: c:/Users/mjb67/AppData/Local/Temp/ghc4332_0

2 1