Alexey Khudyakov [2010.04.28 1518 +0400]:

On Wed, Apr 28, 2010 at 1:11 PM, Norbert Zeh wrote:

...

Hi folks,

I'm using xmobar as my status bar and have problems to get UTF8 characters to display correctly.  I did the following tests:

1.  Start xmobar, reading input from stdin.  Type in some unicode characters.  Xmobar displays them fine.

2.  Add some unicode characters to my loghook in xmonad.  Things come out garbled.

3.  Instead of piping to xmobar, make xmonad write to a file.  The file contents are also garbled.  To be more precise, they are exactly what xmobar shows me in the second test.

So this seems like the problem is on xmonad's end, not xmobar's.  I'm running the latest darcs version of xmonad.  Any ideas what could be causing this?

What version of GHC do you use? Starting from version 6.12 GHC properly encodes strings. Earlier versions just truncated characters to 0-255 range.

If you use GHC 6.10 or earlier you should use encodeString from utf8-string package to encode string in utf8 before writing to pipe.

Thanks Alex and Ivan, before I say anything more on the issue, I need to do a bit more tinkering to find out what's really wrong. I'm running darcs xmonad and darcs xmobar, compiled with GHC 6.12.1. So the utf8 flag for xmobar should have no effect. (I tried with and without, and the result is the same.) What I've found out so far is (1) xmonad seems to double-encode the utf string. Namely, if I change my ppOutput to hPutStrLn handle . UTF8.decodeString, things come out *kind of* fine. (2) By *kind of* in the previous point, I mean that characters like accented letters, which were mangled before, now show up just fine. However, some other characters, like arrows, come out messed up, and for this the problem seems to be at xmobar's end because the same happens if I pipe these characters into xmobar from an xterm. In case, you're wondering: yes, the characters I'm trying to display are in the font I'm using. I want to play around a little more to dig into the second issue, but the first one is certainly strange. Cheers, Norbert

В сообщении от 29 апреля 2010 09:30:22 Norbert Zeh написал:

(1) xmonad seems to double-encode the utf string. Namely, if I change my ppOutput to hPutStrLn handle . UTF8.decodeString, things come out *kind of* fine.

Here is patch to fix this. It does work with GHC6.12 but I couldn't test it with 6.10 or older

* On Thursday, April 29 2010, Khudyakov Alexey wrote:

В сообщении от 29 апреля 2010 15:29:20 вы написали:

...

В сообщении от 29 апреля 2010 09:30:22 Norbert Zeh написал:

...

(1) xmonad seems to double-encode the utf string. Namely, if I change my ppOutput to hPutStrLn handle . UTF8.decodeString, things come out *kind of* fine.

Here is patch to fix this. It does work with GHC6.12 but I couldn't test it with 6.10 or older

Ooops. This patch breaks non-ASCII input in Promt

Here are some guidelines for addressing encoding issues: http://code.google.com/p/xmonad/issues/detail?id=348 -- Adam

Norbert Zeh [2010.04.30 0930 -0300]:

Khudyakov Alexey [2010.04.29 1720 +0400]:

...

В сообщении от 29 апреля 2010 15:29:20 вы написали:

...

В сообщении от 29 апреля 2010 09:30:22 Norbert Zeh написал:

...

(1) xmonad seems to double-encode the utf string. Namely, if I change my ppOutput to hPutStrLn handle . UTF8.decodeString, things come out *kind of* fine.

Here is patch to fix this. It does work with GHC6.12 but I couldn't test it with 6.10 or older

Well, the easier patch at the moment seems to be not to call encodeOutput in DynamicLog.hs. This doesn't seem to break anything else.

However, now here's a problem with xmobar, which rather seems to be an X11 problem. Certain characters (e.g. \x2192 (right arrow)) still get mangled. I was able to verify that they are still intact immediately before xmobar calls wcDrawImageString to render them on screen. So something must be going wrong inside wcDrawImageString.

Further to the point, the same problem happens with wcDrawImageString inside xmonad. I just added \x2192 to the run-or-raise prompt in XMonadContrib, recompiled, and I get the same garbled output as in xmobar. Yet, xterm and emacs display the character fine, using exactly the same font as I use for xmonad's prompts and for xmobar. N.