[GHC] #12213: Setting stdout to unbuffered causes us to switch to ascii output locale

20 Jun 2016

      #12213: Setting stdout to unbuffered causes us to switch to ascii output locale
-------------------------------------+-------------------------------------
           Reporter:  ezyang         |             Owner:
               Type:  bug            |            Status:  new
           Priority:  normal         |         Milestone:
          Component:  Test Suite     |           Version:  8.0.1
           Keywords:                 |  Operating System:  Unknown/Multiple
       Architecture:                 |   Type of failure:  None/Unknown
  Unknown/Multiple                   |
          Test Case:                 |        Blocked By:
           Blocking:                 |   Related Tickets:
Differential Rev(s):                 |         Wiki Page:
-------------------------------------+-------------------------------------
 I had a failing test case which was producing UTF-8 output, and Python was
 choking on it trying to print it. I traced the problem to these lines of
 code in `testsuite/driver/runtests.py`:

 {{{
 sys.stdout.flush()
 if PYTHON3:
     # in Python 3, we output text, which cannot be unbuffered
     sys.stdout = os.fdopen(sys.__stdout__.fileno(), "w")
 else:
     # set stdout to unbuffered (is this the best way to do it?)
     sys.stdout = os.fdopen(sys.__stdout__.fileno(), "w", 0)
 }}}

 On Python 2.7.11 on Linux, with `LANG=en_US.UTF-8` in the environment, if
 you place a `print(u"\u2018")` before these lines of code, Python will
 successfully print it; however, if you place it after, Python will crash:

 {{{
 Traceback (most recent call last):
   File "../../driver/runtests.py", line 279, in <module>
     print(u"\u2018")
 UnicodeEncodeError: 'ascii' codec can't encode character u'\u2018' in
 position 0: ordinal not in range(128)
 }}}

 So, I guess `fdopen` resets the codec to `ascii`, rather than whatever the
 locale detected. Drat!

 I did some brief googling but I couldn't tell what the right way to
 rewrite this line of code is.

--
Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12213
GHC http://www.haskell.org/ghc/
The Glasgow Haskell Compiler