[GHC] #8118: <stdout>: commitAndReleaseBuffer: invalid argument (invalid character)
#8118: <stdout>: commitAndReleaseBuffer: invalid argument (invalid character) ----------------------------------+---------------------------------- Reporter: guest | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 Keywords: | Operating System: OpenBSD Architecture: x86_64 (amd64) | Type of failure: Runtime crash Difficulty: Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | ----------------------------------+---------------------------------- I get the following error... {{{ <stdout>: commitAndReleaseBuffer: invalid argument (invalid character) }}} When running following program: {{{#!haskell {-# LANGUAGE OverloadedStrings #-} import qualified Data.Text as T import qualified Data.Text.IO as T main = T.putStrLn "čušpajž日本語" }}} Was tested in #haskell and I have all the UTF-8 Settings set and so forth. It compiles just fine, but crashes during runtime. gcc -v: {{{ Reading specs from /usr/lib/gcc-lib/amd64-unknown-openbsd5.4/4.2.1/specs Target: amd64-unknown-openbsd5.4 Configured with: OpenBSD/amd64 system compiler Thread model: posix gcc version 4.2.1 20070719 }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8118 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8118: <stdout>: commitAndReleaseBuffer: invalid argument (invalid character) ----------------------------------+---------------------------------- Reporter: guest | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 Resolution: | Keywords: Operating System: OpenBSD | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by YitzGale): This seems to be a problem with iconv in 7.6.3 on OpenBSD. When this was run at the GHCi prompt: {{{ putStrLn . Data.Text.unpack . Data.Text.Encoding.decodeUtf8 $ Data.ByteString.pack [0xc4,0x8d,0x75,0xc5,0xa1,0x70,0x61,0x6a,0xc5,0xbe,0xe6,0x97,0xa5,0xe6,0x9c,0xac,0xe8,0xaa,0x9e] }}} The result was: {{{ *** Exception: <stdout>: hPutChar: invalid argument (invalid character) }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8118#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8118: <stdout>: commitAndReleaseBuffer: invalid argument (invalid character) -------------------------------------+------------------------------------- Reporter: guest | Owner: Type: bug | Status: infoneeded Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 Resolution: | Keywords: Operating System: OpenBSD | Architecture: Unknown/Multiple Type of failure: Runtime | Difficulty: Unknown crash | Blocked By: Test Case: | Related Tickets: Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Changes (by thomie): * status: new => infoneeded * architecture: x86_64 (amd64) => Unknown/Multiple Comment: Can someone try again with GHC 7.8.3, on OpenBSD? Note: [wiki:Building/Porting] mentions something about "--with-iconv- includes and --with-iconv-libraries". -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8118#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8118: <stdout>: commitAndReleaseBuffer: invalid argument (invalid character) -------------------------------------+------------------------------------- Reporter: guest | Owner: Type: bug | Status: infoneeded Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 Resolution: | Keywords: Operating System: OpenBSD | Architecture: Unknown/Multiple Type of failure: Runtime | Difficulty: Unknown crash | Blocked By: Test Case: | Related Tickets: #5666 Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Changes (by thomie): * related: => #5666 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8118#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8118: <stdout>: commitAndReleaseBuffer: invalid argument (invalid character) -------------------------------------+------------------------------------- Reporter: guest | Owner: Type: bug | Status: infoneeded Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 Resolution: | Keywords: Operating System: OpenBSD | Architecture: Type of failure: Runtime crash | Unknown/Multiple Blocked By: | Test Case: Related Tickets: #5666 | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by nsch): I actually ran into the same error on Windows 8.1 with GHC 7.8.3 (and gcc 4.6.3) from the Haskell Platform. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8118#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8118: <stdout>: commitAndReleaseBuffer: invalid argument (invalid character) -------------------------------------+------------------------------------- Reporter: guest | Owner: Type: bug | Status: infoneeded Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime crash | Unknown/Multiple Blocked By: | Test Case: Related Tickets: #5666 | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Changes (by nsch): * cc: mail@… (added) * version: 7.6.3 => 7.8.3 * os: OpenBSD => Unknown/Multiple -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8118#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8118: <stdout>: commitAndReleaseBuffer: invalid argument (invalid character) -------------------------------------+------------------------------------- Reporter: guest | Owner: Type: bug | Status: infoneeded Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime crash | Unknown/Multiple Blocked By: | Test Case: Related Tickets: #5666 | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Changes (by RyanGlScott): * cc: RyanGlScott (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8118#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8118: <stdout>: commitAndReleaseBuffer: invalid argument (invalid character) -------------------------------------+------------------------------------- Reporter: guest | Owner: Type: bug | Status: infoneeded Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: #5666 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by HairyDude): * cc: HairyDude (added) Comment: Same on Windows 10, GHC 7.10.3. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8118#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8118: <stdout>: commitAndReleaseBuffer: invalid argument (invalid character) -------------------------------------+------------------------------------- Reporter: guest | Owner: Type: bug | Status: infoneeded Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: #5666 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by HairyDude): Adding the following seems to fix the error: {{{#!hs import GHC.IO.Encoding main = do ... setLocaleEncoding utf8 .... }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8118#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8118: <stdout>: commitAndReleaseBuffer: invalid argument (invalid character) -------------------------------------+------------------------------------- Reporter: guest | Owner: Type: bug | Status: infoneeded Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: #5666 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by RyanGlScott): Replying to [comment:8 HairyDude]:
Unfortunately this fix isn't very portable.
Also, I ''think'' this "fix" depends on if you're running from MSYS/Cygwin (where it's a sufficient fix) or `cmd.exe`/PowerShell (where it fixes the warning, put doesn't output the character correctly). Can you confirm what shell you're using? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8118#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8118: <stdout>: commitAndReleaseBuffer: invalid argument (invalid character) -------------------------------------+------------------------------------- Reporter: guest | Owner: Type: bug | Status: infoneeded Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: #5666 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by HairyDude): Replying to [comment:9 RyanGlScott]:
Also, I ''think'' this "fix" depends on if you're running from
MSYS/Cygwin (where it's a sufficient fix) or `cmd.exe`/PowerShell (where it fixes the warning, put doesn't output the character correctly). Can you confirm what shell you're using? I'm using console2. But most of my program's output is to files, not the console. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8118#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8118: <stdout>: commitAndReleaseBuffer: invalid argument (invalid character) -------------------------------------+------------------------------------- Reporter: guest | Owner: Type: bug | Status: infoneeded Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: #5666 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by RyanGlScott): Replying to [comment:10 HairyDude]:
I'm using console2. But most of my program's output is to files, not the console.
I suppose I should be more specific. IIRC, you can run both `mintty` (which is what MSYS/Cygwin use) and `cmd.exe` (which PowerShell runs a variant of) from `console2`. Which particular one are you using from `console2`? And it shouldn't matter whether you're outputting to `stdout` or to files, you should still experience roughly the same errors. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8118#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8118: <stdout>: commitAndReleaseBuffer: invalid argument (invalid character) -------------------------------------+------------------------------------- Reporter: guest | Owner: Type: bug | Status: infoneeded Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: #5666 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by HairyDude): Oh, it's cmd.exe. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8118#comment:12 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8118: <stdout>: commitAndReleaseBuffer: invalid argument (invalid character) -------------------------------------+------------------------------------- Reporter: guest | Owner: Type: bug | Status: infoneeded Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: #5666 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by HairyDude): Also I can confirm it does output correctly with `setLocaleEncoding utf8`. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8118#comment:13 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8118: <stdout>: commitAndReleaseBuffer: invalid argument (invalid character) -------------------------------------+------------------------------------- Reporter: guest | Owner: Type: bug | Status: infoneeded Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: #5666 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by RyanGlScott): If it's `cmd.exe`, then I think `setLocaleEncoding utf8` isn't quite enough to both (1) make the error go away, and (2) actually render all Unicode characters correctly. (It might fix (1), but (2) is another question.) For example, what is the output of: {{{#!hs main = do setLocaleEncoding utf8 putStrLn "→" }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8118#comment:14 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8118: <stdout>: commitAndReleaseBuffer: invalid argument (invalid character) -------------------------------------+------------------------------------- Reporter: guest | Owner: Type: bug | Status: infoneeded Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: #5666 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by HairyDude): Ah... {{{ C:\Users\Peter\Documents\hacking>stack ghc -- test.hs -o test Run from outside a project, using implicit global project config Using resolver: lts-4.2 from implicit global project's config file: C:\Users\Peter\Documents\hacking\stack\global-project\stack.yaml [1 of 1] Compiling Main ( test.hs, test.o ) Linking test.exe ... C:\Users\Peter\Documents\hacking>test.exe  }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8118#comment:15 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8118: <stdout>: commitAndReleaseBuffer: invalid argument (invalid character) -------------------------------------+------------------------------------- Reporter: guest | Owner: Type: bug | Status: infoneeded Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: #5666 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by RyanGlScott): OK, thank you for confirming. As I [https://ghc.haskell.org/trac/ghc/ticket/7593#comment:6 noted] in #7593, just changing the encoding to `utf8` is a sufficient fix for `mintty`, but it's not enough for `cmd.exe`. For the latter, the only solution I've found was to use native Windows console API functions such as `WriteConsoleW` and `ReadConsoleW`. This only works for `stdin`/`stdout`, though, and not general file IO. I imagine it would be a pretty significant time investment redesigning Haskell IO to use Windows API calls everywhere, though. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8118#comment:16 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8118: <stdout>: commitAndReleaseBuffer: invalid argument (invalid character) -------------------------------------+------------------------------------- Reporter: guest | Owner: Type: bug | Status: infoneeded Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: #5666 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by HairyDude): This gets correct output: {{{ import GHC.IO.Encoding import System.Win32.Console main = do ... setLocaleEncoding utf8 setCOnsoleOutputCP 65001 ... }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8118#comment:17 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8118: <stdout>: commitAndReleaseBuffer: invalid argument (invalid character) -------------------------------------+------------------------------------- Reporter: guest | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: #5666 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by HairyDude): * status: infoneeded => new -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8118#comment:18 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8118: <stdout>: commitAndReleaseBuffer: invalid argument (invalid character) -------------------------------------+------------------------------------- Reporter: guest | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: #5666 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by RyanGlScott): Replying to [comment:17 HairyDude]:
This gets correct output:
{{{ import GHC.IO.Encoding import System.Win32.Console
main = do ... setLocaleEncoding utf8 setConsoleOutputCP 65001 ... }}}
''If you're only dealing with files''. If you attempt to read/write from the console, things still break: {{{#!hs import GHC.IO.Encoding import System.Win32.Console main :: IO () main = do setLocaleEncoding utf8 setConsoleOutputCP 65001 putStrLn "→" }}} {{{ *Main> main *** Exception: <stdout>: hPutChar: invalid argument (invalid character) }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8118#comment:19 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8118: <stdout>: commitAndReleaseBuffer: invalid argument (invalid character) -------------------------------------+------------------------------------- Reporter: guest | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: #5666 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by HairyDude): Odd. It works if you compile the program, but not from GHCi. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8118#comment:20 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8118: <stdout>: commitAndReleaseBuffer: invalid argument (invalid character) -------------------------------------+------------------------------------- Reporter: guest | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: #5666 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by thomasjm): I just hit this problem in Ubuntu 16.04.2 LTS, with stack resolver lts-8.5 (GHC 8.0.2). Doing "setLocaleEncoding utf8" worked. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8118#comment:21 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8118: <stdout>: commitAndReleaseBuffer: invalid argument (invalid character) -------------------------------------+------------------------------------- Reporter: guest | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: #5666 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by nh2): On Windows Server 2016 (as provisioned by AWS Windows instances, into which I RDP'd), with the Locale set to Japanese (as described in https://www.java.com/en/download/help/locale.xml -- this sets the code page, as shown by `chcp`, to `932` in PowerShell and cmd.exe), when trying to reproduce the bug with the code from the issue description, **I already see GHC itself dying with the same error** during `stack setup`: {{{ PS C:\Users\日\Desktop\ghc-bug-8118> stack setup The GHC located at C:\Users\譌･ \AppData\Local\Programs\stack\x86_64-windows\ghc-8.2.2\bin\ghc.EXE failed to compile a sanity check. Please see: http://docs.haskellstack.org/en/stable/install_and_upgrade/ for more information. Exception was: Running C:\Users\譌･ \AppData\Local\Programs\stack\x86_64-windows\ghc-8.2.2\bin\ghc.EXE C:\Users\譌･\AppData\Local\Temp\4\stack-sanity-check32792\Main.hs -no- user-package-db in directory C:\Users\譌･\AppData\Local\Temp\4\stac k-sanity-check32792\ exited with ExitFailure 1 [1 of 1] Compiling Main ( C:\Users\・ｽ・ｽ\AppData\Local\Temp\4 \stack-sanity-check32792\Main.hs, C:\Users\・ｽ・ｽ\AppData\Local\Temp\4 \stack-sanity-check32792\Main.o ) Linking C:\Users\・ｽ・ｽ\AppData\Local\Temp\4\stack-sanity- check32792\Main.exe ... C:\Users\・ｽ・ｽ\AppData\Local\Temp\4\ghc34616_0\ghc_6.rsp: commitBuffer: invalid argument (invalid character) }}} (Note that this is not how I see it in PowerShell, where all backslashes are replaced by the Yen sign, but this is as I see it when I copy-paste the output into Sublime Text 3 or a Firefox window). -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8118#comment:22 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC