[GHC] #8802: createProcess implictlitly escapes and quotes command line parameters
#8802: createProcess implictlitly escapes and quotes command line parameters -------------------------------------+------------------------------------- Reporter: jstolarek | Owner: Type: bug | Status: new Priority: high | Milestone: 7.8.1 Component: libraries/process | Version: 7.9 Keywords: | Operating System: Unknown/Multiple Architecture: Unknown/Multiple | Type of failure: Runtime crash Difficulty: Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | -------------------------------------+------------------------------------- I've just run into severe usability issue with the `process` library. It turns out that it performs quoting and escaping of command line parameters. The result is that it is impossible to run many shell commands that contain "unsafe" characters: {{{ diff <(echo $ENV_FOO) <(echo $ENV_BAR) awk 'NR<=10' Foo.txt }}} because they are turned into some nonsense. The culprit is `translate` function in libraries/process/System/Process/Internal.hs. I'm not fixing this myself as I assume there was some motivation behind escaping and quoting. I'm assigning high priority as it makes impossible to create shell processes correctly. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8802 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8802: createProcess implictlitly escapes and quotes command line parameters --------------------------------------+------------------------------------ Reporter: jstolarek | Owner: Type: bug | Status: new Priority: high | Milestone: 7.8.1 Component: libraries/process | Version: 7.9 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: --------------------------------------+------------------------------------ Comment (by nomeata): I guess you talking about `runCommand` (which goes via the shell), and not `runProcess` (which executes programs directly)? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8802#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8802: createProcess implictlitly escapes and quotes command line parameters --------------------------------------+------------------------------------ Reporter: jstolarek | Owner: Type: bug | Status: new Priority: high | Milestone: 7.8.1 Component: libraries/process | Version: 7.9 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: --------------------------------------+------------------------------------ Comment (by jstolarek): I am thinking about `createProcess`. Both `runCommand` and `runProcess` are planned to be deprecated (haddock documentation already lists them in section "Old deprecated functions"). -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8802#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8802: createProcess implictlitly escapes and quotes command line parameters --------------------------------------+------------------------------------ Reporter: jstolarek | Owner: Type: bug | Status: new Priority: high | Milestone: 7.8.1 Component: libraries/process | Version: 7.9 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: --------------------------------------+------------------------------------ Comment (by nomeata): I see. But the `translate` function is only used on Windows, for reasons explained in `-- Escaping commands for shells` in System.Process.Internals – is your bug report about Windows? Can you maybe give a concrete example of what should work and does not, so that it can be turned into a testcase? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8802#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8802: createProcess implictlitly escapes and quotes command line parameters --------------------------------------+------------------------------------ Reporter: jstolarek | Owner: Type: bug | Status: new Priority: high | Milestone: 7.8.1 Component: libraries/process | Version: 7.9 Resolution: | Keywords: Operating System: Linux | Architecture: Unknown/Multiple Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: --------------------------------------+------------------------------------ Changes (by jstolarek): * os: Unknown/Multiple => Linux Comment: Replying to [comment:3 nomeata]:
But the `translate` function is only used on Windows No:
{{{ translate :: String -> String #if mingw32_HOST_OS translate xs = '"' : snd (foldr escape (True,"\"") xs) where escape '"' (_, str) = (True, '\\' : '"' : str) escape '\\' (True, str) = (True, '\\' : '\\' : str) escape '\\' (False, str) = (False, '\\' : str) escape c (_, str) = (False, c : str) -- See long comment above for what this function is trying to do. -- -- The Bool passed back along the string is True iff the -- rest of the string is a sequence of backslashes followed by -- a double quote. #else translate "" = "''" translate str -- goodChar is a pessimistic predicate, such that if an argument is -- non-empty and only contains goodChars, then there is no need to -- do any quoting or escaping | all goodChar str = str | otherwise = '\'' : foldr escape "'" str where escape '\'' = showString "'\\''" escape c = showChar c goodChar c = isAlphaNum c || c `elem` "-_.,/" #endif }}}
is your bug report about Windows No, it's about Linux (does Windows even have awk?)
-- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8802#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8802: createProcess implictlitly escapes and quotes command line parameters --------------------------------------+------------------------------------ Reporter: jstolarek | Owner: Type: bug | Status: new Priority: high | Milestone: 7.8.1 Component: libraries/process | Version: 7.9 Resolution: | Keywords: Operating System: Linux | Architecture: Unknown/Multiple Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: --------------------------------------+------------------------------------ Comment (by jstolarek): The example I just attached fails because parameters to `diff` are surrounded with `'` - see comment in the file. This is exactly what `translate` function does. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8802#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8802: createProcess implictlitly escapes and quotes command line parameters --------------------------------------+------------------------------------ Reporter: jstolarek | Owner: Type: bug | Status: closed Priority: high | Milestone: 7.8.1 Component: libraries/process | Version: 7.9 Resolution: invalid | Keywords: Operating System: Linux | Architecture: Unknown/Multiple Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: --------------------------------------+------------------------------------ Changes (by nomeata): * status: new => closed * resolution: => invalid Comment: Well, if you check where `translate` is being used: It is only used on Windows (and on Linux when ''showing'' the command). In your example you are using `proc`, where the main feature is precisely that it is ''not'' being passed through the shell. This is important: With `proc` it is safe to call `proc "echo" [possibly_malicous_string]`. So it is correct behaviour that you cannot used shell features with `proc`. Any security-aware code should only use `proc` or be very careful when using `shell`. If you want shell features, use `shell` instead of `proc`. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8802#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
Well, if you check where `translate` is being used: It is only used on Windows Hm... looks like you're right. In that case which part of the code quotes
#8802: createProcess implictlitly escapes and quotes command line parameters --------------------------------------+------------------------------------ Reporter: jstolarek | Owner: Type: bug | Status: closed Priority: high | Milestone: 7.8.1 Component: libraries/process | Version: 7.9 Resolution: invalid | Keywords: Operating System: Linux | Architecture: Unknown/Multiple Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: --------------------------------------+------------------------------------ Comment (by jstolarek): Replying to [comment:6 nomeata]: parameters to `proc`? Because this clearly is performed at some point.
you are using `proc`, where the main feature is precisely that it is ''not'' being passed through the shell. Ah, now I see. This is poorly documented. Documentation for `proc` says:
With proc it is safe to call proc "echo" [possibly_malicous_string] I don't understand this. Could you give example of how
Construct a `CreateProcess` record for passing to `createProcess`, representing '''a raw command with arguments'''. [highlight by me] My understanding was that "raw command" is supposed to be a shell command. Now I see that `proc` is intended to create a raw process, not a shell command (which is implied by the function's name but certainly not by its documentation). possibly_malicous_string could be dangerous (assuming characters are not escaped)?
Any security-aware code should only use proc Please explain why. If I write a Haskell program that runs external command I can do a lot of bad things even when parameters to `proc` are escaped.
If you want shell features, use shell instead of proc Problem with `shell` is that it runs `sh` shell, not `bash`. Replacing `proc` with `shell` in my example code gives:
{{{ /bin/sh: -c: line 0: syntax error near unexpected token `(' /bin/sh: -c: line 0: `diff <(echo $FOO) <(echo $BAR)' }}} I don't think this ticket should be closed - this is at least a documentation bug. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8802#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
Well, if you check where translate is being used: It is only used on Windows
Hm... looks like you're right. In that case which part of the code quotes parameters to proc? Because this clearly is performed at some
With proc it is safe to call proc "echo" [possibly_malicous_string]
I don't understand this. Could you give example of how
#8802: createProcess implictlitly escapes and quotes command line parameters --------------------------------------+------------------------------------ Reporter: jstolarek | Owner: Type: bug | Status: closed Priority: high | Milestone: 7.8.1 Component: libraries/process | Version: 7.9 Resolution: invalid | Keywords: Operating System: Linux | Architecture: Unknown/Multiple Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: --------------------------------------+------------------------------------ Comment (by nomeata): point. No, they are not escaped and they need not to be; they are put in separate strings and passed to `execve`. Escaping is only required if you use the shell to execute the program – if you don’t use the shell, no escaping is required. possibly_malicous_string could be dangerous (assuming characters are not escaped)? {{{ possibly_malicous_string = "$(rm -rf /)" }}}
If you want shell features, use shell instead of proc
Problem with shell is that it runs sh shell, not bash. Replacing proc with shell in my example code gives:
/bin/sh: -c: line 0: syntax error near unexpected token `(' /bin/sh: -c: line 0: `diff <(echo $FOO) <(echo $BAR)'
Well, if you want a different shell than your system default, I guess you need to invoke it explicitly: {{{ proc "bash" ["-c", some_bash_script] }}}
I don't think this ticket should be closed - this is at least a documentation bug.
I wouldn’t call it a bug; the semantics of `proc` vs. `shell` are quite standard and expected, at least with some background in Unix systems assumed. But of course there is always room for improvement. Any suggestions? Maybe “Because the command is executed directly, and not via a shell, the arguments do not need to be escaped, but you cannot use shell features like output redirection”? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8802#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC