[GHC] #11215: Line endings in quasiquotations are not normalised
#11215: Line endings in quasiquotations are not normalised -------------------------------------+------------------------------------- Reporter: refold | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.3 (Parser) | Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: Incorrect result Unknown/Multiple | at runtime Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- Consider the following [http://hackage.haskell.org/package/raw-strings-qq simple quasiquoter]: {{{#!hs foo = QuasiQuoter { quoteExp = return . LitE . StringL [...] }}} When used like this: {{{#!hs bar :: String bar = [foo|FOO BAR BAZ|] }}} the string `bar` will contain either LF or CRLF line endings depending on which convention is used in the file. This is consistent with documentation, if a bit surprising. However, the problem is that version control systems (e.g. Git) usually store text files using Unix line endings, but allow checking them out in a platform- specific format. So this behaviour gives rise to subtle semantics changes in the compiled program that depend on the user's VCS settings. Therefore IMO it makes sense to always normalise CRLF to LF in quasiquotations. Originally reported [https://github.com/23Skidoo/raw-strings-qq/issues/1 here], a test case can be found [https://github.com/23Skidoo/raw-strings- qq/blob/master/test/Test.hs here]. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11215 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11215: Line endings in quasiquotations are not normalised -------------------------------------+------------------------------------- Reporter: refold | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.3 (Parser) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by goldfire): I'm not convinced the normalizing is something GHC should do. The quasi- quoter could easily do this. Maybe a quasi-quoter wants, for some inscrutable reason, to tell the difference between LF and CRLF. (Perhaps to warn a user about a bad newline encoding. But I admit this possibility is a stretch.) If GHC normalizes, then quasi-quoters have lost this ability. On the other hand, it's dead easy for a quasi-quoter to do this itself. If you feel strongly, I could see the benefit of having a `normaliseLineEndings :: String -> String` available in `Language.Haskell.TH.Quote` but even that seems a little special-cased. And I'm certainly in favor of making this louder in the documentation. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11215#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11215: Line endings in quasiquotations are not normalised -------------------------------------+------------------------------------- Reporter: refold | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.3 (Parser) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by refold): I'll summarise/reiterate my arguments for doing this in GHC: * Other languages do this. For example, multiline raw string literals in Python will have LF line endings regardless of the source file encoding. Same in C++. * Requiring the users to do it themselves is error-prone. Basically everyone who implements a quasiquoter now has to remember about this quirk. * The feature hurts portability but benefits no one, because line endings can change depending on which platform the program is compiled. * This behaviour already caused a [https://github.com/23Skidoo/raw- strings-qq/issues/1 bug in real code], but there're no examples of code actually depending on it. * The problem is exacerbated by the automatic newline conversion in version control systems. * If you really want to preserve this behaviour, it should be opt-in instead of opt-out. Adding `normaliseLineEndings` to the standard library is not a solution. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11215#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11215: Line endings in quasiquotations are not normalised -------------------------------------+------------------------------------- Reporter: refold | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.3 (Parser) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by rwbarton): I'd say at a minimum we should translate CRLF to LF on Windows. I'm surprised the underlying file IO doesn't do this already. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11215#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11215: Line endings in quasiquotations are not normalised -------------------------------------+------------------------------------- Reporter: refold | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.3 (Parser) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: #8424 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by thomie): * related: => #8424 Comment: #8424 is related (or duplicate?) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11215#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11215: Line endings in quasiquotations are not normalised -------------------------------------+------------------------------------- Reporter: refold | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.3 (Parser) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: #8424 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by refold): Yes, #8424 seems to be the same issue. Another example of this behaviour causing problems in real code. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11215#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11215: Line endings in quasiquotations are not normalised -------------------------------------+------------------------------------- Reporter: refold | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.3 (Parser) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: #8424 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by goldfire): Replying to [comment:2 refold]:
I'll summarise/reiterate my arguments for doing this in GHC:
These are convincing arguments. Especially the point about a bug being found in the wild, and the fact that there are two tickets about this issue. Go for it: it's quite easy. I think just adding a processing pass on [https://github.com/ghc/ghc/blob/master/compiler/rename/RnSplice.hs#L344 line 344] (beginning `quoteExpr =`) of !RnSplice should do it. And we'll need a test. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11215#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11215: Line endings in quasiquotations are not normalised -------------------------------------+------------------------------------- Reporter: refold | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.3 (Parser) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: #8424 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by refold): Thanks for the pointer. I'll try to come up with a patch, though I think that the right place to fix this is the lexer (like suggested in #8424). -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11215#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11215: Line endings in quasiquotations are not normalised -------------------------------------+------------------------------------- Reporter: refold | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.3 (Parser) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: #8424 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by goldfire): Perhaps you're right that the lexer is a better place to address this. Up to you. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11215#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC