Ok, to answer my own question, I changed nested_comment to
nested_comment :: P (RealLocated Token) -> Action
nested_comment cont span buf len = do
input <- getInput
go (reverse $ lexemeToString buf len) (1::Int) input
It now starts off with the already lexed part.
On Wed, Oct 29, 2014 at 9:04 PM, Alan & Kim Zimmerman
As part of my ongoing efforts to round-trip source code, I have bumped into an issue around file header pragmas, e.g.
{-# LANGUAGE PatternSynonyms #-} {-# Language DeriveFoldable #-} {-# options_ghc -w #-}
In normal mode, when not called from headerInfo, the file header pragmas are lexed enough to generate a warning about an invalid pragma if enabled, and then lexed to completion and returned as an `ITblockComment` if `Opt_KeepRawTokenStream` is enabled.
The relevant Alex rule is
<0> { -- In the "0" mode we ignore these pragmas "{-#" $whitechar* $pragmachar+ / { known_pragma fileHeaderPrags } { nested_comment lexToken } }
The problem is that the tokens returned are
ITblockComment " PatternSynonyms #" ITblockComment " DeriveFoldable #" ITblockComment " -w #"
It is not possible to reproduce the original comment from these.
It looks like nested comment ignores what has been lexed so far
nested_comment :: P (RealLocated Token) -> Action nested_comment cont span _str _len = do ...
So my question is, is there any way to make the returned comment include the prefix part? Perhaps be a specific variation of nested_comment that uses str and len.
Alan