Hi,
I have recently been playing with GHC's Lexer.lexer in the ghc-parser-lib package.
Given
module HelloWorld where
main = putStrLn "Hello World!\n"
it produces
stack exec -- lexer-exe ./examples/HelloWorld.hs
Lexing&Parsing: ./examples/HelloWorld.hs
module at (1, 1): module
CONID at (1, 8): CONID
where at (1, 19): where
vocurly at (3, 1): vocurly <==== { is inserted automatically!!
VARID at (3, 1): VARID
= at (3, 6): =
VARID at (3, 8): VARID
STRING at (3, 17): STRING
; at (4, 1): ;
By the example above, the lexer automatically inserts an opening brace (i.e. vocurly) right after 'where'. But it does not insert a matching closing brace (i.e., vccurly), which would lead to a failure in parsing a list of tokens produced by the lexer.
My question is how to use the GHC lexer to produce closing braces as well.
All my code is available
To save your time, the relevant part of the code is as follows:
In app/HaskellLexer.hs,
singleHaskellToken :: P (Located Token)
singleHaskellToken =
Lexer.lexer False
(\locatedToken -> P (\pstate -> POk pstate locatedToken))
tokInfos :: [Terminal Token] -> P (Line, Column, [Terminal Token])
tokInfos s = do
locatedToken <- singleHaskellToken
case locatedToken of
L srcspan ITeof ->
let (start_line, start_col, end_line, end_col) = srcSpanToLineCol srcspan in
return (end_line, end_col, s)
L srcspan tok ->
let (start_line, start_col, end_line, end_col) = srcSpanToLineCol srcspan in
tokInfos (Terminal (fromToken tok) start_line start_col (Just tok) : s)
Thanks in advance
Best regards,
Kwanghoon