GHC lexer questions

Hi all, I have a few questions about the GHC lexer (currently using GHC Glasgow Haskell Compiler, Version 6.13.20100320, for Haskell 98, stage 2 booted by GHC version 6.12.1) And I notice that when invoking the lexer I get a few unexpected results that I was hoping someone could clarify. The first part has to do with braces. Opening braces are lexed fine, but the closing ones throw an error. I assume the lexer is keeping some kind of state and trying to do brace matching? (but isn't this supposed to be done by the parser?) ?parseLine("{")->tag cStatelessParseResultSOk ?parseLine("}")->tag cStatelessParseResultSFailed and ?parseLine("{-")->tag cStatelessParseResultSOk ?parseLine("-}")->tag cStatelessParseResultSFailed and in a statement "let foo = .." "foo" isn't lexed as an identifier, "f" is lexed as a ITvocurly and the "oo" as an identifier. I find this rather odd, and maybe it's me doing something wrong but doubt it. The original Haskell code I call from FFI is lexSourceString :: String -> IO (StatelessParseResult [Located Token]) lexSourceString source = do buffer <- stringToStringBuffer source let srcLoc = mkSrcLoc (mkFastString "internal:string") 1 1 let dynFlag = defaultDynFlags let result = lexTokenStream buffer srcLoc dynFlag return $ convert result -- | convert the build in ParseResult to out custom StateLessParseResult -- mainly because the State is difficult to marshal and we don't really seem to need it convert :: ParseResult a -> StatelessParseResult a convert (POk _ ty) = SOk ty convert (PFailed src msg) = SFailed src (showSDocDump msg)
participants (1)
-
Phyx