2010/10/26 Stephen Tetley
<stephen.tetley@gmail.com>
The lexer was wrong - but it was the lexer function not the lexer spec
- try the one below.
Note that you have to take 'len' chars from the original input.
Previously you were taking the whole of the "rest-of--input":
lexer :: (TheToken -> P a) -> P a
lexer f input@(_,_,instr) =
case alexScan input 0 of
AlexEOF -> f Eof input
AlexError (pos, _, _) ->
Failed $ showPos pos ++ ": lexical error"
AlexSkip input' len -> lexer f input'
AlexToken (pos, c, str) len act ->
let (Token newpos thetok) = act pos (take len instr)
in f thetok (newpos, c, str)