Re: Case-insensitive lexing with Alex

On Jan 28, 2008, at 2:44 PM, Geoffrey Mainland wrote:
map toLower onto your input before you pass it to your lexer? Or do you only want keywords to be case-insensitive?
Just keywords. You can have "Array" or "array" or "aRrAy". -- http://wagerlabs.com

Joel Reymont wrote:
On Jan 28, 2008, at 2:44 PM, Geoffrey Mainland wrote:
map toLower onto your input before you pass it to your lexer? Or do you only want keywords to be case-insensitive?
Just keywords. You can have "Array" or "array" or "aRrAy".
One old trick for reducing the size of lexers is to not put keyword recognition into the lexer rules (that is, don't write rules to match keywords explicitly), but to look up identifiers in a map from strings to keyword tokens. In your case, apply toLower to the identifier before you look it up in the map from keywords to tokens. The C parsing library I gave you a link to off-list does its lexing using this technique if you want a concrete example. Geoff
participants (2)
-
Geoffrey Mainland
-
Joel Reymont