Hi Folks,
I am learning how to use the Haskell lexical analyzer tool called "Alex" [1].
I am trying to implement a lexical analyzer for this string (an email "From:" header):
From: "John Doe"
I want to break it up into this list of tokens:
[
From,
DisplayName "John Doe",
Email,
LocalName "john",
Domain "doe.org"
]
Below is my implementation. It works fine if the string doesn't contain a display name. That is, this works fine:
let s = "From: "
alexScanTokens s
However, when I include a display name, I get this error message:
[From*** Exception: lexical error
That is, this results in an error:
let s = "From: \"John Doe\" "
alexScanTokens s
I am guessing that this part of my "Alex" program is causing the error:
\"[a-zA-Z ]+\" { \s -> DisplayName (init (tail s)) }
In Alex the left side is a regular expression:
\"[a-zA-Z ]+\"
and the right side is the action to be taken when a string is found that matches the regular expression:
{ \s -> DisplayName (init (tail s)) }
Any thoughts on what the problem might be?
Here is my lexical analyzer program:
-------------------------------------------------
{
module Main (main) where
}
%wrapper "basic"
$digit = 0-9 -- digits
$alpha = [a-zA-Z] -- alphabetic characters
tokens :-
$white+ ;
From: { \s -> From }
\"[a-zA-Z ]+\" { \s -> DisplayName (init (tail s)) }
\< { \s -> Email }
[$alpha]+@ { \s -> LocalPart (init s) }
[$alpha\.]+> { \s -> Domain (init s) }
{
-- Each action has type :: String -> Token
-- The token type:
data Token =
From |
DisplayName String |
Email |
LocalPart String |
Domain String
deriving (Eq,Show)
main = do
s <- getContents
print (alexScanTokens s)
}
-------------------------------------------------
[1] The "Alex" lexical analyzer tool may be found at this URL: http://www.haskell.org/alex/doc/html/introduction.html
Costello, Roger L.