Hi all, In the process of writing an SQL parser I started by writing a lexer. The code can be found here: http://hpaste.org/fastcgi/hpaste.fcgi/view?id=4736#a4736 You can run it like this in ghci: Prelude SQL.Lexer> runLexer "select * from TABLE order by FIELD" [Token Reserved "select",Token Space " ",Token Operator "*",Token Space " ",Token Reserved "from",Token Space " ",Token Identifier "TABLE",Token Space " ",Token Reserved "order",Token Space " ",Token Reserved "by",Token Space " ",Token Identifier "FIELD"] Since this is pretty much my first Haskell project over 10 lines long, I'm looking for some feedback of any kind. Ultimately I would like to use this lexer to build a functional SQL parser using Parsec. Thanks, Patrick -- ===================== Patrick LeBoutillier Rosemère, Québec, Canada
This may be a better approach then my code. I didn't create a separate
parser/lexer.
-Keith
On Fri, May 8, 2009 at 9:53 PM, Patrick LeBoutillier
Hi all,
In the process of writing an SQL parser I started by writing a lexer. The code can be found here: http://hpaste.org/fastcgi/hpaste.fcgi/view?id=4736#a4736
You can run it like this in ghci:
Prelude SQL.Lexer> runLexer "select * from TABLE order by FIELD" [Token Reserved "select",Token Space " ",Token Operator "*",Token Space " ",Token Reserved "from",Token Space " ",Token Identifier "TABLE",Token Space " ",Token Reserved "order",Token Space " ",Token Reserved "by",Token Space " ",Token Identifier "FIELD"]
Since this is pretty much my first Haskell project over 10 lines long, I'm looking for some feedback of any kind. Ultimately I would like to use this lexer to build a functional SQL parser using Parsec.
Thanks,
Patrick
-- ===================== Patrick LeBoutillier Rosemère, Québec, Canada _______________________________________________ Beginners mailing list Beginners@haskell.org http://www.haskell.org/mailman/listinfo/beginners
Patrick LeBoutillier wrote:
Hi all,
In the process of writing an SQL parser I started by writing a lexer. The code can be found here: http://hpaste.org/fastcgi/hpaste.fcgi/view?id=4736#a4736
You can run it like this in ghci:
Prelude SQL.Lexer> runLexer "select * from TABLE order by FIELD" [Token Reserved "select",Token Space " ",Token Operator "*",Token Space " ",Token Reserved "from",Token Space " ",Token Identifier "TABLE",Token Space " ",Token Reserved "order",Token Space " ",Token Reserved "by",Token Space " ",Token Identifier "FIELD"]
Since this is pretty much my first Haskell project over 10 lines long, I'm looking for some feedback of any kind. Ultimately I would like to use this lexer to build a functional SQL parser using Parsec.
Thanks,
Patrick
Hi Patrick - I like it! I'm still a Haskell beginner, but even to me, your code seems very clear and easy to understand. I've been thinking of doing some simple parsers too, so I was wondering - may I use this code of yours as a base for them? Well done, and thanks for doing this! - Andy
Hi Patrick - I like it! I'm still a Haskell beginner, but even to me, your code seems very clear and easy to understand. I've been thinking of doing some simple parsers too, so I was wondering - may I use this code of yours as a base for them?
Sure, anyone here can use the code if they have any interest in it. Patrick
Well done, and thanks for doing this!
- Andy
Hello Patrick, that's a nice one. I have two notes for you: 1) Why do you handle spaces as tokens? Can't you just skip them completely? It could make the parser simpler. 2) Your lexer does not provide any information about positions of tokens in the source code. You may want to do that in order to: a) report better error messages b) use parsec's `token` function [1] to make parsec work with your tokens. Note that parsec wants you to use its data type `SourcePos` to represent positions. See [2] for a simple example and further stuff. [1]: http://legacy.cs.uu.nl/daan/download/parsec/parsec.html#token [2]: http://legacy.cs.uu.nl/daan/download/parsec/parsec.html#SeperateScanners Sincerely, Jan. On Fri, May 08, 2009 at 09:53:04PM -0400, Patrick LeBoutillier wrote:
Hi all,
In the process of writing an SQL parser I started by writing a lexer. The code can be found here: http://hpaste.org/fastcgi/hpaste.fcgi/view?id=4736#a4736
You can run it like this in ghci:
Prelude SQL.Lexer> runLexer "select * from TABLE order by FIELD" [Token Reserved "select",Token Space " ",Token Operator "*",Token Space " ",Token Reserved "from",Token Space " ",Token Identifier "TABLE",Token Space " ",Token Reserved "order",Token Space " ",Token Reserved "by",Token Space " ",Token Identifier "FIELD"]
Since this is pretty much my first Haskell project over 10 lines long, I'm looking for some feedback of any kind. Ultimately I would like to use this lexer to build a functional SQL parser using Parsec.
Thanks,
Patrick
-- ===================== Patrick LeBoutillier Rosemère, Québec, Canada _______________________________________________ Beginners mailing list Beginners@haskell.org http://www.haskell.org/mailman/listinfo/beginners
-- Heriot-Watt University is a Scottish charity registered under charity number SC000278.
On Mon, May 11, 2009 at 6:33 AM, Jan Jakubuv
Hello Patrick,
that's a nice one. I have two notes for you:
1) Why do you handle spaces as tokens? Can't you just skip them completely? It could make the parser simpler.
I added that as a debug tool (that way I can recreate the SQL exactly and use diff to make sure it tokenised properly). I agree that besides that it's not very useful.
2) Your lexer does not provide any information about positions of tokens in the source code. You may want to do that in order to:
a) report better error messages b) use parsec's `token` function [1] to make parsec work with your tokens. Note that parsec wants you to use its data type `SourcePos` to represent positions. See [2] for a simple example and further stuff.
[1]: http://legacy.cs.uu.nl/daan/download/parsec/parsec.html#token [2]: http://legacy.cs.uu.nl/daan/download/parsec/parsec.html#SeperateScanners
That's for phase 2... :). Actually I was planning on adding that in later. Thanks a lot for the links, I'll have a look at them. Thanks a lot for your feedback, Patrick
Sincerely, Jan.
On Fri, May 08, 2009 at 09:53:04PM -0400, Patrick LeBoutillier wrote:
Hi all,
In the process of writing an SQL parser I started by writing a lexer. The code can be found here: http://hpaste.org/fastcgi/hpaste.fcgi/view?id=4736#a4736
You can run it like this in ghci:
Prelude SQL.Lexer> runLexer "select * from TABLE order by FIELD" [Token Reserved "select",Token Space " ",Token Operator "*",Token Space " ",Token Reserved "from",Token Space " ",Token Identifier "TABLE",Token Space " ",Token Reserved "order",Token Space " ",Token Reserved "by",Token Space " ",Token Identifier "FIELD"]
Since this is pretty much my first Haskell project over 10 lines long, I'm looking for some feedback of any kind. Ultimately I would like to use this lexer to build a functional SQL parser using Parsec.
Thanks,
Patrick
-- ===================== Patrick LeBoutillier Rosemère, Québec, Canada _______________________________________________ Beginners mailing list Beginners@haskell.org http://www.haskell.org/mailman/listinfo/beginners
-- Heriot-Watt University is a Scottish charity registered under charity number SC000278.
-- ===================== Patrick LeBoutillier Rosemère, Québec, Canada
patrick,
i was wondering if parts of the code could be more abstract.
(i'm no genius in that area)
but how about using some high order function for the numeric literal or the
alpohanumeric literal?
maybe a lexer is a hand-wise algorithm, but maybe it can be streched somehow
:-)
comments?
best regards
haroldo
2009/5/8 Patrick LeBoutillier
Hi all,
In the process of writing an SQL parser I started by writing a lexer. The code can be found here: http://hpaste.org/fastcgi/hpaste.fcgi/view?id=4736#a4736
You can run it like this in ghci:
Prelude SQL.Lexer> runLexer "select * from TABLE order by FIELD" [Token Reserved "select",Token Space " ",Token Operator "*",Token Space " ",Token Reserved "from",Token Space " ",Token Identifier "TABLE",Token Space " ",Token Reserved "order",Token Space " ",Token Reserved "by",Token Space " ",Token Identifier "FIELD"]
Since this is pretty much my first Haskell project over 10 lines long, I'm looking for some feedback of any kind. Ultimately I would like to use this lexer to build a functional SQL parser using Parsec.
Thanks,
Patrick
-- ===================== Patrick LeBoutillier Rosemère, Québec, Canada _______________________________________________ Beginners mailing list Beginners@haskell.org http://www.haskell.org/mailman/listinfo/beginners
participants (6)
-
Andy Elvey -
Haroldo Stenger -
Jan Jakubuv -
Keith Sheppard -
Patrick LeBoutillier -
patrick.leboutillier＠gmail.com