January 2006 - template-haskell

automatic instances for pretty printing and parsing
by Johannes Waldmann 09 Sep '06

09 Sep '06

Dear all, I use Text.PrettyPrint.HughesPJ and Text.ParserCombinators.Parsec heavily so I made some DrIFT rules for deriving the "obvious" instances for class ToDoc a where toDoc :: a -> Doc class Reader a where readerPrec :: Int -> Parser a http://141.57.11.163/cgi-bin/cvsweb/drift/src/UserRuleToDoc.hs?rev=1.5 Still I'd like to avoid external preprocessors (for hygienic reasons, and because they do not seem to fit well with the Cabal model) so I'd like to investigate whether I could get the same effect with Template Haskell (in ghc) - and what this would cost. Perhaps someone has solved a similar problem before. On the other hand, HaXml does need DrIFT it seems. -- -- Johannes Waldmann -- Tel/Fax (0341) 3076 6479/80 -- ---- http://www.imn.htwk-leipzig.de/~waldmann/ -------

3 2

RE: [Template-haskell] quasi quotes and Q monad
by Simon Peyton-Jones 14 Apr '06

14 Apr '06

I've read your message, but I can't figure out what problem you are trying to solve. Can you give a small example that demonstrates it? Simon | -----Original Message----- | From: template-haskell-bounces(a)haskell.org [mailto:template-haskell-bounces@haskell.org] On | Behalf Of Ch. A. Herrmann | Sent: 02 January 2006 16:17 | To: template-haskell(a)haskell.org | Subject: [Template-haskell] quasi quotes and Q monad | | Dear TH experts, | | I have a problem concerning the interaction of quasi quotes and the | quotation monad. Assume a code generating function f (... -> Q Exp) | which is parameterized by a code generating function g (of type | Exp -> Q Exp, or(?) Q Exp -> Q Exp). | | Expressing the problem in the simplest form, the actual instance for g | is (\x -> [| h($x) |]), where h is a toplevel Haskell function working | on arbitrary types, and function f instantiates x | with an expression which consists just of a single variable (VarE). In | order to splice x in the code ($x), the type of x must be (Q Exp). The | reason for that, as mentioned in the 2002 paper | by Sheard and Peyton Jones "Template Metaprogramming for Haskell", is | that the computation of x must be able to access the Q monad. The place | inside f where the actual name for the variable x is generated, has | already access to the Q monad and the *result* of g is | embedded in this monad, no problem. However, I cannot figure out how | this monad can be passed as an *argument* to g and conceptually, there | is no justification to pass this monad: it is just an offspring version | of the one where the lexical scope of g belongs to. | | The value I want to pass for x is of type Exp. Of course, I could turn | this type into (Q Exp) by applying return, but this artificial instance | of the Q monad would come from nowhere, not being connected with the | regular instance used, e.g., for the fresh name generation. | | Especially, I have the following questions: | * Is there a simple solution to this problem? If so, please tell me and | forget about the following questions. | * Is the quasi quote mechanism at all appropriate for what I want to do | or should one better change to the concrete AST representation? That | | would be unfortunate because my aim is to develop Template Haskell | examples which demonstrate ease of use. | * If return is used to turn an expression into monadic form before | splicing, is it possible that | (a) the consistency of fresh name generation is lost, even if one | does the name generation for the spliced expression oneself, | (b) something else goes wrong? | | Many thanks in advance and a Happy New Year | -- | Christoph Herrmann | | | | _______________________________________________ | template-haskell mailing list | template-haskell(a)haskell.org | http://www.haskell.org/mailman/listinfo/template-haskell

3 3

Re: [Template-haskell] new TH tutorial (request for comments)
by Johannes Ahlmann 26 Jan '06

26 Jan '06

Bulat Ziganshin wrote: > i hope that you don't crosspost your answer to maillist by accident :) > at least i want to check my imhos with other TH users sorry for not answering sooner... > i started (and not finished) writing my own TH tutorial. its first > version is at http://freearc.narod.ru/th.htm and current version is at > http://freearc.narod.ru/th3.htm i tried linking to your tutorial, but somehow the host "narod.ru" is blacklisted in the wiki. > quasi-quotation is the mechanism which should hide much of this > complexity for the real user. but does it accomplish this goal? i > think no - any rather complex TH work can't be written without > knowledge about Exp, Q and other low-level stuff. hmm, quasi-quotes aren't all bad, but i agree that for most real applications you'll have to go far beyond what Quasi-Quotes are trying to hide from you. > so imho starting from the low-level > stuff really goes to the economy of reader's time and mind. with > current state of TH it's better to study it as the AST-generation tool > with some help in creating some ASTs rather than macro-generation tool > which has some advanced low-level additions. i don't necessarily disagree, but my "tutorial" was less supposed to explain TH once and for all, but to give a _readable_ account what it can be used for. so maybe our ideas what a tutorial is supposed to do also differ... > i have even more radical offer - just join our tutorials so that my > part explains low-level stuff and quasi-quotation mechanism while you > say about 'Name/''Name/reification and give all these neat examples of > TH usage. plus join our explanations of low-level stuff i have considered all your criticisms in my tutorial and changed it to that end. but some alterations would have forced me to basically rewrite it and i don't have time for that right now :) i am not sure that low-level first is the right approach for the kind of tutorial i was writing, but i do invite you to do any chnages to the wiki-page you want to. after all i hope that others will improve on my work! > imho, the final goal of this > project must be to give one enough explanations that he don't need to > read anything more on TH to use all its features and understand all > what he does. i don't see that to be possible in a tutorial style document. there are so many cases to cover that it becomes more of a paper than a casual tutorial. in my mind, looking at the code or some documentation will never be done away by a TH tutorial. but maybe you can do it! >>> 4. your tutorial does not mention many non-trivial details of TH, >>> [... for] example, difference between mkName and newName. i was aware of the differences; it's just that all the different ways of name resolution can be quite overwhelming. the difference of mkName and newName is buried somewhere deep down in the tutorial, just not presented in a very structured fashion. maybe it would be better to introduce the reader in a structered way to name resolution in TH! thank you for your criticisms, i tried to address every one of them in the few changes i made to my tutorial, Johannes Ahlmann

1 0

Fwd: Re[2]: [Template-haskell] new TH tutorial (request for comments)
by Bulat Ziganshin 25 Jan '06

25 Jan '06

you are not answered, may be because my private message not reached you? This is a forwarded message From: Bulat Ziganshin <bulatz(a)HotPOP.com> To: Johannes Ahlmann <softpro(a)gmx.net> Date: Monday, January 23, 2006, 8:24:34 PM Subject: [Template-haskell] new TH tutorial (request for comments) ===8<==============Original message text=============== Hello Johannes, Monday, January 23, 2006, 2:14:13 AM, you wrote: i hope that you don't crosspost your answer to maillist by accident :) at least i want to check my imhos with other TH users JA> first off, thanks a lot for your criticism! after sitting hours and JA> hours learning TH and writing this tutorial, obviously some pretty JA> visible quirks got into the "tutorial". and i'll try to fix those ASAP. JA> but with the little documentation/tutorials/readable examples of TH i JA> could find, i thought my attempt at a tutorial would be better than not JA> having it at all. you are absolutely true. i don't against any possible TH titorial, and yours contains some things that i really like, especially examples and ease in introducing rather complex concepts JA> the "tutorial" tries to teach by example because that's how i learn JA> best. obviously this won't convey the many intricate details of TH but JA> it gives you a nice overview of whether TH is something you would want JA> to get involved in. i agree that "one example worth thousand explanatios". but i'm disagree with the second sentence - if you just give overview of TH then it's not tutorial any more, it's just like an excursion, but before real using of TH one must read some other papers. does this make much meaning? i started (and not finished) writing my own TH tutorial. its first version is at http://freearc.narod.ru/th.htm and current version is at http://freearc.narod.ru/th3.htm may be my approach too boring and verbose, but i tried to eliminate problem of original TH paper - what it introduces pretty-looking but absolutely bottomless code for printf on the first page and going to really explain it only on the last pages. instead i selected a rather complex but clear way of teaching how to use low-level TH constructs and then advertize high-level constructs just as shortcuts which helps to write code easier, but in fact does the same work TH is very novel and unusual tool and i think that just this makes its learning and usage so difficult. sometimes i think that my problems in mastering TH is like problems of engineer which is used calculator and then goes to write programs in typeful language. are he ever thinks about difference between floating and integer values? :) are we ever thinks that 'x' in pattern and expression is so different beasts? quasi-quotation is the mechanism which should hide much of this complexity for the real user. but does it accomplish this goal? i think no - any rather complex TH work can't be written without knowledge about Exp, Q and other low-level stuff. so we anyway should explain to the user both low-level and high-level things, and i think it's better to go in my way rather than start from high-level stuff and then break conception already builded in user's mind by talking about all these low-level things. so imho starting from the low-level stuff really goes to the economy of reader's time and mind. with current state of TH it's better to study it as the AST-generation tool with some help in creating some ASTs rather than macro-generation tool which has some advanced low-level additions. imho, as usual JA> Bulat Ziganshin wrote: >> you >> describe ASTs without prior giving reader knowledge about how they >> will be used. i think that prior information about splicing and that a >> spliced expression must return AST representing list of declarations >> or expression, would help here. JA> yes, a little more introductory text might be needed here. i have even more radical offer - just join our tutorials so that my part explains low-level stuff and quasi-quotation mechanism while you say about 'Name/''Name/reification and give all these neat examples of TH usage. plus join our explanations of low-level stuff even together these two tutorials don't cover all the TH intrinsics, but at least we can start with it. imho, the final goal of this project must be to give one enough explanations that he don't need to read anything more on TH to use all its features and understand all what he does. of course, that is the long-standing goal, but at least i try to explain im my tutorial as much as i can >> 2. actually, splicing is reverse >> operation for the quasi-quotation, which don't get your speciall >> attention. and that is another tutorial leak - quasi-quotation is not >> so simple and obvious operation, as seems at the first look. JA> i'm not trying to explain how TH works, but how to use it. and from a JA> users perspective quasi-quotes do a lot of background work (which IMHO JA> the user doesn't necessarily have to be aware of as a TH novice) i think that TH papers contains only the information essential to using TH (and we both don't read anything else ;). another question is that some explanations can be divided - in order to start using q.q. one don't need to know all about them. but i think that any deliberate using of q.q. requires knowledge that it generates ExpQ, i.e. it's some computation in Q monad that returns Exp; and that spliced subexpressions should also have this type. >> later you call quasi-quotation as "reification" and that's farther >> complicates the terminology - in TH, reification, i.e. call to "reify" >> function, retrieves information about given Name from Q monad >> environment, while quasi-quotation is a syntax sugar what allows to >> write AST expression in the form of literal Haskell code JA> hmm, maybe i just got the nomenclature wrong, but as far as i understood JA> it, everywhere in metaprogramming what quasi-quotes do is called JA> "reification". but that might be a mixup on my part. and even more JA> gravely, i might have presented the terms in a confusing fashion. i can't say about other metaprogramming tools because i don't know any other than CPP and TH :) but at least TH docs use this term only in respect to "reify" function >> 4. your tutorial does not mention many non-trivial details of TH, >> [... for] example, difference between mkName and newName. JA> yes, i should add that. but the many forms of "dyn", quoted symbols, JA> mkName, newName are very confusing even to me; but maybe that's the JA> problem :) see at the Name definition. newName and its equivalent - introducing new variable inside of q.q. - produces a "unique name" with NameU flavour. Q monad provides sequential numbering of all these "unique names", so that names generated by different invocations of newName cannot coincide. mkName/dyn just wraps the String with identifier in the Name constructor with NameS/NameQ flavour so that the resulting code will use just exact identifier you specified. that allows to bind names dynamically - if you use (mkName "x") in different parts of your TH code, this "x" name nevertheless will refer to the same "x" variable - of course, if it is accessble in the generated code. you can see it as unsafe trick to significantly easify building of AST; moreover, in many cases generated code will be safe because there is no other "x" variables and no possibility that this variable will be redefined in some inner code. but in overall it's unsafe technique and i selected just avoiding saying to novices about it instead of saying and then explaining why it should not be used and then explaining where is still may be used :))) quoted symbols is like local ones generated by mkName/dyn, but they use NameG flavour because they are statically bounded at the context where '/'' was used. that allows to pass references to the global identifiers between modules even if identifier you use don't imported by the module where your code is spliced. afaik, implicit importing performed in such case. for example: module MyTHLibrary: import Binary deriveBinary t = let binaryType = ''Binary in .... (some code that generates derived Binary instance for the given type) module Application: import MyTHLibrary type Binary = String $(deriveBinary ''Int) in this case, spliced code will contain reference to the Binary class available in the MyTHLibrary module. second module don'y import this definition, moreover it defines another symbol with the same name. nevertheless, TH successfully uses the appropriate definition of Binary when it splices generated code. in another words, reference to Binary resolved statically, at the place of using '/'', while for mkName (which also can define qualified names), this reference is resolved dynamically, according to the context where the top-level SPLICING occurs >> a "Restrictions" >> chapter is non-readable in some parts - a least, i understand "In its >> current implementation reificiation of non-quasi-quoted code seems >> very restricted" after heavy thinking and only because i already know >> that. and i don't understand the "Easily pattern matching the tree based >> on its nodes' types seems limited." JA> yes, it's too much work in progress; i'll clean that up. JA> about the node types: i was alluding to the fact that when you want to JA> dispatch on some cases of an overloaded function, figuring out which JA> version you were dealing with and which types the operands had could be JA> tricky... now i understands more, but not all. really, i never tried to reify expressions and so don't understand problems in this area :) -- Best regards, Bulat mailto:bulatz@HotPOP.com ===8<===========End of original message text=========== -- Best regards, Bulat mailto:bulatz@HotPOP.com

1 0

new TH tutorial (request for comments)
by Johannes Ahlmann 23 Jan '06

23 Jan '06

hi, (excuse the repost, but i guess my last post wasn't presented in a way that would readily incite responses...) i've recently written a tutorial on template haskell located at (http://www.haskell.org/hawiki/TemplateHaskellTutorial) it contains examples of loop unrolling, code transformation, etc and i would very much appreciate any feedback or corrections. thanks a lot, Johannes Ahlmann

3 4

RE: [Template-haskell] new TH tutorial (request for comments)
by Simon Peyton-Jones 23 Jan '06

23 Jan '06

I understand that the intention is to migrate all of Hawiki --> HaskellWiki They differ in technology (MoinMoin --> MediWiki), but not in intent. (Except that the new HaskellWiki has taken the opportunity to make explicit that it has a very permissive license, whereas the Hawiki stuff is a bit unclear.) John Peterson's recent posts to the Haskell mailing list amplify. Simon | -----Original Message----- | From: Johannes Ahlmann [mailto:softpro@gmx.net] | Sent: 22 January 2006 23:15 | To: Simon Peyton-Jones | Cc: template-haskell(a)haskell.org | Subject: Re: [Template-haskell] new TH tutorial (request for comments) | | > It'd be great to migrate both to the new HaskellWiki, so that GHC's user | > documentation is uniformly in HaskellWiki. Would you feel like doing | > that? | | i could easily do that, but the sheer abundance of haskell.org wikis is | starting to confuse me! how should the old locations of wiki pages on | "hawiki" be treated? should they relay the user to the new wiki? and how | about links to other hawiki-pages? | | Johannes

1 0

RE: [Template-haskell] new TH tutorial (request for comments)
by Simon Peyton-Jones 22 Jan '06

22 Jan '06

Johannes and other TH folk Excellent work! We have just switched GHC's web pages to use Wikis much more. The basic plan is User documentation: in HaskellWiki (uses MediaWiki underlying) Developer documentation: in a Trac wiki If you click the Documentation link in the sidebar of GHC's home page, you'll now get to http://haskell.org/haskellwiki/GHC:Documentation There's a section on "Collaborative documentation", which points to more HaskellWiki pages. Including one on Template Haskell. Well, nearly. Actually the TH link points to the old MoinMoin Wiki: http://www.haskell.org/hawiki/TemplateHaskell and that in turn points to your tutorial. It'd be great to migrate both to the new HaskellWiki, so that GHC's user documentation is uniformly in HaskellWiki. Would you feel like doing that? Simon | -----Original Message----- | From: template-haskell-bounces(a)haskell.org [mailto:template-haskell-bounces@haskell.org] On | Behalf Of Johannes Ahlmann | Sent: 20 January 2006 04:10 | To: template-haskell(a)haskell.org | Subject: [Template-haskell] new TH tutorial (request for comments) | | hi, | | (excuse the repost, but i guess my last post wasn't presented in a way | that would readily incite responses...) | | i've recently written a tutorial on template haskell located at | (http://www.haskell.org/hawiki/TemplateHaskellTutorial) | | it contains examples of loop unrolling, code transformation, etc and i | would very much appreciate any feedback or corrections. | | thanks a lot, | | Johannes Ahlmann | _______________________________________________ | template-haskell mailing list | template-haskell(a)haskell.org | http://www.haskell.org/mailman/listinfo/template-haskell

2 1

Stuck... is it even possible?
by Tim Newsham 20 Jan '06

20 Jan '06

I'm having problems with my template generator. I have everything working except for the "flen" function which I'm faking for now: http://lava.net/~newsham/x/jethr0/ The directory has 4 source files (with or without .txt extensions for your browsing enjoyment): ByteContainer.lhs - defining my classes IP.lhs - defining my test data type Gen.lhs - defining my TH generator tst.lhs - my test case. You can see it go by running "ghc -c -fth -ddump-splices tst.lhs". The problem: flen should compute and return the size of each field. Concretely: flen ("Word8",_,_) should compute "containerLength newWord8" flen ("Word16",_,_) should compute "containerLength newWord16" flen ("IP",_,_) should compute "containerLength newIP" flen (n,_,_) should compute "containerLength" of ("new"++n) The flen function is used during template generation, and needs to know the integer length while expanding the template (ie. flen doesnt generate anything). Is this possible? Tim Newsham http://www.lava.net/~newsham/

2 3

TH question
by Tim Newsham 19 Jan '06

19 Jan '06

Hi, I have yet to write anything in TH, although I've been toying with it a little bit and asking some questions on IRC. I have come across a problem that doesn't seem to have a good solution without using some sort of preprocessor. I want to take a simple spec and generate some marshalling code using a straighforward transformation. The spec should look something like: > $(declPkt "ip" [ > ("Word8", "v_hl", "0x45"), > ("Word8", "tos", "0"), > ("Word16", "len", "0"), > ("Word16", "id", "0"), > ("Word16", "off", "0"), > ("Word8", "ttl", "64"), > ("Word8", "p", "6"), > ("Word16", "sum", "0"), > ("Word32", "src", "0"), > ("Word32", "dst", "0")]) and should generate something like: > data IP = IP { > ip_v_hl :: Word8, > ip_tos :: Word8, > ip_len :: Word16, > ip_id :: Word16, > ip_off :: Word16, > ip_ttl :: Word8, > ip_p :: Word8, > ip_sum :: Word16, > ip_src :: Word32, > ip_dst :: Word32 > } deriving Show > newIP = IP 0x45 0 20 0 0 64 6 0 0 0 > > instance ByteContainer IP where > getByteAt b 0 = getByteAt (ip_v_hl b) 0 > getByteAt b 1 = getByteAt (ip_tos b) 0 > getByteAt b 2 = getByteAt (ip_len b) 0 > getByteAt b 3 = getByteAt (ip_len b) 1 > getByteAt b 4 = getByteAt (ip_id b) 0 > getByteAt b 5 = getByteAt (ip_id b) 1 > getByteAt b 6 = getByteAt (ip_off b) 0 > getByteAt b 7 = getByteAt (ip_off b) 1 > getByteAt b 8 = getByteAt (ip_ttl b) 0 > getByteAt b 9 = getByteAt (ip_p b) 0 > getByteAt b 10 = getByteAt (ip_sum b) 0 > getByteAt b 11 = getByteAt (ip_sum b) 1 > getByteAt b 12 = getByteAt (ip_src b) 0 > getByteAt b 13 = getByteAt (ip_src b) 1 > getByteAt b 14 = getByteAt (ip_src b) 2 > getByteAt b 15 = getByteAt (ip_src b) 3 > getByteAt b 16 = getByteAt (ip_dst b) 0 > getByteAt b 17 = getByteAt (ip_dst b) 1 > getByteAt b 18 = getByteAt (ip_dst b) 2 > getByteAt b 19 = getByteAt (ip_dst b) 3 > > containerLength b = 20 > > makeContainer b = IP { > ip_v_hl = makeContainer (Slice b 0), > ip_tos = makeContainer (Slice b 1), > ip_len = makeContainer (Slice b 2), > ip_id = makeContainer (Slice b 4), > ip_off = makeContainer (Slice b 6), > ip_ttl = makeContainer (Slice b 8), > ip_p = makeContainer (Slice b 9), > ip_sum = makeContainer (Slice b 10), > ip_src = makeContainer (Slice b 12), > ip_dst = makeContainer (Slice b 16) > } This definitely looks feasible using TH. However, it looks like I might have to mess with ASTs manually. My questions is -- is there an easy way to perform this transformation making use of quasi-quotes for much of the code, or will it be necessary to construct all of the ASTs manually? This seems like something that should have a fairly simple solution but so far I haven't been able to come up with one :(. Tim Newsham http://www.lava.net/~newsham/

2 2

FW: TH page
by Simon Peyton-Jones 06 Jan '06

06 Jan '06

Anyone willing to volunteer? Not a big task TH Hawiki page: http://www.haskell.org/hawiki/TemplateHaskell TH page to merge into it: http://www.haskell.org/th Simon -----Original Message----- From: Ian Lynagh [mailto:igloo@earth.li] Sent: 06 January 2006 02:22 To: Simon Peyton-Jones Subject: Re: TH page Hi Simon, On Fri, Dec 16, 2005 at 02:44:52PM -0000, Simon Peyton-Jones wrote: > > I rather think that haskell.org/th could be merged into the Hawiki page > for TH. What do you think? I'm happy for someone to merge it. Thanks Ian

1 0