..Comments are simply discarded, and to be honest I really don't see how they could be kept in general, except in specific pre-defined places (like for Haddock). I'm sure you have ideas on that though. Layout is also not preserved exactly, and do { ... ; ... } results in the same AST as if done with layout instead of { ; }.
just recalling some previous discussions: a) add a whitespace&layout field to every AST construct + should work well enough for source location info and for braces yes/no flags - doesn't work so well for comments: do they belong to the next or the previous item, or are they standalone text? b) add a comment construct to the AST + avoids association issue, comments stand on their own - AST traversals have to deal with comments - AST transformations still have to decide which comments to move with which AST fragments + comments in untransformed code remain c) add source location and layout info to every AST construct, keep comments out of the AST, link source locations to comment sections + keeps AST traversals simple - AST transformations still need to decide which comments to move with which AST fragments + allows finding related comments after AST transformations (the ghc ticket suggests two lookup functions, for comments before/after a given AST fragment) + comments in untransformed code remain d) ..? i'd be interested in other alternatives so far, it seems that c is the way to go (HaRe used the Programatica's token stream as comment storage, with every identifier having source location info, and some heuristics for matching comments to AST fragments), though i'd be interested in other alternatives - keeping the parts of a split-up source representation in sync during transformation and weaving everything back together before presentation proved tedious.
Indeed, and I'm afraid haskell-src-exts will join the crowd in that regard. But to be honest I'm not sure haskell-src-exts *should* do those things you ask for, since the added machinery would be rather heavy-weight for applications that just want the basic stuff, which I guess is the vast majority of applications.
would the added machinery really affect applications that don't care about comments/layout? with (c), there'd be one field not to look at and a comment store to ignore. and the bonus would be to enable a large variety of applications based on source to source transformations (refactoring, pretty printing, desugaring, layout<->braces, optimizations, ..). but perhaps that isn't the target for haskell-src-ext, and things like HaRe are currently hoping/aiming for the ghc api.
ps it is good to hear that src-ext is supported, follows language developments, and is separated from the other parts of your projects!-)
Thanks, and yes I try to keep them as separate as possible, knowing their usefulness to others.
keep up the good work!-) claus