HaXML incorrect interpretation of XML spec!
The XML spec defines the prolog(1) as follows: [22] prolog ::= XMLDecl? Misc* (doctypedecl Misc*)? In other words you can have Misc before AND after the doctype declaration! HaXml defines the prolog(2) as: data Prolog = Prolog (Maybe XMLDecl) (Maybe DocTypeDecl) HaXml therefore does not allow PIs before the beginning of the top level element. This is a problem in practice for people who want to use e.g. XML-Stylesheets(3) where if one is using XSL to produce HTML it is necessary to put the stylesheet declaration BEFORE the root element (if the root element is not HTML). e.g. <?xml version="1.0"?> <?xml:stylesheet type="text/xsl" href="style.xsl"?> <foo id="57" category="2"/> Is there some way to puta PI before the root element in combinators? -Alex- (1) http://www.w3.org/TR/REC-xml/#sec-prolog-dtd (2) http://www.cs.york.ac.uk/fp/HaXml/HaXml/Text.XML.HaXml.Types.html#Prolog (3) http://www.w3.org/TR/xml-stylesheet/ ______________________________________________________________ S. Alexander Jacobson tel:917-770-6565 http://alexjacobson.com
I modified the Prolog type to be data Prolog = Prolog (Maybe XMLDecl) [Misc] (Maybe DocTypeDecl) [Misc] and then modified the Prolog parser (which actually was correct) to actually use the misc values in constucting the Prolog. I replace the prolog function in pretty.hs with prolog (Prolog x m1 dtd m2) = maybe xmldecl x $$ vcat (map misc m1) $$ maybe doctypedecl dtd $$ vcat (map misc m2) and did a few more similar things in Haskell2XML and XML2Haskell (about a 5 min operation) and it all seems to work properly. Implementation question: Why is there so much replicated code in HaXML/Html (parse.hs and pretty.hs) Given that this fix was so very easy and given that the parser was already spec consistent, I now have to assume that there was good reason for the Prolog to be spec inconsistent, but I don't know what it is... Thoughts? -Alex- On Wed, 27 Oct 2004, S. Alexander Jacobson wrote:
The XML spec defines the prolog(1) as follows:
[22] prolog ::= XMLDecl? Misc* (doctypedecl Misc*)?
In other words you can have Misc before AND after the doctype declaration!
HaXml defines the prolog(2) as:
data Prolog = Prolog (Maybe XMLDecl) (Maybe DocTypeDecl)
HaXml therefore does not allow PIs before the beginning of the top level element. This is a problem in practice for people who want to use e.g. XML-Stylesheets(3) where if one is using XSL to produce HTML it is necessary to put the stylesheet declaration BEFORE the root element (if the root element is not HTML). e.g.
<?xml version="1.0"?> <?xml:stylesheet type="text/xsl" href="style.xsl"?> <foo id="57" category="2"/>
Is there some way to puta PI before the root element in combinators?
-Alex-
(1) http://www.w3.org/TR/REC-xml/#sec-prolog-dtd (2) http://www.cs.york.ac.uk/fp/HaXml/HaXml/Text.XML.HaXml.Types.html#Prolog (3) http://www.w3.org/TR/xml-stylesheet/ ______________________________________________________________ S. Alexander Jacobson tel:917-770-6565 http://alexjacobson.com _______________________________________________ Haskell mailing list Haskell@haskell.org http://www.haskell.org/mailman/listinfo/haskell
______________________________________________________________ S. Alexander Jacobson tel:917-770-6565 http://alexjacobson.com
"S. Alexander Jacobson" <alex@alexjacobson.com> writes:
I modified the Prolog type to be data Prolog = Prolog (Maybe XMLDecl) [Misc] (Maybe DocTypeDecl) [Misc] and then modified the Prolog parser
Thanks for spotting this bug and providing a fix. I also note that the XML spec allows "misc*" to follow the document top-level element: document ::= prolog element Misc* and this too is incorrect in HaXml. There may well be other occurrences of the same omission.
Given that this fix was so very easy and given that the parser was already spec consistent, I now have to assume that there was good reason for the Prolog to be spec inconsistent, but I don't know what it is...
I originally assumed that Misc's were unimportant and could be discarded, like comments are discarded by a compiler. I failed to notice that PI's should be passed through to the application.
Implementation question: Why is there so much replicated code in HaXML/Html (parse.hs and pretty.hs)
The HTML parser does some correction of mal-formed input, which is not otherwise permitted by the XML spec. Likewise, the HTML pretty-printer makes some wild and unjustified assumptions about the way that humans like to format their documents, whereas the XML pp is more strictly-conforming. Once XHTML becomes common, the HTML parser/pp will be obsolete. Regards, Malcolm
Is there a good entry point into HaXml? I've now spent some time trying to understand it and feel like I've gotten nowhere. The Haddock documentation enumerates what each function does, but I still don't know how to produce a valid XML document? For example, this is obviously the wrong way to go: simp2 = document $ Document (Prolog Nothing [] Nothing []) [] $ Elem "root" [("attr",AttValue [Left "v\"al"])] [CString False "<<<<<>>&&&"] Because, it produces the obviously wrong: <root attr="v"al"><<<<<>>&&&</root> I assume/hope that the combinators properly encode/escape attribute values and CDATA, but can't figure out how to generate even the simple XML above. And once I've done so, is there a way to put PIs in via the combinators or do I have to import Types and risk have unescaped stuff in my document? -Alex- On Thu, 28 Oct 2004, Malcolm Wallace wrote:
"S. Alexander Jacobson" <alex@alexjacobson.com> writes:
I modified the Prolog type to be data Prolog = Prolog (Maybe XMLDecl) [Misc] (Maybe DocTypeDecl) [Misc] and then modified the Prolog parser
Thanks for spotting this bug and providing a fix. I also note that the XML spec allows "misc*" to follow the document top-level element:
document ::= prolog element Misc*
and this too is incorrect in HaXml. There may well be other occurrences of the same omission.
Given that this fix was so very easy and given that the parser was already spec consistent, I now have to assume that there was good reason for the Prolog to be spec inconsistent, but I don't know what it is...
I originally assumed that Misc's were unimportant and could be discarded, like comments are discarded by a compiler. I failed to notice that PI's should be passed through to the application.
Implementation question: Why is there so much replicated code in HaXML/Html (parse.hs and pretty.hs)
The HTML parser does some correction of mal-formed input, which is not otherwise permitted by the XML spec. Likewise, the HTML pretty-printer makes some wild and unjustified assumptions about the way that humans like to format their documents, whereas the XML pp is more strictly-conforming. Once XHTML becomes common, the HTML parser/pp will be obsolete.
Regards, Malcolm
______________________________________________________________ S. Alexander Jacobson tel:917-770-6565 http://alexjacobson.com
Hmmm... it's not strictly an entry to the mainstream HaXml, but I have unit test code for my modified version of HaXml, which might be of some use: http://www.ninebynine.org/Software/HaskellUtils/HaXml-1.12/test/ http://www.ninebynine.org/Software/HaskellUtils/HaXml-1.12/test/TestXml.hs The test cases include some round-tripping, e.g. using doXmlParseFormat. #g -- At 08:48 28/10/04 -0400, S. Alexander Jacobson wrote:
Is there a good entry point into HaXml? I've now spent some time trying to understand it and feel like I've gotten nowhere.
The Haddock documentation enumerates what each function does, but I still don't know how to produce a valid XML document?
For example, this is obviously the wrong way to go:
simp2 = document $ Document (Prolog Nothing [] Nothing []) [] $ Elem "root" [("attr",AttValue [Left "v\"al"])] [CString False "<<<<<>>&&&"]
Because, it produces the obviously wrong:
<root attr="v"al"><<<<<>>&&&</root>
I assume/hope that the combinators properly encode/escape attribute values and CDATA, but can't figure out how to generate even the simple XML above.
And once I've done so, is there a way to put PIs in via the combinators or do I have to import Types and risk have unescaped stuff in my document?
-Alex-
On Thu, 28 Oct 2004, Malcolm Wallace wrote:
"S. Alexander Jacobson" <alex@alexjacobson.com> writes:
I modified the Prolog type to be data Prolog = Prolog (Maybe XMLDecl) [Misc] (Maybe DocTypeDecl) [Misc] and then modified the Prolog parser
Thanks for spotting this bug and providing a fix. I also note that the XML spec allows "misc*" to follow the document top-level element:
document ::= prolog element Misc*
and this too is incorrect in HaXml. There may well be other occurrences of the same omission.
Given that this fix was so very easy and given that the parser was already spec consistent, I now have to assume that there was good reason for the Prolog to be spec inconsistent, but I don't know what it is...
I originally assumed that Misc's were unimportant and could be discarded, like comments are discarded by a compiler. I failed to notice that PI's should be passed through to the application.
Implementation question: Why is there so much replicated code in HaXML/Html (parse.hs and pretty.hs)
The HTML parser does some correction of mal-formed input, which is not otherwise permitted by the XML spec. Likewise, the HTML pretty-printer makes some wild and unjustified assumptions about the way that humans like to format their documents, whereas the XML pp is more strictly-conforming. Once XHTML becomes common, the HTML parser/pp will be obsolete.
Regards, Malcolm
______________________________________________________________ S. Alexander Jacobson tel:917-770-6565 http://alexjacobson.com _______________________________________________ Haskell mailing list Haskell@haskell.org http://www.haskell.org/mailman/listinfo/haskell
------------ Graham Klyne For email: http://www.ninebynine.org/#Contact
"S. Alexander Jacobson" <alex@alexjacobson.com> writes:
Is there a good entry point into HaXml? I've now spent some time trying to understand it and feel like I've gotten nowhere.
It is a large package with many diverse facilities, so I'm not surprised. I take it you have read the ICFP'99 paper linked to from the HaXml webpage? To give a fuller answer, it would be helpful to know more about your specific XML needs.
The Haddock documentation enumerates what each function does, but I still don't know how to produce a valid XML document?
Where does your document come from? Has it been parsed already, then manipulated, and you want to spit it out again? Or are you trying to generate a fresh document from nothing? Or perhaps you have some existing Haskell data-structure you want to convert to XML for external representation only?
For example, this is obviously the wrong way to go:
simp2 = document $ Document (Prolog Nothing [] Nothing []) [] $ Elem "root" [("attr",AttValue [Left "v\"al"])] [CString False "<<<<<>>&&&"]
Because, it produces the obviously wrong:
<root attr="v"al"><<<<<>>&&&</root>
Ah. Escaping of special characters within text is a separate issue. It need only be done once, just before output. See Text.XML.HaXml.Escape - specifically you want something like simp2 = document $ Document (Prolog Nothing [] Nothing []) [] $ xmlEscape stdXmlEscaper $ Elem "root" [("attr",AttValue [Left "v\"al"])] [CString False "<<<<<>>&&&"]
I assume/hope that the combinators properly encode/escape attribute values and CDATA,
No, at the moment they don't. You can always do it one-shot at the end, as in the example above, although it would probably be better from a correctness point of view if the combinators did as you suggest.
And once I've done so, is there a way to put PIs in via the combinators
Currently, there are no combinators specifically for generating PIs (simply because no-one has asked for them before), but it would be extremely easy to add. For instance: mkPI :: String -> String -> CFilter mkPI pitarget str = \t-> [ CMisc (PI (pitarget,str)) ] Regards, Malcolm
On Thu, 28 Oct 2004, Malcolm Wallace wrote:
It is a large package with many diverse facilities, so I'm not surprised. I take it you have read the ICFP'99 paper linked to from the HaXml webpage? To give a fuller answer, it would be helpful to know more about your specific XML needs.
It would be helpful to have an example of using combinators to take a value of type e.g. import Network.URI import System.Time data Foo = Foo {uri::URI, name::String, stamp::ClockTime} and produce e.g. a string such as <? xml version="1.0" ?> <? xml-stylesheet type="text/xsl" href="myStyle.xsl"?> <Foo xmlns:Foo="http://mynamespace" uri="http://someURL" name="blah" stamp="Mon, 11 Oct 2004 09:05:31 -0400"/> Note, I think I've figured out how to do this using raw types (after modifying to allow PIs in the prolog!), but I assume using the combinators is more correct.
Ah. Escaping of special characters within text is a separate issue. It need only be done once, just before output. See Text.XML.HaXml.Escape - specifically you want something like
simp2 = document $ Document (Prolog Nothing [] Nothing []) [] $ xmlEscape stdXmlEscaper $ Elem "root" [("attr",AttValue [Left "v\"al"])] [CString False "<<<<<>>&&&"]
Yes, thank you. I figured this much out after posting the last mail. It would have been nice if the documents made the use of both document and xmlEscape more clear at the beginning. But, I actually think the document function should *automatically* implement xmlEscape. Otherwise, it is too easy for the user to produce incorrect documents notwithstanding what the library claims to promise!
I assume/hope that the combinators properly encode/escape attribute values and CDATA,
No, at the moment they don't. You can always do it one-shot at the end, as in the example above, although it would probably be better from a correctness point of view if the combinators did as you suggest.
Actually I think the one shot at the end is correct. Everything else is manipulation of the data model and therefore shouldn't worry about escaping. But, as I said above, the user should not, by default, have to worry about escaping.
And once I've done so, is there a way to put PIs in via the combinators
Currently, there are no combinators specifically for generating PIs (simply because no-one has asked for them before), but it would be extremely easy to add. For instance:
mkPI :: String -> String -> CFilter mkPI pitarget str = \t-> [ CMisc (PI (pitarget,str)) ]
Will this work to insert a PI before the first element? -Alex- ______________________________________________________________ S. Alexander Jacobson tel:917-770-6565 http://alexjacobson.com
Followup: You have a Dtd2Haskell tool, but DTDs are very inexpressive and the result is string based types. Is there a tool for converting e.g. relaxNG into haskell types? Then we get direct support for e.g. doubles or URLs. I know there is a relaxNG validator written in haskell, but that is not the same thing... -Alex- http://www.thaiopensource.com/relaxng/derivative.html On Thu, 28 Oct 2004, Malcolm Wallace wrote:
"S. Alexander Jacobson" <alex@alexjacobson.com> writes:
Is there a good entry point into HaXml? I've now spent some time trying to understand it and feel like I've gotten nowhere.
It is a large package with many diverse facilities, so I'm not surprised. I take it you have read the ICFP'99 paper linked to from the HaXml webpage? To give a fuller answer, it would be helpful to know more about your specific XML needs.
The Haddock documentation enumerates what each function does, but I still don't know how to produce a valid XML document?
Where does your document come from? Has it been parsed already, then manipulated, and you want to spit it out again? Or are you trying to generate a fresh document from nothing? Or perhaps you have some existing Haskell data-structure you want to convert to XML for external representation only?
For example, this is obviously the wrong way to go:
simp2 = document $ Document (Prolog Nothing [] Nothing []) [] $ Elem "root" [("attr",AttValue [Left "v\"al"])] [CString False "<<<<<>>&&&"]
Because, it produces the obviously wrong:
<root attr="v"al"><<<<<>>&&&</root>
Ah. Escaping of special characters within text is a separate issue. It need only be done once, just before output. See Text.XML.HaXml.Escape - specifically you want something like
simp2 = document $ Document (Prolog Nothing [] Nothing []) [] $ xmlEscape stdXmlEscaper $ Elem "root" [("attr",AttValue [Left "v\"al"])] [CString False "<<<<<>>&&&"]
I assume/hope that the combinators properly encode/escape attribute values and CDATA,
No, at the moment they don't. You can always do it one-shot at the end, as in the example above, although it would probably be better from a correctness point of view if the combinators did as you suggest.
And once I've done so, is there a way to put PIs in via the combinators
Currently, there are no combinators specifically for generating PIs (simply because no-one has asked for them before), but it would be extremely easy to add. For instance:
mkPI :: String -> String -> CFilter mkPI pitarget str = \t-> [ CMisc (PI (pitarget,str)) ]
Regards, Malcolm
______________________________________________________________ S. Alexander Jacobson tel:917-770-6565 http://alexjacobson.com
Looking at the before-and-after versions of HaXml code that I have modified [1] (with the primary goal of adding namespace support, but also numerous other changes), it seems that the parser always allowed PIs to appear before and after the DTD, but did not store them in the resulting data structure. (That is still the situation with my code.) Here's the original code for the prolog parser that I have: [[ prolog :: XParser Prolog prolog = do x <- maybe xmldecl many misc dtd <- maybe doctypedecl many misc return (Prolog x dtd) ]] It's clear from this that the PIs (and comments) are parsed and discarded, so your change to capture these in the prolog makes sense. #g -- [1] http://www.ninebynine.org/Software/HaskellUtils/HaXml-1.12/ (My modified version.) At 00:44 28/10/04 -0400, S. Alexander Jacobson wrote:
I modified the Prolog type to be
data Prolog = Prolog (Maybe XMLDecl) [Misc] (Maybe DocTypeDecl) [Misc]
and then modified the Prolog parser (which actually was correct) to actually use the misc values in constucting the Prolog. I replace the prolog function in pretty.hs with
prolog (Prolog x m1 dtd m2) = maybe xmldecl x $$ vcat (map misc m1) $$ maybe doctypedecl dtd $$ vcat (map misc m2)
and did a few more similar things in Haskell2XML and XML2Haskell (about a 5 min operation) and it all seems to work properly.
Implementation question: Why is there so much replicated code in HaXML/Html (parse.hs and pretty.hs)
Given that this fix was so very easy and given that the parser was already spec consistent, I now have to assume that there was good reason for the Prolog to be spec inconsistent, but I don't know what it is...
Thoughts?
-Alex-
On Wed, 27 Oct 2004, S. Alexander Jacobson wrote:
The XML spec defines the prolog(1) as follows:
[22] prolog ::= XMLDecl? Misc* (doctypedecl Misc*)?
In other words you can have Misc before AND after the doctype declaration!
HaXml defines the prolog(2) as:
data Prolog = Prolog (Maybe XMLDecl) (Maybe DocTypeDecl)
HaXml therefore does not allow PIs before the beginning of the top level element. This is a problem in practice for people who want to use e.g. XML-Stylesheets(3) where if one is using XSL to produce HTML it is necessary to put the stylesheet declaration BEFORE the root element (if the root element is not HTML). e.g.
<?xml version="1.0"?> <?xml:stylesheet type="text/xsl" href="style.xsl"?> <foo id="57" category="2"/>
Is there some way to puta PI before the root element in combinators?
-Alex-
(1) http://www.w3.org/TR/REC-xml/#sec-prolog-dtd (2) http://www.cs.york.ac.uk/fp/HaXml/HaXml/Text.XML.HaXml.Types.html#Prolog (3) http://www.w3.org/TR/xml-stylesheet/ ______________________________________________________________ S. Alexander Jacobson tel:917-770-6565 http://alexjacobson.com _______________________________________________ Haskell mailing list Haskell@haskell.org http://www.haskell.org/mailman/listinfo/haskell
______________________________________________________________ S. Alexander Jacobson tel:917-770-6565 http://alexjacobson.com _______________________________________________ Haskell mailing list Haskell@haskell.org http://www.haskell.org/mailman/listinfo/haskell
------------ Graham Klyne For email: http://www.ninebynine.org/#Contact
participants (3)
-
Graham Klyne -
Malcolm Wallace -
S. Alexander Jacobson