Hmm, I just tried with a 112MB file and I ended up having to kill it after it chewed through 3GB of memory. That's rather worrying. Hopefully someone on cafe can point out whether it's an inherent issue with the package, a bug or whether we're just doing something wrong. On 09/02/13 15:30, grant wrote:

Mateusz Kowalczyk writes:

...

I don't know about xml-conduit but I know that such thing is possible in HXT. See the `Modifying a Node' section at [1] for a trivial example. You probably will have to read the whole page to somewhat understand what's going on though.

[1] - http://adit.io/posts/2012-04-14-working_with_HTML_in_haskell.html

Thanks for the tip, which looks really promising. The problem is that I cannot get large xml files to load with HXT. I tried a 24mb file and it ran out of memory, whereas with xml-conduit it took 15 seconds to load. Is there something I'm missing?

Grant

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Michael Snoyman writes:

Hi Grant, As you might expect from immutable data structures, there's no way to update in place. The approach you'd take to XSLT: traverse the tree, check each node, and output a new structure. I put together the following as an example, but I could ...

Thanks a lot, I really appreciate it, Grant

On Sun, Feb 10, 2013 at 8:51 PM, grant wrote:

Michael Snoyman writes:

...

Hi Michael,

Just one last thought. Does it make any sense that xml-conduit could be rewritten as a lens instead of a cursor? Or leverage the lens package somehow?

That's a really interesting idea, I'd never thought about it before. It's definitely something worth playing around with. However, I think in this case the Cursor is providing a totally different piece of functionality than what lenses would do. The Cursor is really working as a Zipper, allowing you to walk the node tree and do queries about preceding and following siblings and ancestors. Now given that every time I'm on #haskell someone mentions zippers in the context of lens, maybe lens *would* solve this use case as well, but I'm still a lens novice (if that), so I can't really speak on the matter. Maybe someone with more lens experience could provide some insight. Either way, some kind of lens add-on sounds really useful. Michael

Err: That first link into Zipper.hs should instead be: https://github.com/ekmett/lens/blob/f8dfe3fd444648f61b8594cd672c25e70c8a30ff... On Sun, Feb 10, 2013 at 9:40 PM, Michael Sloan wrote:

I'm no lens authority by any means, but indeed, it looks like something like Cursor / Axis could be done with the lens zipper.

https://github.com/snoyberg/xml/blob/0367af336e86d723bd9c9fbb49db0f86d1f989e...

This cursor datatype is very much like the (:>) zipper type (I'm linking to old code, because that's when I understood it - the newer stuff is semantically the same, but more efficient, more confusing, and less directly relatable):

https://github.com/ekmett/lens/blob/f8dfe3fd444648f61b8594cd672c25e70c8a30ff...

Which is built out of the following two datatypes:

1) parent (and the way to rebuild the tree on the way back up) is provided by this datatype:

https://github.com/ekmett/lens/blob/f8dfe3fd444648f61b8594cd672c25e70c8a30ff...

2) precedingSibling / followingSibling / node is provided by this datatype (which is pretty much the familiar list zipper!):

https://github.com/ekmett/lens/blob/f8dfe3fd444648f61b8594cd672c25e70c8a30ff...

One way that this would be powerful is that some of the Axis constructors could return a zipper. In particular, all of the axis yielding functions except the following would be supported:

parent, precedingSibling, followingSibling, ancestor, descendent, orSelf, check

This is because zippers can be used for modification, which doesn't work out very well when you can navigate to something outside of your focii's children. If we have a new datatype, that represents a node's payload, then we could conceivably represent all of the axis yielding operations except for parent / ancestor. However, those operations would be navigations to payloads - further xml-hierarchy level navigation would be impossible because you'd no longer have references to children. (further navigation into payloads on the other hand, would still be possible)

So, that's just my thoughts after looking at it a bit - I hope it's comprehensible / helpful! An XML zipper would be pretty awesome.

-Michael

On Sun, Feb 10, 2013 at 8:34 PM, Michael Snoyman wrote:

...

On Sun, Feb 10, 2013 at 8:51 PM, grant wrote:

...

Michael Snoyman writes:

...

Hi Michael,

Just one last thought. Does it make any sense that xml-conduit could be rewritten as a lens instead of a cursor? Or leverage the lens package somehow?

That's a really interesting idea, I'd never thought about it before. It's definitely something worth playing around with. However, I think in this case the Cursor is providing a totally different piece of functionality than what lenses would do. The Cursor is really working as a Zipper, allowing you to walk the node tree and do queries about preceding and following siblings and ancestors.

Now given that every time I'm on #haskell someone mentions zippers in the context of lens, maybe lens *would* solve this use case as well, but I'm still a lens novice (if that), so I can't really speak on the matter. Maybe someone with more lens experience could provide some insight.

Either way, some kind of lens add-on sounds really useful.

Michael

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

OK, after some experimentation, I've decided that this would be something really cool, but I don't have the experience with lens to do it myself. Here's what I came up with so far: https://gist.github.com/snoyberg/4755679 (Also available on School of Haskell[1], but our version of lens is too old for this snippet.) So if someone wants to pursue this, I'd be really interested to see the results. Michael [1] https://haskell.fpcomplete.com/user/snoyberg/random-code-snippets/xml-condui... On Mon, Feb 11, 2013 at 8:09 AM, Michael Sloan wrote:

I realized that the term "payload" wouldn't make much sense in the context of XML. What I meant was "elementName" with "elementAttributes" (but not "elementNodes" - that's the point). So, such navigations could yield a datatype containing those.

-Michael

On Sun, Feb 10, 2013 at 9:41 PM, Michael Sloan wrote:

...

Err: That first link into Zipper.hs should instead be:

https://github.com/ekmett/lens/blob/f8dfe3fd444648f61b8594cd672c25e70c8a30ff...

On Sun, Feb 10, 2013 at 9:40 PM, Michael Sloan wrote:

...

I'm no lens authority by any means, but indeed, it looks like something like Cursor / Axis could be done with the lens zipper.

https://github.com/snoyberg/xml/blob/0367af336e86d723bd9c9fbb49db0f86d1f989e...

This cursor datatype is very much like the (:>) zipper type (I'm linking to old code, because that's when I understood it - the newer stuff is semantically the same, but more efficient, more confusing, and less directly relatable):

https://github.com/ekmett/lens/blob/f8dfe3fd444648f61b8594cd672c25e70c8a30ff...

Which is built out of the following two datatypes:

1) parent (and the way to rebuild the tree on the way back up) is provided by this datatype:

https://github.com/ekmett/lens/blob/f8dfe3fd444648f61b8594cd672c25e70c8a30ff...

2) precedingSibling / followingSibling / node is provided by this datatype (which is pretty much the familiar list zipper!):

https://github.com/ekmett/lens/blob/f8dfe3fd444648f61b8594cd672c25e70c8a30ff...

One way that this would be powerful is that some of the Axis constructors could return a zipper. In particular, all of the axis yielding functions except the following would be supported:

parent, precedingSibling, followingSibling, ancestor, descendent, orSelf, check

This is because zippers can be used for modification, which doesn't work out very well when you can navigate to something outside of your focii's children. If we have a new datatype, that represents a node's payload, then we could conceivably represent all of the axis yielding operations except for parent / ancestor. However, those operations would be navigations to payloads - further xml-hierarchy level navigation would be impossible because you'd no longer have references to children. (further navigation into payloads on the other hand, would still be possible)

So, that's just my thoughts after looking at it a bit - I hope it's comprehensible / helpful! An XML zipper would be pretty awesome.

-Michael

On Sun, Feb 10, 2013 at 8:34 PM, Michael Snoyman wrote:

...

On Sun, Feb 10, 2013 at 8:51 PM, grant wrote:

...

Michael Snoyman writes:

...

Hi Michael,

Just one last thought. Does it make any sense that xml-conduit could be rewritten as a lens instead of a cursor? Or leverage the lens package somehow?

That's a really interesting idea, I'd never thought about it before. It's definitely something worth playing around with. However, I think in this case the Cursor is providing a totally different piece of functionality than what lenses would do. The Cursor is really working as a Zipper, allowing you to walk the node tree and do queries about preceding and following siblings and ancestors.

Now given that every time I'm on #haskell someone mentions zippers in the context of lens, maybe lens *would* solve this use case as well, but I'm still a lens novice (if that), so I can't really speak on the matter. Maybe someone with more lens experience could provide some insight.

Either way, some kind of lens add-on sounds really useful.

Michael

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe