How to get a heap visualization

I need a simple heap visualization for debugging purposes. I'm using GHC 8.0.2 to compile a large and complex yesod-based web app. What's the quickest and easiest way? Vacuum looks simple and nice. But it has some long-outstanding PRs against it to support GHC 7.10 and GHC 8.0 that were never applied. https://github.com/thoughtpolice/vacuum/issues/9 Getting ghc-vis to compile looks hopeless, for a number of reasons. The dependencies on gtk and cairo are huge. It hasn't been updated on Hackage for a year and a half. It requires base < 4.9. I need to run the visualizer either on a headless Ubuntu 16.04 server, or locally on Windows. And anyway, the fancy GUI in ghc-vis is way overkill for me. The heap scraper backend for ghc-vis, ghc-heap-view, looks usable, and better supported than vacuum. But is there a quick and simple visualizer for its output, without ghc-vis? Is there anything else? Is the best option to fork vacuum and and try to apply the PRs?

Why not the plain old heap profiler? Edward Excerpts from Yitzchak Gale's message of 2017-08-30 18:34:05 +0300:
I need a simple heap visualization for debugging purposes. I'm using GHC 8.0.2 to compile a large and complex yesod-based web app. What's the quickest and easiest way?
Vacuum looks simple and nice. But it has some long-outstanding PRs against it to support GHC 7.10 and GHC 8.0 that were never applied.
https://github.com/thoughtpolice/vacuum/issues/9
Getting ghc-vis to compile looks hopeless, for a number of reasons. The dependencies on gtk and cairo are huge. It hasn't been updated on Hackage for a year and a half. It requires base < 4.9. I need to run the visualizer either on a headless Ubuntu 16.04 server, or locally on Windows. And anyway, the fancy GUI in ghc-vis is way overkill for me.
The heap scraper backend for ghc-vis, ghc-heap-view, looks usable, and better supported than vacuum. But is there a quick and simple visualizer for its output, without ghc-vis?
Is there anything else? Is the best option to fork vacuum and and try to apply the PRs?

I wrote:
I need a simple heap visualization for debugging purposes... Vacuum... has some long-outstanding PRs against it... that were never applied... Getting ghc-vis to compile looks hopeless... ghc-heap-view... is there a quick and simple visualizer for its output, without ghc-vis?
Edward Z. Yang wrote:
Why not the plain old heap profiler?
That didn't prove helpful in this case. We need to dive down into the structure of certain large and complex objects to find out what is happening. My plan is to see if I can apply the vacuum PR manually locally and see if I can get it working reasonably soon. If not, I guess I'll try running ghc-heap-view without visualization and see if I can make sense of the textual output. Actually, the profiling was done by someone else, not me. Now that you mention it, maybe I'll first give that another try myself and see if I can get any farther. Still, vacuum-style heap visualization is a really nice tool. It's a shame that it has fallen into such a state of disrepair. Thanks, Yitz

Have you gotten in touch with Joachim? I think he's touched that space in
the not too murky past.
On Aug 31, 2017 11:18 AM, "Yitzchak Gale"
I wrote:
I need a simple heap visualization for debugging purposes... Vacuum... has some long-outstanding PRs against it... that were never applied... Getting ghc-vis to compile looks hopeless... ghc-heap-view... is there a quick and simple visualizer for its output, without ghc-vis?
Edward Z. Yang wrote:
Why not the plain old heap profiler?
That didn't prove helpful in this case. We need to dive down into the structure of certain large and complex objects to find out what is happening.
My plan is to see if I can apply the vacuum PR manually locally and see if I can get it working reasonably soon. If not, I guess I'll try running ghc-heap-view without visualization and see if I can make sense of the textual output.
Actually, the profiling was done by someone else, not me. Now that you mention it, maybe I'll first give that another try myself and see if I can get any farther.
Still, vacuum-style heap visualization is a really nice tool. It's a shame that it has fallen into such a state of disrepair.
Thanks, Yitz _______________________________________________ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users

Hi, author of ghc-heap-view here. Am Mittwoch, den 30.08.2017, 18:34 +0300 schrieb Yitzchak Gale:
Getting ghc-vis to compile looks hopeless, for a number of reasons. The dependencies on gtk and cairo are huge.
Is that really a problem?
It hasn't been updated on Hackage for a year and a half. It requires base < 4.9.
GitHub is already ahead. I guess this just needs to be uploaded? https://github.com/def-/ghc-vis/blob/master/ghc-vis.cabal
I need to run the visualizer either on a headless Ubuntu 16.04 server, or locally on Windows.
Ok, that is more tricky.
The heap scraper backend for ghc-vis, ghc-heap-view, looks usable, and better supported than vacuum. But is there a quick and simple visualizer for its output, without ghc-vis?
Well, the :printHeap command that comes with it does “visualize” things as something resembling Haskell syntax: let x1 = "A Value" x16 = True : False : x16 in (x1,x1,x16) I don’t know of anything more graphical besides ghc-vis, but you could roll your own, if you want to; you can use ghc-heap-view to get a graph using http://hackage.haskell.org/package/ghc-heap-view-0.5.9/docs/GHC-HeapView.htm... and then visualize that as you like. Greetings, Joachim -- Joachim “nomeata” Breitner mail@joachim-breitner.de https://www.joachim-breitner.de/

Joachim, first and foremost, thanks for the awesome libraries ghc-vis and ghc-heap-view. The design trade-offs for ghc-vis do make sense if you think of it as a didactic tool. But as a debugging tool, the most important factor is that it should "Just Work", with no big builds, no fiddling, no googling. When you reach for a debugging tool, you are *already* in a debugging situation; you don't have patience for also debugging the tool. I wrote:
Getting ghc-vis to compile looks hopeless, for a number of reasons..The dependencies on gtk and cairo are huge.
You wrote:
Is that really a problem?
Admittedly I haven't tried in quite a while. In the past, getting all the Haskell libraries and C libraries to match up and work together on any given platform at matching versions was a huge task. If nowadays you can just type "stack build" or "cabal install" (with new-build or in a sandbox) on even one platform and have it just work, reliably every time, month after month, that would be an amazing feat. But in my case, I need even more than that. I need it to "Just Work" on certain specific platforms, none of which have ever been known as the best for running GTK: Windows, and headless Ubuntu.
The heap scraper backend for ghc-vis, ghc-heap-view, looks usable, and better supported than vacuum. But is there a quick and simple visualizer for its output, without ghc-vis?
Well, the :printHeap command that comes with it does “visualize” things as something resembling Haskell syntax... I don’t know of anything more graphical besides ghc-vis, but you could roll your own, if you want to; you can use ghc-heap-view to get a graph using buildHeapGraph and then visualize that as you like.
Yes, this is the approach I will probably take. I'll start with just printing the output into a text file. Maybe I can massage that and find what I need. Thanks, Yitz

Yitzchak Gale
Joachim, first and foremost, thanks for the awesome libraries ghc-vis and ghc-heap-view.
The design trade-offs for ghc-vis do make sense if you think of it as a didactic tool. But as a debugging tool, the most important factor is that it should "Just Work", with no big builds, no fiddling, no googling. When you reach for a debugging tool, you are *already* in a debugging situation; you don't have patience for also debugging the tool.
I should point out that there is a differential (D3055) by erikd from earlier this year which folded ghc-heap-view into GHC. The goal was to make heap-view a supported library which could be relied upon by third party tools. As far as I can remember most of the hard work is done; it just needed some finishing touches. Perhaps you would be interested in picking it up? Cheers, - Ben

Ben Gamari wrote:
Yitzchak Gale
writes: Joachim, first and foremost, thanks for the awesome libraries ghc-vis and ghc-heap-view.
The design trade-offs for ghc-vis do make sense if you think of it as a didactic tool. But as a debugging tool, the most important factor is that it should "Just Work", with no big builds, no fiddling, no googling. When you reach for a debugging tool, you are *already* in a debugging situation; you don't have patience for also debugging the tool.
I should point out that there is a differential (D3055) by erikd from earlier this year which folded ghc-heap-view into GHC. The goal was to make heap-view a supported library which could be relied upon by third party tools. As far as I can remember most of the hard work is done; it just needed some finishing touches. Perhaps you would be interested in picking it up?
Sorry if I gave the wrong impression, but IMO ghc-heap-view is a *long* way from being ready to integrate into GHC. ghc-heap-view currently supports most of the normal/simple heap object types. but has no support for more complex objects. Furthermore, levity polymorphism adds a whole new dimension to the problem. My estimate of the amount of work required to make ghc-heap-view cover all the heap objects was such that I abandoned the project. I would love to see someone pick it up and run with it. Erik -- ---------------------------------------------------------------------- Erik de Castro Lopo http://www.mega-nerd.com/

Erik de Castro Lopo
Ben Gamari wrote:
I should point out that there is a differential (D3055) by erikd from earlier this year which folded ghc-heap-view into GHC. The goal was to make heap-view a supported library which could be relied upon by third party tools. As far as I can remember most of the hard work is done; it just needed some finishing touches. Perhaps you would be interested in picking it up?
Sorry if I gave the wrong impression, but IMO ghc-heap-view is a *long* way from being ready to integrate into GHC. ghc-heap-view currently supports most of the normal/simple heap object types. but has no support for more complex objects. Furthermore, levity polymorphism adds a whole new dimension to the problem. My estimate of the amount of work required to make ghc-heap-view cover all the heap objects was such that I abandoned the project. I would love to see someone pick it up and run with it.
I'm not sure it's actually so far off. As far as I recall the big issues were, * teaching it to understand profiled and debugged info tables (annoying, but not difficult) * teaching it to understand unlifted fields As far as levity polymorphism is concerned, this shouldn't affect heap view at all as its simply a Core-level change; the STG machine and its associated heap layout doesn't change at all. It's easy for me to say having never worked on the patch, but I'd be surprised if it were more than a weekend away from completion. I should also mention that if you just need to poke around at the heap inside of gdb, you might consider using my GDB Python extension [1]. It's not terribly well documented, but I think it's pretty self-explanatory and I'm happy to add some documentation is there is demand. Cheers, - Ben [1] https://github.com/bgamari/ghc-utils/blob/master/ghc-gdb.py
participants (6)
-
Ben Gamari
-
David Feuer
-
Edward Z. Yang
-
Erik de Castro Lopo
-
Joachim Breitner
-
Yitzchak Gale