Will generated stack traces be different that --- Ömer Sinan Ağacan http://osa1.net 2014-08-13 19:56 GMT+03:00 Johan Tibell :

Yes, it doesn't use any code modification so it doesn't have runtime overhead (except when generating the actual trace) or interfere with compiler optimizations. In other words you can actually have it enabled at all time. It only requires that you compile with -g, just like with a C compiler.

On Wed, Aug 13, 2014 at 6:45 PM, Ömer Sinan Ağacan wrote:

...

Is this stack trace support different than what we have currently? (e.g. the one implemented with GHC.Stack and cost centers)

--- Ömer Sinan Ağacan http://osa1.net

2014-08-13 18:02 GMT+03:00 Johan Tibell :

...

Hi,

How's the integration of DWARF support coming along? It's probably one of the most important improvements to the runtime in quite some time since unlocks *two* important features, namely

* trustworthy profiling (using e.g. Linux perf events and other low-overhead, code preserving, sampling profilers), and * stack traces.

The former is really important to move our core libraries performance up a notch. Right now -prof is too invasive for it to be useful when evaluating the hotspots in these libraries (which are already often heavily tuned).

The latter one is really important for real life Haskell on the server, where you can sometimes can get some crash that only happens once a day under very specific conditions. Knowing where the crash happens is then *very* useful.

-- Johan

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

Without any overhead we'll get the runtime stack trace, which isn't exactly the same as what we can get with emulation, but has the benefit that we can leave it on in all of our shipped code if we like. This latter is a really crucial property for stack traces to be widely useful. On Wed, Aug 13, 2014 at 7:13 PM, Ömer Sinan Ağacan wrote:

Sorry for my previous email. (used a gmail shortcut by mistake)

We won't have stacks as we have in imperative(without TCO) and strict languages. So we still need some kind of emulation and I think this means some extra run-time operations. I'm wondering about two things:

1) Do we still get same traces as we get using GHC.Stack right now? 2) If yes, then how can we have that without any runtime costs?

Thanks and sorry again for my previous email.

--- Ömer Sinan Ağacan http://osa1.net

...

Will generated stack traces be different that

--- Ömer Sinan Ağacan http://osa1.net

2014-08-13 19:56 GMT+03:00 Johan Tibell :

...

Yes, it doesn't use any code modification so it doesn't have runtime overhead (except when generating the actual trace) or interfere with compiler optimizations. In other words you can actually have it enabled at all time. It only requires that you compile with -g, just like with a C compiler.

On Wed, Aug 13, 2014 at 6:45 PM, Ömer Sinan Ağacan < omeragacan@gmail.com> wrote:

...

Is this stack trace support different than what we have currently? (e.g. the one implemented with GHC.Stack and cost centers)

--- Ömer Sinan Ağacan http://osa1.net

2014-08-13 18:02 GMT+03:00 Johan Tibell :

...

Hi,

How's the integration of DWARF support coming along? It's probably

one

...

...

of the most important improvements to the runtime in quite some time since unlocks *two* important features, namely

* trustworthy profiling (using e.g. Linux perf events and other low-overhead, code preserving, sampling profilers), and * stack traces.

The former is really important to move our core libraries

...

...

...

...

a notch. Right now -prof is too invasive for it to be useful when evaluating the hotspots in these libraries (which are already often heavily tuned).

The latter one is really important for real life Haskell on the server, where you can sometimes can get some crash that only happens once a day under very specific conditions. Knowing where the crash happens is

2014-08-13 20:08 GMT+03:00 Ömer Sinan Ağacan : performance up then

...

...

...

...

*very* useful.

-- Johan

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

What's the minimal amount of work we need to do to just get the dwarf data in the codegen by 7.10 (RC late december) so we can start using e.g. linux perf events to profile Haskell programs? On Wed, Aug 13, 2014 at 7:31 PM, Arash Rouhani wrote:

Hi Johan!

I haven't done much (just been lazy) lately, I've tried to benchmark my results but I don't get any sensible results at all yet.

Last time Peter said he's working on a more portable way to read dwarf information that doesn't require Linux. But I'm sure he'll give a more acurate update than me soon in this mail thread.

As for stack traces, I don't think there's any big tasks left, but I summarize what I have in mind:

- The haskell interface is done and I've iterated on it a bit, so it's in a decent shape at least. Some parts still need testing. - I wish I could implement the `forceCaseContinuation` that I've described in my thesis. If someone is good with code generation (I just suck at it, it's probably simple) and is willing to assist me a bit, please say so. :) - I tried benchmarking, I gave up after not getting any useful results. - I'm unfortunately totally incapable to help out with dwarf debug data generation, only Peter knows that part, particularly I never grasped his theoretical framework of causality in Haskell. - Peter and I have finally agreed on a simple and sensible way to implement `catchWithStack` that have all most good properties you would like. I just need to implement it and test it. I can definitely man up and implement this. :)

Here's my master thesis btw [1], it should answer Ömer's question of how we retrieve a stack from a language you think won't have a stack. :)

Cheers, Arash

[1]: http://arashrouhani.com/papers/master-thesis.pdf

On 2014-08-13 17:02, Johan Tibell wrote:

Hi,

How's the integration of DWARF support coming along? It's probably one of the most important improvements to the runtime in quite some time since unlocks *two* important features, namely

* trustworthy profiling (using e.g. Linux perf events and other low-overhead, code preserving, sampling profilers), and * stack traces.

The former is really important to move our core libraries performance up a notch. Right now -prof is too invasive for it to be useful when evaluating the hotspots in these libraries (which are already often heavily tuned).

The latter one is really important for real life Haskell on the server, where you can sometimes can get some crash that only happens once a day under very specific conditions. Knowing where the crash happens is then *very* useful.

-- Johan

_______________________________________________ ghc-devs mailing listghc-devs@haskell.orghttp://www.haskell.org/mailman/listinfo/ghc-devs

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

At this point I have a bit more time on my hands again (modulo post-thesis vacations), but we are basically still in “review hell”. I think “just” for perf_events support we’d need the following patches[1]: 1. Source notes (Core support) 2. Source notes (CorePrep & Stg support) 3. Source notes (Cmm support) 4. Tick scopes 5. Debug data extraction (NCG support) 6. Generate .loc/.file directives We have a basic “okay” from the Simons up to number 2 (conditional on better documentation). Number 4 sticks out because Simon Marlow wanted to have a closer look at it - this is basically about how to maintain source ticks in a robust fashion on the Cmm level (see also section 5.5 of my thesis[2]). Meanwhile I have ported NCG DWARF generation over to Mac Os, and am working on reviving LLVM support. My plan was to check that I didn’t accidentally break Linux support, then push for review again in a week or so (Phab?). Greetings, Peter [1] https://github.com/scpmw/ghc/commits/profiling-import [2] http://www.personal.leeds.ac.uk/~scpmw/static/thesis.pdf On 13 Aug 2014, at 20:01, Johan Tibell mailto:johan.tibell@gmail.com> wrote: What's the minimal amount of work we need to do to just get the dwarf data in the codegen by 7.10 (RC late december) so we can start using e.g. linux perf events to profile Haskell programs? On Wed, Aug 13, 2014 at 7:31 PM, Arash Rouhani mailto:rarash@student.chalmers.se> wrote: Hi Johan! I haven't done much (just been lazy) lately, I've tried to benchmark my results but I don't get any sensible results at all yet. Last time Peter said he's working on a more portable way to read dwarf information that doesn't require Linux. But I'm sure he'll give a more acurate update than me soon in this mail thread. As for stack traces, I don't think there's any big tasks left, but I summarize what I have in mind: * The haskell interface is done and I've iterated on it a bit, so it's in a decent shape at least. Some parts still need testing. * I wish I could implement the `forceCaseContinuation` that I've described in my thesis. If someone is good with code generation (I just suck at it, it's probably simple) and is willing to assist me a bit, please say so. :) * I tried benchmarking, I gave up after not getting any useful results. * I'm unfortunately totally incapable to help out with dwarf debug data generation, only Peter knows that part, particularly I never grasped his theoretical framework of causality in Haskell. * Peter and I have finally agreed on a simple and sensible way to implement `catchWithStack` that have all most good properties you would like. I just need to implement it and test it. I can definitely man up and implement this. :) Here's my master thesis btw [1], it should answer Ömer's question of how we retrieve a stack from a language you think won't have a stack. :) Cheers, Arash [1]: http://arashrouhani.com/papers/master-thesis.pdf On 2014-08-13 17:02, Johan Tibell wrote: Hi, How's the integration of DWARF support coming along? It's probably one of the most important improvements to the runtime in quite some time since unlocks *two* important features, namely * trustworthy profiling (using e.g. Linux perf events and other low-overhead, code preserving, sampling profilers), and * stack traces. The former is really important to move our core libraries performance up a notch. Right now -prof is too invasive for it to be useful when evaluating the hotspots in these libraries (which are already often heavily tuned). The latter one is really important for real life Haskell on the server, where you can sometimes can get some crash that only happens once a day under very specific conditions. Knowing where the crash happens is then *very* useful. -- Johan _______________________________________________ ghc-devs mailing list ghc-devs@haskell.orgmailto:ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs _______________________________________________ ghc-devs mailing list ghc-devs@haskell.orgmailto:ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

Johan Tibell wrote: Do you mind expanding on what tick scopes are. It sounds scarily like something that happens at runtime. :) It’s a pretty basic problem - for Core we can always walk the tree upwards to find some source ticks that might be useful. Cmm on the other hand is flat: Given one block without any annotations on its own, there is no robust way we could "look around" for debugging information. This is especially tricky because Cmm stages want to be able to liberally add or remove blocks. So let’s say we have an extra GC block added: Which source location should we see as associated with it? And if two blocks are combined using common block elimination: What is now the best source location? And how do we express all this in a way that won’t make code generation more complicated? The latter is an important consideration, because code generation is very irregular in how it treats code - often alternating between accumulating it in a monad and passing it around by hand. I have found it quite tricky to find a good solution in this design space - the current idea is that we associate every piece of generated Cmm with a “tick scope”, which decides how far a tick will “apply”. So for example a GC block would be generated using the same tick scope as the function’s entry block, and therefore will get all ticks associated with the function’s top level, which is probably the best choice. On the other hand, for merging blocks we can “combine” the scopes in a way that guarantees that we find (at least) the same ticks as before, therefore losing no information. And yes, this design could be simplified somewhat for pure DWARF generation. After all, for that particular purpose every tick scope will just boil down to a single source location anyway. So we could simply replace scopes with the source link right away. But I think it would come down to about the same code complexity, plus having a robust structure around makes it easier to carry along extra information such as unwind information, extra source ticks or the generating Core. Greetings, Peter On Wed, Aug 13, 2014 at 8:49 PM, Peter Wortmann mailto:scpmw@leeds.ac.uk> wrote: At this point I have a bit more time on my hands again (modulo post-thesis vacations), but we are basically still in “review hell”. I think “just” for perf_events support we’d need the following patches[1]: 1. Source notes (Core support) 2. Source notes (CorePrep & Stg support) 3. Source notes (Cmm support) 4. Tick scopes 5. Debug data extraction (NCG support) 6. Generate .loc/.file directives We have a basic “okay” from the Simons up to number 2 (conditional on better documentation). Number 4 sticks out because Simon Marlow wanted to have a closer look at it - this is basically about how to maintain source ticks in a robust fashion on the Cmm level (see also section 5.5 of my thesis[2]). Meanwhile I have ported NCG DWARF generation over to Mac Os, and am working on reviving LLVM support. My plan was to check that I didn’t accidentally break Linux support, then push for review again in a week or so (Phab?). Greetings, Peter [1] https://github.com/scpmw/ghc/commits/profiling-import [2] http://www.personal.leeds.ac.uk/~scpmw/static/thesis.pdf On 13 Aug 2014, at 20:01, Johan Tibell mailto:johan.tibell@gmail.commailto:johan.tibell@gmail.com>> wrote: What's the minimal amount of work we need to do to just get the dwarf data in the codegen by 7.10 (RC late december) so we can start using e.g. linux perf events to profile Haskell programs? On Wed, Aug 13, 2014 at 7:31 PM, Arash Rouhani mailto:rarash@student.chalmers.semailto:rarash@student.chalmers.se>> wrote: Hi Johan! I haven't done much (just been lazy) lately, I've tried to benchmark my results but I don't get any sensible results at all yet. Last time Peter said he's working on a more portable way to read dwarf information that doesn't require Linux. But I'm sure he'll give a more acurate update than me soon in this mail thread. As for stack traces, I don't think there's any big tasks left, but I summarize what I have in mind: * The haskell interface is done and I've iterated on it a bit, so it's in a decent shape at least. Some parts still need testing. * I wish I could implement the `forceCaseContinuation` that I've described in my thesis. If someone is good with code generation (I just suck at it, it's probably simple) and is willing to assist me a bit, please say so. :) * I tried benchmarking, I gave up after not getting any useful results. * I'm unfortunately totally incapable to help out with dwarf debug data generation, only Peter knows that part, particularly I never grasped his theoretical framework of causality in Haskell. * Peter and I have finally agreed on a simple and sensible way to implement `catchWithStack` that have all most good properties you would like. I just need to implement it and test it. I can definitely man up and implement this. :) Here's my master thesis btw [1], it should answer Ömer's question of how we retrieve a stack from a language you think won't have a stack. :) Cheers, Arash [1]: http://arashrouhani.com/papers/master-thesis.pdf On 2014-08-13 17:02, Johan Tibell wrote: Hi, How's the integration of DWARF support coming along? It's probably one of the most important improvements to the runtime in quite some time since unlocks *two* important features, namely * trustworthy profiling (using e.g. Linux perf events and other low-overhead, code preserving, sampling profilers), and * stack traces. The former is really important to move our core libraries performance up a notch. Right now -prof is too invasive for it to be useful when evaluating the hotspots in these libraries (which are already often heavily tuned). The latter one is really important for real life Haskell on the server, where you can sometimes can get some crash that only happens once a day under very specific conditions. Knowing where the crash happens is then *very* useful. -- Johan _______________________________________________ ghc-devs mailing list ghc-devs@haskell.orgmailto:ghc-devs@haskell.orgmailto:ghc-devs@haskell.org> http://www.haskell.org/mailman/listinfo/ghc-devs _______________________________________________ ghc-devs mailing list ghc-devs@haskell.orgmailto:ghc-devs@haskell.orgmailto:ghc-devs@haskell.org> http://www.haskell.org/mailman/listinfo/ghc-devs

Peter Wortmann writes:

Okay, I have uploaded the “core” set of patches to Phab:

https://phabricator.haskell.org/D155

Surely you mean D169 [1]? Cheers, - Ben [1] https://phabricator.haskell.org/D169