#11645: Heap profiling - hp2ps: samples out of sequence -------------------------------------+------------------------------------- Reporter: thomie | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Profiling | Version: 8.0.1-rc2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: | libraries/hpc/tests/fork/hpc_fork Blocked By: | Blocking: Related Tickets: #664 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by Thomas Miedema ): In [changeset:"90fa8cf2bf1677545c3f4a8bc967b1674822e90a/ghc" 90fa8cf2/ghc]: {{{ #!CommitTicketReference repository="ghc" revision="90fa8cf2bf1677545c3f4a8bc967b1674822e90a" Mark tests for #11643, #11644, #11645 and #9406 expect_broken For opt_ways or prof_ways only. indexed-types/should_compile/all.T called setTestOpts to not run the tests with opt_ways. Since I'm finding regressions for opt_ways, I removed it. This only makes a difference when running `./validate --slow` or `make slowtest`. Update submodule hpc. }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11645#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#11645: Heap profiling - hp2ps: samples out of sequence -------------------------------------+------------------------------------- Reporter: thomie | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Profiling | Version: 8.0.1-rc2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: | libraries/hpc/tests/fork/hpc_fork Blocked By: | Blocking: Related Tickets: #664 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by jme): The problem is that the profiler doesn't work with `forkProcess` (''e.g.'', see #8862). In this particular case, the issue is that the elapsed ''per-process'' time is being reported for each sample. Since the parent spends more time performing the forks than each child does executing `threadDelay`, its elapsed time is the largest. But the parent outputs its sample before either child, and so the samples appear out of order. The reason this behavior does not cause a problem when using 7.10.3 is that the sample times used to only be reported in hundredths of a second (and so were all 0.00). As of 1da3bbd2bd82ea11f8a1d760385df84708bbea63, they are being reported with full precision. One possible way to work around this is to simply increase the `threadDelay` intervals for each child (400000 for the first and 800000 for the second worked for me). -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11645#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#11645: Heap profiling - hp2ps: samples out of sequence -------------------------------------+------------------------------------- Reporter: thomie | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Profiling | Version: 8.0.1-rc2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: | libraries/hpc/tests/fork/hpc_fork Blocked By: | Blocking: Related Tickets: #664 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): David also reported seeing this while profiling GHC (#14006). However, I think we should keep this ticket to describe the incompatibility with `forkProcess`. Perhaps comment on #14006 instead. It might be helpful if you could include the `hp` file as well. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11645#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#11645: Heap profiling - hp2ps: samples out of sequence -------------------------------------+------------------------------------- Reporter: thomie | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Profiling | Version: 8.0.1-rc2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: | libraries/hpc/tests/fork/hpc_fork Blocked By: | Blocking: Related Tickets: #664 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by Fuuzetsu): I hit this today on 8.2.1. This is on a proprietary project so I am unable to provide any substantial code to aid debugging this. Below is a highly inefficient program which re-sequences .hp so that you can at least use the data. {{{#!hs {-# LANGUAGE LambdaCase #-} {-# LANGUAGE OverloadedStrings #-} module Main (main) where import Control.Monad (guard) import Data.Attoparsec.Combinator as P import Data.Attoparsec.Text as P import Data.List (sort) import Data.Monoid ((<>)) import Data.Text as T import qualified Data.Text.IO as T import System.Environment (getArgs) import System.Exit (exitFailure) import System.IO (hPutStrLn, stderr) data Sample = Sample !Double ![Text] deriving (Eq) instance Ord Sample where compare (Sample x _) (Sample y _) = compare x y data F = F ![Text] ![Sample] parseF :: Parser F parseF = do ls <- P.manyTill (P.takeTill isEndOfLine <* endOfLine) (P.lookAhead "BEGIN_SAMPLE") s <- P.many' parseSample pure $! F ls s parseSample :: Parser Sample parseSample = do s <- "BEGIN_SAMPLE " *> double <* endOfLine let endSample = do es <- "END_SAMPLE " *> double <* endOfLine guard (es == s) l = P.takeTill isEndOfLine <* endOfLine ls <- P.manyTill l endSample pure $! Sample s ls renderSample :: Sample -> Text renderSample (Sample d ls) = T.unlines $ ("BEGIN_SAMPLE " <> T.pack (show d)) : ls ++ [ "END_SAMPLE " <> T.pack (show d) ] main :: IO () main = getArgs >>= \case [input, output] -> do c <- T.readFile input case parseOnly parseF c of Left err -> do hPutStrLn stderr ("Parse failed: " <> err) exitFailure Right (F startLines samples) -> do let s' = T.concat . Prelude.map renderSample $ sort samples T.writeFile output (T.unlines startLines <> s') _ -> do hPutStrLn stderr "usage: fix-hp inputFile outputFile" exitFailure }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11645#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#11645: Heap profiling - hp2ps: samples out of sequence -------------------------------------+------------------------------------- Reporter: thomie | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Profiling | Version: 8.0.1-rc2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: | libraries/hpc/tests/fork/hpc_fork Blocked By: | Blocking: Related Tickets: #664, #14257 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * related: #664 => #664, #14257 Comment: I was looking at a similar issue, #14257, and thought I had a theory. Unfortunately it fell apart. I'll try to get back to this soon. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11645#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#11645: Heap profiling - hp2ps: samples out of sequence -------------------------------------+------------------------------------- Reporter: thomie | Owner: (none) Type: bug | Status: closed Priority: normal | Milestone: 8.4.1 Component: Profiling | Version: 8.0.1-rc2 Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: | libraries/hpc/tests/fork/hpc_fork Blocked By: | Blocking: Related Tickets: #664, #14257 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * status: new => closed * resolution: => fixed * milestone: => 8.4.1 Comment: I believe this is another manifestation of #14257. I have verified that this can no longer be reproduced on `master`. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11645#comment:12 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler