Thanks for the feedback, Ömer. I've added a Git Notes Format https://ghc.haskell.org/trac/ghc/wiki/Performance/Tests#GitNotesFormat section. I've also moved the drift issue into the "Remaining Work" section and gave it its own subsection. I think a solution to this is not obvious, but I agree that we could benefit from a plan here. - David On 13/09/18 13:25, Ömer Sinan Ağacan wrote:

Thanks for doing this. I think it's great that someone's working on the test suite.

About storing in git notes: what is the format here? If I want to see numbers for a given commit does `git notes show <commit>` show me all the numbers at that commit or only the differences from previous commit? Do we have an example note to look at to get an idea of what the format is?

I think it'd be better if we could make a more concrete plan about the "drifting" problem mentioned in the future work before merging this. I don't know what are the plans here (how much time will be spent on this work) but that future work may never happen, so perhaps for the time being we want to keep the reference values (not sure where to keep them though) to avoid making some programs run 20x slower in a few years. I think it'd also be better if we could highlight this change more in the proposal as I think it's an important one.

Ömer

David Eichmann , 13 Eyl 2018 Per, 11:25 tarihinde şunu yazdı:

...

Hello all,

I've recently resumed some work started by Jared Weakly on the GHC test suite. This specifically regards performance tests. The work aims to, among some other things, reduce manual work and to log performance test results from our CI server. The proposed change is described in more detail on this wiki page: https://ghc.haskell.org/trac/ghc/wiki/Performance/Tests. I'd appreciate any feedback or questions on this.

Thank you and have a great day, David Eichmann

-- David Eichmann, Haskell Consultant Well-Typed LLP, http://www.well-typed.com

Registered in England & Wales, OC335890 118 Wymering Mansions, Wymering Road, London W9 2NF, England

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

-- David Eichmann, Haskell Consultant Well-Typed LLP, http://www.well-typed.com Registered in England & Wales, OC335890 118 Wymering Mansions, Wymering Road, London W9 2NF, England

Thanks, David! The rebase of the branch has been sitting half complete on my laptop for a shamefully long time as I've been swamped at work. Please feel free to reach out if you have questions and let me know if there's anything I can do to speed things up. On Wed, Oct 10, 2018, 2:42 AM David Eichmann wrote:

Hello all,

After some time and feedback, bgamari and and I have decided to slowly press on with the proposed changes to the performance test-suite. I'll continue to update the wiki page https://ghc.haskell.org/trac/ghc/wiki/Performance/Tests in the future and will inform ghc-devs of significant milestones. Note I've augmented the wiki page's drift issue section and moved it to future work.

Thank you,

David Eichmann

P.S.

On 13/09/18 09:24, David Eichmann wrote:

Hello all,

I've recently resumed some work started by Jared Weakly on the GHC test suite. This specifically regards performance tests. The work aims to, among some other things, reduce manual work and to log performance test results from our CI server. The proposed change is described in more detail on this wiki page: https://ghc.haskell.org/trac/ghc/wiki/Performance/Tests. I'd appreciate any feedback or questions on this.

Thank you and have a great day, David Eichmann

-- David Eichmann, Haskell Consultant Well-Typed LLP, http://www.well-typed.com

Registered in England & Wales, OC335890 118 Wymering Mansions, Wymering Road, London W9 2NF, England

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs