Hi, I am currently working on a piece of code (an analysis to solve #7994) where I’d like to make sure that my changes do not regress over what I had before. But I find it unnecessarily hard to write our usual test-case styles for them: * I’d like to test against very small Core that does not involve anything unnecessary. But it is hard to write Haskell that has, when it hits my analysis, this shape. It requires lots of {-# NOINLINE #-} and other tricks. * To test the result, I either have to write a performance test, but it is not always easy to come up with a program where the gains are massive enough to become a reliable test. It is possible, but work, and doing it maybe half a dozen times for various inputs is tricky. * Alternative, I can dump the Core and add that to the test cases. But now other changes to the compiler can easily trigger my test case failing. So I thought about writing a test case that simply imports my module from the ghc library, generates artificial, minimal core, and checks the output for precisely what I want (in my case, some fields of the IdInfo of various binders). I don’t see any examples for that in the test suite. Is that just because noone has done that before, or is there inherently bad about this approach that we do _not_ want to that? Also, we don’t have a parser for Core, so I’ll have to build my syntax trees using the stuff from MkCore et al, right? Thanks, Joachim -- Joachim “nomeata” Breitner mail@joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata@joachim-breitner.de • GPG-Key: 0x4743206C Debian Developer: nomeata@debian.org
Let me take a different slice at this question, inspired more by Joachim's subject line than his text: On a number of occasions I've wanted to write unit tests against a certain function or set of functions. The role inference algorithm is a prime example, but it's happened elsewhere, too. The testsuite only performs end-to-end testing. Sometimes it's easy/possible to build a test that gets at what I want, but sometimes it's very hard. (Case in point: I revised the varSetElemsKvsFirst function on a branch -- it's really hard to test that thoroughly in an end-to-end test!)
So, is there a way / does someone know how to make a way to do proper unit testing? The ability to do such tests is treated as a key virtue of (pure) functional programming, and yet we don't do it! :)
For my varSetElemsKvsFirst problem, I ended up copying the code to a new file, writing dummy data structures to get it to compile, and then ran unit tests. I fixed my bug, but there was no way to integrate the testing work into a regression test, sadly.
Any thoughts?
Thanks,
Richard
On Jan 29, 2014, at 4:58 AM, Joachim Breitner
Hi,
I am currently working on a piece of code (an analysis to solve #7994) where I’d like to make sure that my changes do not regress over what I had before. But I find it unnecessarily hard to write our usual test-case styles for them: * I’d like to test against very small Core that does not involve anything unnecessary. But it is hard to write Haskell that has, when it hits my analysis, this shape. It requires lots of {-# NOINLINE #-} and other tricks. * To test the result, I either have to write a performance test, but it is not always easy to come up with a program where the gains are massive enough to become a reliable test. It is possible, but work, and doing it maybe half a dozen times for various inputs is tricky. * Alternative, I can dump the Core and add that to the test cases. But now other changes to the compiler can easily trigger my test case failing.
So I thought about writing a test case that simply imports my module from the ghc library, generates artificial, minimal core, and checks the output for precisely what I want (in my case, some fields of the IdInfo of various binders).
I don’t see any examples for that in the test suite. Is that just because noone has done that before, or is there inherently bad about this approach that we do _not_ want to that?
Also, we don’t have a parser for Core, so I’ll have to build my syntax trees using the stuff from MkCore et al, right?
Thanks, Joachim
-- Joachim “nomeata” Breitner mail@joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata@joachim-breitner.de • GPG-Key: 0x4743206C Debian Developer: nomeata@debian.org _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Hi, Am Mittwoch, den 29.01.2014, 10:12 -0500 schrieb Richard Eisenberg:
So, is there a way / does someone know how to make a way to do proper unit testing? The ability to do such tests is treated as a key virtue of (pure) functional programming, and yet we don't do it! :)
I’m now doing this: http://git.haskell.org/ghc.git/commitdiff/aa970ca1e81118bbf37386b8833a01a379... (Patch “Add a unit test for CallArity” on branch wip/T7994, in case the hash id is invalid later) which I believe is okaish. Greetings, Joachim PS: I’m subscribed to the list, no need to send a copy to my private address. -- Joachim “nomeata” Breitner mail@joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata@joachim-breitner.de • GPG-Key: 0x4743206C Debian Developer: nomeata@debian.org
Having an easy way to write unit tests would be great. I made an attempt to do this during my internship (this wasn't merged): https://github.com/jstolarek/testsuite/blob/0829be3a00eecfab0d7026c9a5dc18dc... https://github.com/jstolarek/testsuite/blob/0829be3a00eecfab0d7026c9a5dc18dc... But that is far from a real unit test: results of many unit tests coupled together in one stdout file, lots of boilerplate and reinventing the wheel. I would love to see the testsuite extended with an easy way to write unit tests but I guess that might require some substantial effort.
PS: I’m subscribed to the list, no need to send a copy to my private address. This is typically done to alert someone that he/she has been addressed directly in a discussion. I for example have my filters set in such a way that all ghc-devs mails are automatically marked as read unless I am CC'd.
Janek
Hi, Am Mittwoch, den 29.01.2014, 23:18 +0100 schrieb Jan Stolarek:
PS: I’m subscribed to the list, no need to send a copy to my private address.
This is typically done to alert someone that he/she has been addressed directly in a discussion. I for example have my filters set in such a way that all ghc-devs mails are automatically marked as read unless I am CC'd.
if someone really needs urgent attention from me, putting me in CC is fine: Mail directed to me will cause popups and land in my Inbox. But doing so carelessly makes this distinction useless; for example with the recent pattern synonym thread, I once made a minor comment and got a dozend mails explicitly sent to me. This is alerting thing is clearly not working – and I am tempted to the opposite of what you do: Automatically delete any mail reaching my inbox that also goes to ghc-dev (and stop whining here). But before doing that, I’ll try using the Reply-To header, let’s see if that works better. BTW, does everyone know about Reply-To-List (sometimes calle Group Reply, Ctrl-L in evolution) instead of Reply-To-All? But I heard rumors that Outlook does not support that, and – unlike in the Debian community – that would be a problem. I guess Reply-To can help then. Greetings, Joachim -- Joachim “nomeata” Breitner mail@joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata@joachim-breitner.de • GPG-Key: 0x4743206C Debian Developer: nomeata@debian.org
On 29/01/14 22:47, Joachim Breitner wrote:
Hi,
Am Mittwoch, den 29.01.2014, 23:18 +0100 schrieb Jan Stolarek:
PS: I’m subscribed to the list, no need to send a copy to my private address.
This is typically done to alert someone that he/she has been addressed directly in a discussion. I for example have my filters set in such a way that all ghc-devs mails are automatically marked as read unless I am CC'd.
if someone really needs urgent attention from me, putting me in CC is fine: Mail directed to me will cause popups and land in my Inbox. But doing so carelessly makes this distinction useless; for example with the recent pattern synonym thread, I once made a minor comment and got a dozend mails explicitly sent to me. This is alerting thing is clearly not working – and I am tempted to the opposite of what you do: Automatically delete any mail reaching my inbox that also goes to ghc-dev (and stop whining here).
But before doing that, I’ll try using the Reply-To header, let’s see if that works better.
BTW, does everyone know about Reply-To-List (sometimes calle Group Reply, Ctrl-L in evolution) instead of Reply-To-All? But I heard rumors that Outlook does not support that, and – unlike in the Debian community – that would be a problem. I guess Reply-To can help then.
Thunderbird has this, Reply List. I think the problem is that the reply headers sometimes get messed up: I always use Reply List but even then, Thunderbird often either replies to the person in question and CCs the list or the other way around. For this e-mail, I am only sending to the list, using Reply List. I think it adds the person by default if they address mail e-mail and similar thing probably happens for others. I try to remove such occurrences manually but I'm sure it sometimes slips by. I think that as long as you read ghc-devs, it's fine to set up your client to not show you things in your inbox.
Greetings, Joachim
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
-- Mateusz K.
Ah, yes, the reply-to worked great.
I use the Mac Mail app, which lacks a reply-to-list feature (as far as I can tell). So, without a reply-to in the header, I have two choices without manual override: reply sender (omits the list) or reply all (includes both sender and list). Of course, I can tailor my headers before I send, but it honestly never occurred to me that this mattered. Now that you point out your filtering strategy, it's obvious. In any case, by putting in the reply-to, all is better.
Keeping in mind that others are likely using a similar filtering strategy, I will try to pay more attention to this in the future.
Richard
On Jan 29, 2014, at 5:47 PM, Joachim Breitner
Hi,
Am Mittwoch, den 29.01.2014, 23:18 +0100 schrieb Jan Stolarek:
PS: I’m subscribed to the list, no need to send a copy to my private address.
This is typically done to alert someone that he/she has been addressed directly in a discussion. I for example have my filters set in such a way that all ghc-devs mails are automatically marked as read unless I am CC'd.
if someone really needs urgent attention from me, putting me in CC is fine: Mail directed to me will cause popups and land in my Inbox. But doing so carelessly makes this distinction useless; for example with the recent pattern synonym thread, I once made a minor comment and got a dozend mails explicitly sent to me. This is alerting thing is clearly not working – and I am tempted to the opposite of what you do: Automatically delete any mail reaching my inbox that also goes to ghc-dev (and stop whining here).
But before doing that, I’ll try using the Reply-To header, let’s see if that works better.
BTW, does everyone know about Reply-To-List (sometimes calle Group Reply, Ctrl-L in evolution) instead of Reply-To-All? But I heard rumors that Outlook does not support that, and – unlike in the Debian community – that would be a problem. I guess Reply-To can help then.
Greetings, Joachim
-- Joachim “nomeata” Breitner mail@joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata@joachim-breitner.de • GPG-Key: 0x4743206C Debian Developer: nomeata@debian.org _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Ironically, this is the first ghc-devs email to ever get sent to my bulk
folder. :)
I can't speak for others, but I try to read most of the email in devs and
related lists (for good or for ill). Though this certainly gets in the way
of getting work done sometimes :)
On Wednesday, January 29, 2014, Joachim Breitner
Hi,
PS: I’m subscribed to the list, no need to send a copy to my private address.
This is typically done to alert someone that he/she has been addressed
Am Mittwoch, den 29.01.2014, 23:18 +0100 schrieb Jan Stolarek: directly in a discussion. I
for example have my filters set in such a way that all ghc-devs mails are automatically marked as read unless I am CC'd.
if someone really needs urgent attention from me, putting me in CC is fine: Mail directed to me will cause popups and land in my Inbox. But doing so carelessly makes this distinction useless; for example with the recent pattern synonym thread, I once made a minor comment and got a dozend mails explicitly sent to me. This is alerting thing is clearly not working – and I am tempted to the opposite of what you do: Automatically delete any mail reaching my inbox that also goes to ghc-dev (and stop whining here).
But before doing that, I’ll try using the Reply-To header, let’s see if that works better.
BTW, does everyone know about Reply-To-List (sometimes calle Group Reply, Ctrl-L in evolution) instead of Reply-To-All? But I heard rumors that Outlook does not support that, and – unlike in the Debian community – that would be a problem. I guess Reply-To can help then.
Greetings, Joachim
-- Joachim “nomeata” Breitner mail@joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata@joachim-breitner.de • GPG-Key: 0x4743206C Debian Developer: nomeata@debian.org
participants (5)
-
Carter Schonwald -
Jan Stolarek -
Joachim Breitner -
Mateusz Kowalczyk -
Richard Eisenberg