Hi, Am Freitag, den 09.12.2016, 09:50 +0800 schrieb Moritz Angermann:

Hi,

let me thank you perusing this!

...

...

I am not sure how useful this is going to be:  + Tests lots of common and important real-world libraries.  − Takes a lot of time to compile, includes CPP macros and C code. (More details in the README linked above).

another problem with the approach of taking modern real-world code: It uses a lot of non-boot libraries that are quite compiler-close and do low-level stuff (e.g. using Template Haskell, or stuff like the). If we add that not nofib, we’d have to maintain its compatibility with GHC as we continue developing GHC, probably using lots of CPP. This was less an issue with the Haskell98 code in nofib.

But is there a way to test realistic modern code without running into this problem?

what are the reasons besides fragmentation for a modern real-world test suite outside of ghc (maybe even maintained by a different set of people)?

I am not sure what you are saying. Are you proposing the maintain a benchmark set outside GHC, or did you get the impression that I am proposing it?

At some point you would also end up having a matrix of performance measurements due to the evolution of the library and the evolution of ghc. Fixing the library to profile against ghc will likely end at some point in incompatibility with ghc. Fixing ghc will similarly at some point end with the inability to compile the library.

My motivation right now is to provide something to measure GHC, so this would involve fixing the library. And that is what I am worried about: Too much maintenance effort in keeping this large piece of code compatible with GHC. But maybe it is ok if it part of nofib, and hence of GHC, so that every breaking change in GHC can immediately be accounted for in the benchmark code. A nice side effect of this might be that GHC developers can get a better idea of how much code their change breaks.

What measurements did you collect? Are these broken down per module?

Nothing yet, this is on the TODO list.

Something I’ve recently had some success with was dumping measurements into influxdb[1] (or a similar data point collections service) and hook that up to grafana[2] for visualization.

Nice! Although these seem to be tailored for data-over-time, not data-over-commit. This mismatch in the data model was part of the motivation for me to create gipeda, which powers https://perf.haskell.org/ghc/ Greetings, Joachim -- Joachim “nomeata” Breitner   mail@joachim-breitner.de • https://www.joachim-breitner.de/   XMPP: nomeata@joachim-breitner.de • OpenPGP-Key: 0xF0FBF51F   Debian Developer: nomeata@debian.org

Hi, Am Freitag, den 09.12.2016, 13:54 +0800 schrieb Moritz Angermann:

...

I am not sure what you are saying. Are you proposing the maintain a benchmark set outside GHC, or did you get the impression that I am proposing it?

Yes, that’s what *I* am proposing for the reasons I mentioned; one I did not yet mention is time. Running nofib takes time, adding more time consuming performance tests would reduce their likelihood of being run in my experience.  As I see this as being almost completely scriptable, this could live outside of ghc i think. 

I don’t think the running time of nofib is a constraint at the moment, and I expect most who run nofib to happily let it run for a few minutes more in order to get more meaningful results.

...

But maybe it is ok if it part of nofib, and hence of GHC, so that every breaking change in GHC can immediately be accounted for in the benchmark code.

A nice side effect of this might be that GHC developers can get a better idea of how much code their change breaks.

I’m not much a fan of this, but that’s just my opinion :-)

What is the alternative? Keep updating the libraries? But libraries change APIs. Then you need to keep updating the program itself? That seems to be too many moving parts for a benchmark suite.

...

...

Something I’ve recently had some success with was dumping measurements into influxdb[1] (or a similar data point collections service) and hook that up to grafana[2] for visualization.

Nice! Although these seem to be tailored for data-over-time, not data-over-commit. This mismatch in the data model was part of the motivation for me to create gipeda, which powers https://perf.haskell.org/ghc/

Assuming we confine this to a particular branch, or discriminate by branch, commits would be measured in sequence anyway, and the timestamp could be the time of the reporting of the measurement, and the respective ghc commit hash end up being an annotation. While this is not very pretty (and I would hope that grafana has some other ability to enrich the hover-tooltips) it could present a flexible solution without requiring additional engineering effort.

However, if gipeda is sufficient, please ignore my comment :)

Oh, we could certainly do better here! (But it serves my purposes for now, so I’ll stick to it until someone sets up something better, in which case I happily dump my code.) Greetings, Joachim -- Joachim “nomeata” Breitner   mail@joachim-breitner.de • https://www.joachim-breitner.de/   XMPP: nomeata@joachim-breitner.de • OpenPGP-Key: 0xF0FBF51F   Debian Developer: nomeata@debian.org

On 2016-12-09 08:31, Bardur Arantsson wrote:

Actually, now that I think about it: What about if this were integrated into the Cabal infrastructure? If I specify "upload-perf-numbers: True" in my .cabal file, any project on (e.g.) GitHub that wanted to opt-in could do so, they could build using Travis, and voila!

Post-shower addendum: If we had the right hooks in Cabal we could even also track the *runtimes* of all the tests. (Obviously a bit more brittle because one expects that adding tests would cause a performance hit, but could still be valuable information for the projects themselves to have -- which could be a motivating factor for opting in to this scheme.) Obviously it would have to be made very easy[1] to compile with GHC HEAD on travis for this to have much value for tracking regressions "as they happen" and perhaps a "hey-travis-rebuild-project" trigger would have to be implemented to get daily/weekly builds even when the project itself has no changes. We could perhaps also marshal a bit of the Hackage infrastructure instead? Anyway, loads of variations on this theme. The key point here is that the burden of keeping the "being tested" code working with GHC HEAD is on the maintainers of said projects... and they already have motivation to do so if they can get early feedback on breakage og regressions on compile times and run times. Regards,

I have wanted telemetry for years. ("Telemetry" is the term Microsoft, and I think others, use for the phone-home feature.) It would tell us how many people are using GHC; currently I have literally no idea. It could tell us which language features are most used. Perhaps it could tell us about performance, but I'm not sure how we could make use of that info without access to the actual source. The big issue is (a) design and implementation effort, and (b) dealing with the privacy issues. I think (b) used to be a big deal, but nowadays people mostly assume that their software is doing telemetry, so it feels more plausible. But someone would need to work out whether it had to be opt-in or opt-out, and how to actually make it work in practice. Simon | -----Original Message----- | From: ghc-devs [mailto:ghc-devs-bounces@haskell.org] On Behalf Of | Bardur Arantsson | Sent: 09 December 2016 07:32 | To: ghc-devs@haskell.org | Subject: Re: Attempt at a real world benchmark | | On 2016-12-08 17:04, Joachim Breitner wrote: | > Hi, | > | > Am Donnerstag, den 08.12.2016, 01:03 -0500 schrieb Joachim Breitner: | >> I am not sure how useful this is going to be: | >> + Tests lots of common and important real-world libraries. | >> − Takes a lot of time to compile, includes CPP macros and C code. | >> (More details in the README linked above). | > | > another problem with the approach of taking modern real-world code: | > It uses a lot of non-boot libraries that are quite compiler-close | and | > do low-level stuff (e.g. using Template Haskell, or stuff like the). | > If we add that not nofib, we’d have to maintain its compatibility | with | > GHC as we continue developing GHC, probably using lots of CPP. This | > was less an issue with the Haskell98 code in nofib. | > | > But is there a way to test realistic modern code without running | into | > this problem? | > | | This may be a totally crazy idea, but has any thought been given a | "Phone Home"-type model? | | Very simplistic approach: | | a) Before it compiles, GHC computes a hash of the file. | b) GHC has internal profiling "markers" in its compilation pipeline. | c) GHC sends those "markers" + hash to some semi-centralized highly- | available service somewhere under *.haskell.org. | | The idea is that the fact that "hashes are equal" => "performance | should be comparable". Ideally, it'd probably be best to be able to | have the full source, but that may be a tougher sell, obviously. | | (Obviously would have to be opt-in, either way.) | | There are a few obvious problems with this, but an obvious win would | be that it could be done on a massively *decentralized* scale. Most | problematic part might be that it wouldn't be able to track things | like "I changed $this_line and now it compiles twice as slow". | | Actually, now that I think about it: What about if this were | integrated into the Cabal infrastructure? If I specify "upload-perf- | numbers: True" | in my .cabal file, any project on (e.g.) GitHub that wanted to opt-in | could do so, they could build using Travis, and voila! | | What do you think? Totally crazy, or could it be workable? | | Regards, | | _______________________________________________ | ghc-devs mailing list | ghc-devs@haskell.org | https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.h | askell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc- | devs&data=02%7C01%7Csimonpj%40microsoft.com%7Cf467f18af1cf48b14f7708d4 | 20056e03%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6361686550849222 | 57&sdata=vhdrztwo4%2FG8yTrSI%2B5aWSZblqoTTBWlIc5LpOMKquQ%3D&reserved=0

...

It could tell us which language features are most used.

Language features are hard if they are not available in separate libs. If in libs, then IIRC debian is packaging those in separate packages, again you can use their package contest.

What in particular makes them hard? Sorry if this seems like a stupid question to you, I'm just not that knowledgeable yet. One reason I can think of would be that we would want attribution, i.e. did the developer turn on the extension himself, or is it just used in a lib or template – but that should be easy to solve with a source hash, right? That source hash itself might need a bit of thought though. Maybe it should not be a hash of a source file, but of the parse tree.

...

The big issue is (a) design and implementation effort, and (b) dealing with the privacy issues. I think (b) used to be a big deal, but nowadays people mostly assume that their software is doing telemetry, so it feels more plausible. But someone would need to work out whether it had to be opt-in or opt-out, and how to actually make it work in practice.

Privacy here is complete can of worms (keep in mind you are dealing with a lot of different law systems), I strongly suggest not to even think about it for a second. Your note "but nowadays people mostly assume that their software is doing telemetry" may perhaps be true in sick mobile apps world, but I guess is not true in the world of developing secure and security related applications for either server usage or embedded.

My first reaction to "nowadays people mostly assume that their software is doing telemetry" was to amend it with "* in the USA" in my mind. But yes, mobile is another place. Nowadays I do assume most software uses some sort of phone-home feature, but that's because it's on my To Do list of things to search for on first configuration. Note that I am using "phone home" instead of "telemetry" because some companies hide it in "check for updates" or mix it with some useless "account" stuff. Finding out where it's hidden and how much information they give about the details tells a lot about the developers, as does opt-in vs opt-out. Therefore it can be a reason to not choose a piece of software or even an ecosystem after a first try. (Let's say an operating system almost forces me to create an online account on installation. That not only tells me I might not want to use that operating system, it also sends a marketing message that the whole ecosystem is potentially toxic to my privacy because they live in a bubble where that appears to be acceptable.) So I do have that aversion even in non-security-related contexts. I would say people are aware that telemetry exists, and developers in particular. I would also say developers are aware of the potential benefits, so they might be open to it. But what they care and worry about is /what/ is reported and how they can /control/ it. Software being Open Source is a huge factor in that, because they know that, at least in theory, they could vet the source. But the reaction might still be very mixed – see Mozilla Firefox. My suggestion would be a solution that gives the developer the feeling of making the choices, and puts them in control. It should also be compatible with configuration management so that it can be integrated into company policies as easily as possible. Therefore my suggestions would be * Opt-In. Nothing takes away the feeling of being in control more than perceived "hijacking" of a device with "spy ware". This also helps circumvent legal problems because the users or their employers now have the responsibility. * The switches to turn it on or off should be in a configuration file. There should be several staged configuration files, one for a project, one for a user, one system-wide. This is for compatibility with configuration management. Configuration higher up the hierarchy override ones lower in the hierarchy, but they can't force telemetry to be on – at least not the sensitive kind. * There should be several levels or a set of options that can be switched on or off individually, for fine-grained control. All should be very well documented. Once integrated and documented, they can never change without also changing the configuration flag that switches them on. There still might be some backlash, but a careful approach like this could soothe the minds. If you are worried that we might get too little data this way, here's another thought, leading back to performance data: The most benefit in that regard would come from projects that are built regularly, on different architectures, with sources that can be inspected and with an easy way to get diffs. In other words, projects that live on github and travis anyway. Their maintainers should be easy to convince to set that little switch to "on". Regards, MarLinn

Simon Peyton Jones via ghc-devs writes:

Just to say:

· Telemetry is a good topic

· It is clearly a delicate one as we’ve already seen from two widely differing reactions. That’s why I have never seriously contemplated doing anything about it.

· I’m love a consensus to emerge on this, but I don’t have the bandwidth to drive it.

Incidentally, when I said “telemetry is common” I meant that almost every piece of software I run on my PC these days automatically checks for updates. It no longer even asks me if I want to do that.. it just does it. That’s telemetry right there: the supplier knows how many people are running each version of their software.

Does this necessarily count as telemetry? To be useful for statistics each installation would need to be uniquely identifiable; it's not clear to me for what fraction of software this holds. Certainly in the open-source world it's rather uncommon to tie telemetry to updates. I suppose in the Windows world this sort of thing may be more common. I'll point out that in general telemetry isn't a terribly common thing to find in open-source software save a few major projects (e.g. Firefox, Debian's popcon). I think we would be the first widely-used compiler to use such technology which does give me pause. Developers in particular tend to be more sensitive to this sort of thing than your average user. Cheers, - Ben

I certainly see the value of telemetry in being able to produce a higher quality product through understanding user behavior. However, I am not sure it is realistic. My clients are very conscious of intellectual property and data privacy concerns, and for some of them, even discussing the possibility of allowing telemetry in any part of the technology stack would damage their trust in me. Even if there were an easy opt-out feature, I would be very concerned that I or someone on my team would accidentally build client code without opting out, and I would have to take serious steps to ensure that this could never occur. I would be thrilled, of course, if such analyses could be easily performed against publicly available code on Hackage, as many tests have done in the past. This would also provide an additional small incentive for people to open source code that they do not have a strategic need to keep secret. MarLinn's idea sounds like a good approach to me, although I agree that it has difficulties. I think the key would be to make the report produced brief, human-readable, and clear enough that a CTO or other executive could easily sign off on "declassifying" it. We could then ask that companies voluntarily submit this report if they wish to have an impact on prioritizing the future of the language. I suppose this is a very strict version of "opt-in", and generally I think that opt-in would be fine, as long as we're very confident that we'll never have a bug that makes it opt-out instead. Ryan On Fri, Dec 9, 2016 at 12:13 PM, MarLinn via ghc-devs wrote:

Pretty random idea: What if ghc exposed measurement points for performance and telemetry, but a separate tool would handle the read-out, configuration, upload etc. That would keep the telemetry from being built-in, while still being a way to get *some* information.

Such a support tool might be interesting for other projects, too, or even for slightly different use cases like monitoring servers. The question is if such a tool would bring enough benefit to enough projects for buy-in and to attract contributors. And just separating it doesn't solve the underlying issues of course, so attracting contributors and buy-in might be even harder than it already is for "normal" projects. Close ties to ghc might improve that, but I doubt how big such an effect would be.

Additionally, this approach would just shift many of the questions over to Haskell-platform and/or Stack instead of addressing them – or even further, on that volatile front-line space where inner-community conflict roared recently. It wouldn't be the worst place to address them, but I would hesitate to throw yet another potential point of contention onto that burned field.

Basically: I like that idea, but I might just have proven it fruitless anyway.

Cheers, MarLinn _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Simon Peyton Jones via ghc-devs :

Just to say:

· Telemetry is a good topic · It is clearly a delicate one as we’ve already seen from two widely differing reactions. That’s why I have never seriously contemplated doing anything about it. · I’m love a consensus to emerge on this, but I don’t have the bandwidth to drive it.

Incidentally, when I said “telemetry is common” I meant that almost every piece of software I run on my PC these days automatically checks for updates. It no longer even asks me if I want to do that.. it just does it. That’s telemetry right there: the supplier knows how many people are running each version of their software.

I think, it is important to notice that the expectations of users varies quite significantly from platform to platform. For example, macOS users on average expect more privacy protections than Windows users and Linux users expect more than macOS users. In particular, a lot of 3rd party software on macOS still asks whether you want to enable automatic update checks. Moreover, while most people tolerate that end user GUI software performs some analytics, I am sure that most users of command line (and especially developer tools) would be very surprised to learn that it performs analytics. Finally, once you gather analytics you need to have a privacy policy in many/most jurisdictions (certainly in EU and AU) these days, which explains what data is gathered, where it is stored, etc. This typically also involves statements about sharing that data. All quite easily covered by a software business, but hard to do in an open source project unless you limit access to the data to a few people. (Even if you ask users for permission to gather data, I am quite sure, you still need a privacy policy.) Manuel

From: ghc-devs [mailto:ghc-devs-bounces@haskell.org] On Behalf Of MarLinn via ghc-devs Sent: 09 December 2016 14:52 To: ghc-devs@haskell.org Subject: Re: Telemetry (WAS: Attempt at a real world benchmark)

It could tell us which language features are most used.

Language features are hard if they are not available in separate libs. If in libs, then IIRC debian is packaging those in separate packages, again you can use their package contest.

What in particular makes them hard? Sorry if this seems like a stupid question to you, I'm just not that knowledgeable yet. One reason I can think of would be that we would want attribution, i.e. did the developer turn on the extension himself, or is it just used in a lib or template – but that should be easy to solve with a source hash, right? That source hash itself might need a bit of thought though. Maybe it should not be a hash of a source file, but of the parse tree.

The big issue is (a) design and implementation effort, and (b) dealing with the privacy issues. I think (b) used to be a big deal, but nowadays people mostly assume that their software is doing telemetry, so it feels more plausible. But someone would need to work out whether it had to be opt-in or opt-out, and how to actually make it work in practice.

Privacy here is complete can of worms (keep in mind you are dealing with a lot of different law systems), I strongly suggest not to even think about it for a second. Your note "but nowadays people mostly assume that their software is doing telemetry" may perhaps be true in sick mobile apps world, but I guess is not true in the world of developing secure and security related applications for either server usage or embedded.

My first reaction to "nowadays people mostly assume that their software is doing telemetry" was to amend it with "* in the USA" in my mind. But yes, mobile is another place. Nowadays I do assume most software uses some sort of phone-home feature, but that's because it's on my To Do list of things to search for on first configuration. Note that I am using "phone home" instead of "telemetry" because some companies hide it in "check for updates" or mix it with some useless "account" stuff. Finding out where it's hidden and how much information they give about the details tells a lot about the developers, as does opt-in vs opt-out. Therefore it can be a reason to not choose a piece of software or even an ecosystem after a first try. (Let's say an operating system almost forces me to create an online account on installation. That not only tells me I might not want to use that operating system, it also sends a marketing message that the whole ecosystem is potentially toxic to my privacy because they live in a bubble where that appears to be acceptable.) So I do have that aversion even in non-security-related contexts.

I would say people are aware that telemetry exists, and developers in particular. I would also say developers are aware of the potential benefits, so they might be open to it. But what they care and worry about is what is reported and how they can control it. Software being Open Source is a huge factor in that, because they know that, at least in theory, they could vet the source. But the reaction might still be very mixed – see Mozilla Firefox.

My suggestion would be a solution that gives the developer the feeling of making the choices, and puts them in control. It should also be compatible with configuration management so that it can be integrated into company policies as easily as possible. Therefore my suggestions would be

· Opt-In. Nothing takes away the feeling of being in control more than perceived "hijacking" of a device with "spy ware". This also helps circumvent legal problems because the users or their employers now have the responsibility.

· The switches to turn it on or off should be in a configuration file. There should be several staged configuration files, one for a project, one for a user, one system-wide. This is for compatibility with configuration management. Configuration higher up the hierarchy override ones lower in the hierarchy, but they can't force telemetry to be on – at least not the sensitive kind.

· There should be several levels or a set of options that can be switched on or off individually, for fine-grained control. All should be very well documented. Once integrated and documented, they can never change without also changing the configuration flag that switches them on.

There still might be some backlash, but a careful approach like this could soothe the minds.

If you are worried that we might get too little data this way, here's another thought, leading back to performance data: The most benefit in that regard would come from projects that are built regularly, on different architectures, with sources that can be inspected and with an easy way to get diffs. In other words, projects that live on github and travis anyway. Their maintainers should be easy to convince to set that little switch to "on".

Regards, MarLinn

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Am 10.12.2016 um 19:10 schrieb Moritz Angermann : Two items I would like to note, that *do* phone home and are *out out*:

- homebrew[1] package manager that I assume quite a few people use (because it works rather well), see the Analytics.md[2], especially the opt-out section[3]. - cocoapods[4] (iOS/macOS library repository), which sends back statistics about package usage[5]

Package managers are inherently different to compilers as the *core* functionality of a package manager requires network traffic. As network traffic on every sane server infrastructure is logged, the use of a package manager obviously creates a trail. Now, the package manager can capture more or less local information and server side the data can be kept for varying amounts of time. In contrast, it is a reasonable expectation that a compiler does not initiate any network traffic as it is not needed for its core functionality. In other words, adding more analytics to cabal would probably be fairly uncontroversial (if it anonymises the data properly), but adding it to GHC will make some people very unhappy. Manuel

...

On Dec 10, 2016, at 1:34 PM, Manuel M T Chakravarty wrote:

...

Simon Peyton Jones via ghc-devs :

Just to say:

· Telemetry is a good topic · It is clearly a delicate one as we’ve already seen from two widely differing reactions. That’s why I have never seriously contemplated doing anything about it. · I’m love a consensus to emerge on this, but I don’t have the bandwidth to drive it.

Incidentally, when I said “telemetry is common” I meant that almost every piece of software I run on my PC these days automatically checks for updates. It no longer even asks me if I want to do that.. it just does it. That’s telemetry right there: the supplier knows how many people are running each version of their software.

I think, it is important to notice that the expectations of users varies quite significantly from platform to platform. For example, macOS users on average expect more privacy protections than Windows users and Linux users expect more than macOS users. In particular, a lot of 3rd party software on macOS still asks whether you want to enable automatic update checks.

Moreover, while most people tolerate that end user GUI software performs some analytics, I am sure that most users of command line (and especially developer tools) would be very surprised to learn that it performs analytics.

Finally, once you gather analytics you need to have a privacy policy in many/most jurisdictions (certainly in EU and AU) these days, which explains what data is gathered, where it is stored, etc. This typically also involves statements about sharing that data. All quite easily covered by a software business, but hard to do in an open source project unless you limit access to the data to a few people. (Even if you ask users for permission to gather data, I am quite sure, you still need a privacy policy.)

Manuel

...

From: ghc-devs [mailto:ghc-devs-bounces@haskell.org] On Behalf Of MarLinn via ghc-devs Sent: 09 December 2016 14:52 To: ghc-devs@haskell.org Subject: Re: Telemetry (WAS: Attempt at a real world benchmark)

It could tell us which language features are most used.

Language features are hard if they are not available in separate libs. If in libs, then IIRC debian is packaging those in separate packages, again you can use their package contest.

What in particular makes them hard? Sorry if this seems like a stupid question to you, I'm just not that knowledgeable yet. One reason I can think of would be that we would want attribution, i.e. did the developer turn on the extension himself, or is it just used in a lib or template – but that should be easy to solve with a source hash, right? That source hash itself might need a bit of thought though. Maybe it should not be a hash of a source file, but of the parse tree.

The big issue is (a) design and implementation effort, and (b) dealing with the privacy issues. I think (b) used to be a big deal, but nowadays people mostly assume that their software is doing telemetry, so it feels more plausible. But someone would need to work out whether it had to be opt-in or opt-out, and how to actually make it work in practice.

Privacy here is complete can of worms (keep in mind you are dealing with a lot of different law systems), I strongly suggest not to even think about it for a second. Your note "but nowadays people mostly assume that their software is doing telemetry" may perhaps be true in sick mobile apps world, but I guess is not true in the world of developing secure and security related applications for either server usage or embedded.

My first reaction to "nowadays people mostly assume that their software is doing telemetry" was to amend it with "* in the USA" in my mind. But yes, mobile is another place. Nowadays I do assume most software uses some sort of phone-home feature, but that's because it's on my To Do list of things to search for on first configuration. Note that I am using "phone home" instead of "telemetry" because some companies hide it in "check for updates" or mix it with some useless "account" stuff. Finding out where it's hidden and how much information they give about the details tells a lot about the developers, as does opt-in vs opt-out. Therefore it can be a reason to not choose a piece of software or even an ecosystem after a first try. (Let's say an operating system almost forces me to create an online account on installation. That not only tells me I might not want to use that operating system, it also sends a marketing message that the whole ecosystem is potentially toxic to my privacy because they live in a bubble where that appears to be acceptable.) So I do have that aversion even in non-security-related contexts.

I would say people are aware that telemetry exists, and developers in particular. I would also say developers are aware of the potential benefits, so they might be open to it. But what they care and worry about is what is reported and how they can control it. Software being Open Source is a huge factor in that, because they know that, at least in theory, they could vet the source. But the reaction might still be very mixed – see Mozilla Firefox.

My suggestion would be a solution that gives the developer the feeling of making the choices, and puts them in control. It should also be compatible with configuration management so that it can be integrated into company policies as easily as possible. Therefore my suggestions would be

· Opt-In. Nothing takes away the feeling of being in control more than perceived "hijacking" of a device with "spy ware". This also helps circumvent legal problems because the users or their employers now have the responsibility.

· The switches to turn it on or off should be in a configuration file. There should be several staged configuration files, one for a project, one for a user, one system-wide. This is for compatibility with configuration management. Configuration higher up the hierarchy override ones lower in the hierarchy, but they can't force telemetry to be on – at least not the sensitive kind.

· There should be several levels or a set of options that can be switched on or off individually, for fine-grained control. All should be very well documented. Once integrated and documented, they can never change without also changing the configuration flag that switches them on.

There still might be some backlash, but a careful approach like this could soothe the minds.

If you are worried that we might get too little data this way, here's another thought, leading back to performance data: The most benefit in that regard would come from projects that are built regularly, on different architectures, with sources that can be inspected and with an easy way to get diffs. In other words, projects that live on github and travis anyway. Their maintainers should be easy to convince to set that little switch to "on".

Regards, MarLinn

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

I would opt-in. I also agree with Simon that privacy is no longer a big deal although I do believe that most companies do telemetry with an opt in policy. If it's opt-in why would anyone have a problem with telemetry? On Fri, Dec 9, 2016 at 1:46 PM Tom Murphy wrote:

On Fri, Dec 9, 2016 at 4:50 AM, Simon Peyton Jones via ghc-devs < ghc-devs@haskell.org> wrote:

I have wanted telemetry for years. ("Telemetry" is the term Microsoft, and I think others, use for the phone-home feature.)

It would tell us how many people are using GHC; currently I have literally no idea.

In practice I think the best data we could get is "how many people are using GHC && are willing to opt into phone-home," which seems like a rougher number than e.g. downloads of ghc/HP or number of downloads of base/containers or something similar. I also would not opt in.

Tom _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

On Fri, Dec 9, 2016 at 9:50 AM, Simon Peyton Jones via ghc-devs < ghc-devs@haskell.org> wrote:

The big issue is (a) design and implementation effort, and (b) dealing with the privacy issues.

And (c) not everyone is going to upgrade their ghc, even if you backport the telemetry to older versions (potentially back to 7.6.3 or even earlier), so likely you'd only get telemetry from new users. -- brandon s allbery kf8nh sine nomine associates allbery.b@gmail.com ballbery@sinenomine.net unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net