Re: [Haskell-community] 2018 state of Haskell survey

Hi Taylor. We're discussing this in the committee. I agree that to the extent they can accurately reflect something, language surveys are useful, and appreciate that you want to run a useful survey, and certainly want to encourage and help you in making it as broad and useful as possible. That said, I don't know if slapping a "haskell.org" label on the survey will help manage the biggest drivers of selection bias -- which is not only about who chooses to respond, but about who is reached through what mechanisms. (I don't know the relative importance of response-bias vs. outreach-bias in general, and would be curious if somebody has some good research on that to point to). I honestly don't know if we have enough channels _in general_ to do a good survey, no matter who runs it or how at all! Regardless of the decision we come to, here are a few of my personal thoughts on the questions you have thus far, and what could be added: 1) A question "how did you hear about this survey" -- this could at least help to disentangle outreach-bias, or notice it, depending on if it induces any correlations. 2) A question on how long after a new GHC release users upgrade -- both personally, and at work. 3) Distinguishing between personal and work build-systems in the relevant question. 4) I think the sorts of questions that make sense to ask in this early part can resemble those in the first part of the Go survey: https://blog.golang.org/survey2017-results (I especially like the questions about the area of development and server vs cli apps vs libraries). I also like their questions about what environments teams deploy programs to. It would also be worth asking if the apps developed are customer-facing or internal. 5) I think an interesting question would be what preferred js solution, if any, teams adopt -- i.e. ghcjs, typescript, purescript, raw js, etc. 6) for the "why did you stop" question, there are a good range of potential multi-choice answers that can be drawn from with the rust user survey: https://blog.rust-lang.org/2017/09/05/Rust-2017-Survey-Results.html Cheers, Gershom On Sun, Oct 14, 2018 at 3:32 PM Taylor Fausak wrote:

I'm not entirely sure what official support would look like. A few things come to mind:

1. Simply putting "official" somewhere in the title, such as: Official 2018 state of Haskell survey.

2. Putting something about Haskell.org in the description of the survey, such as: Sponsored by Haskell.org.

3. Announcing the survey through channels that I may not be aware of. Or helping me announce the survey through various channels (mailing lists, Reddit, and so on) by mentioning Haskell.org.

I'm sure there are more ways that I'm not thinking of.

Perhaps it would be better to state my goal and see what, if anything, can be to achieve that goal? My goal is for this survey to be *the* authoritative Haskell survey and for the community to broadly accept it results. In particular I would like to avoid reactions like these to the recent FP Complete survey:

...

I browse r/haskell all the time and follow FPco on Twitter, and I wasn't aware of this survey. [1]

...

It should not be surprising to think that fp complete has much better outreach to Stackage users than to non Stackage users. [2]

...

Any survey hosted by FPComplete is biased towards users of stack, for reasons that should be self evident. [3]

Similar sentiments have been expressed about last year's Haskell Weekly survey:

...

Haskell Weekly's reputation is tainted as it appears to be seen as partisan (and I tend to agree). [4]

...

one of those surveys is from FP Complete and one of them is from someone who I would consider very partisan in these kind of discussions. [5]

...

For the love of god stop posting those surveys. They're not convincing and obviously flawed. [6]

I don't expect to be able to make everyone happy, but I think that presenting this year's survey as sponsored by both Haskell Weekly and Haskell.org would go a long way toward making it acceptable to a broad range of the Haskell community.

I hope that helps!

[1]: https://np.reddit.com/r/haskell/comments/9mm05d/2018_haskell_survey_results/... [2]: https://np.reddit.com/r/haskell/comments/8tc8pr/fp_complete_launches_new_blo... [3]: https://np.reddit.com/r/haskell/comments/9mm05d/2018_haskell_survey_results/... [4]: https://np.reddit.com/r/haskell/comments/9mm05d/2018_haskell_survey_results/... [5]: https://np.reddit.com/r/haskell/comments/8tc8pr/fp_complete_launches_new_blo... [6]: https://np.reddit.com/r/haskell/comments/8uw9hw/psa_cabal_breaks_with_yaml08...

On Sun, Oct 14, 2018, at 2:21 PM, Neil Mitchell wrote:

Hi Taylor,

What does official support look like? I don't think there's anything above and beyond what you're already doing.

Thanks, Neil

On Sun, 14 Oct 2018 at 3:49 pm, Taylor Fausak wrote:

(I have CCed because the haskell-community mailing list seems relatively quiet and I want to make sure this is seen.)

Hello! My name is Taylor Fausak. I run the Haskell Weekly newsletter. Last year I published a survey [1] for the Haskell community. I collected and reported [2] on about 1,335 responses. I plan on publishing another survey this year on the same date, November 1st. I am developing it in the open again [3] and would love to hear from any interested parties. Please let me know if you have any ideas about the survey!

This year I am interested in making the survey official by seeking support from Haskell.org. Is such a thing possible and desirable?

Thanks for your consideration! I hope to hear from you soon.

[1]: https://haskellweekly.news/surveys/2017.html [2]: https://taylor.fausak.me/2017/11/15/2017-state-of-haskell-survey-results/ [3]: https://github.com/haskellweekly/haskellweekly.github.io/issues/206 _______________________________________________ Haskell-community mailing list Haskell-community@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

_______________________________________________ Haskell-community mailing list Haskell-community@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community