Hi all, I'd like to share with you a few of the things that happened during the past months on and around DataHaskell (DH), and a summary of the current state of things. Time permitting, this will become a regular newsletter, with the idea to keep up to date those who don't usually hang out on our Gitter chatroom [1]. ---------------------------------------------------------------------------------------------------------------- * Outreach activities and related meetings ICFP 2017 : Early September 2017 we had an hour-long lunch mini-workshop at ICFP during which a few people presented how they use Haskell in their numerical and data-crunching work. Mostly positive opinions, as well as a "contrarian" viewpoint (from an expert practitioner) who lamented the low-performance of native Haskell numerical code (e.g. the large memory footprint of boxed data). Michal Gajda presented how he uses IHaskell for interactive data exploration (btw, it's a thing and you can try it today!), Trevor McDonell introduced Accelerate (the high-performance array library that can target CUDA GPUs), Adam Scibior presented `monad-bayes` and Praveen Narayan presented `hakaru` (two probabilistic progamming laguages embedded in Haskell). ZuriHac 2018 : community bonding mostly (read: no hacking was done). Received more feedback of the usual sort "how do I do X in Haskell?" "I was expecting a more advanced state of machine learning capabilities in Haskell", etc. There may or may not be a set of Haskell bindings for Apache Arrow in the works, stay tuned! ICFP 2018 : There will be two very interesting workshops (FHPC, functional programming in high-performance computing and NPFL, numerical programming in functional languages), for those who will be attending the conference. One of the workshop chairs of NPFL is Dominic Steinitz of Tweag, author of the excellent idontgetoutmuch.wordpress.com blog. ---------------------------------------------------------------------------------------------------------------- * DH survey On April 5 I've published a survey [2], that tried to gauge the community interest and pain points related to doing "data science" in our beloved language. The survey is still open but I won't be collecting data anymore (70 people have taken the questionnaire and 62 have completed it), and I've scraped and formatted the dataset (you can find it at https://github.com/DataHaskell/surveys). Some of the free text answers are particularly interesting. I don't have much time currently to create the plots and publish a blog post about it, but it would be a Good Thing to have, if anyone wants to step up. The project backbone is ready for new contributors! ---------------------------------------------------------------------------------------------------------------- * Contributors wanted : External projects At regular intervals, new exciting projects appear and it's becoming quite hard to keep track of everything that's happening in this area. I'd like to highlight here a few that are seeing significant activity as of late and have large impact potential. ** General-purpose (e.g. numerics-related) `numhask` [3] is an experiment at replacing the numerical typeclasses of `base`, in particular those related to `Num`, in favor of a finer-grained and more principled hierarchy. It is already available and receiving quite a bit of attention recently, but there still are a number of areas that could use some help, for example property checking, test coverage, the `accelerate` bindings, etc. ** Data science `boke-hs` [4] : a native Haskell interface to generate Bokeh plots. The project was mostly developed during the ZuriHac weekend; currently it's functional and you can create line plots with it, but helping out with the domain mapping would be a very valuable contribution. In particular, the library emits JSON blobs that are interpreted in the browser by BokehJS ; "domain mapping" means creating the Haskell data that will serialize into JSON appropriately. A currently-unnamed library for linking databases and Haskell dataframes. This is Gagandeep Bhatia's ongoing Summer of Haskell project, and you can find a writeup and links to his current work (which uses `beam` as a database binding library and `Frames` as type-safe in-memory representation) here: https://www.gagandeepbhatia.com/blog/ . `haskell.do` [5] : a native Haskell notebook/interactive editor/IDE for the browser. The project started well, has a website and a few initial releases, but currently needs some love to achieve its full potential. Interactive development with visual feedback is a crucial part of data science work, and contributions to this project will be very valuable. ** Machine learning `hasktorch` [6] is in developer beta ! This is a set of Haskell bindings for `torch` (the deep learning C library), and comes wrapped in a typed interface that provide statically-checked vector dimensions, and the like. ** Probabilistic programming languages `deanie` [7] is a probabilistic EDSL. Currently it's lacking code for its general-purpose inference engine (see https://github.com/jtobin/deanie/blob/master/lib/Deanie/Inference/Comonadic....). The implementation technique is related to "Co-Free for interpreters" as shown in [8]. ---------------------------------------------------------------------------------------------------------------- * Contributors wanted : DataHaskell internal projects We do have a few projects set up that could use some collaborators. If you are willing to contribute to any of these, please open a ticket on the project issue tracker and the maintainers will be in touch shortly. * `type-providers` : a unified code generation library for accessing structured data in a type-safe way : https://github.com/DataHaskell/type-providers . Michal Gajda is willing to mentor a student who wants to translate his own `json-autotype` to a new library that can generate type-safe code for XML, and can be queried in-memory via Frames. Contributing to this library will likely require some knowledge of Template Haskell. * A fork of the venerable `statistics` library, that separates the dense linear algebra library as a standalone library : https://github.com/DataHaskell/statistics . There is currently a PR to the upstream library (https://github.com/bos/statistics/pull/143); longer-term plans include giving it a typeclass-based interface, for example based on `numhask`. * `numhask-linear-algebra` : this is a longer shot at unifying native linear algebra libraries under a single typeclass representation, as provided by `numhask` (project at https://github.com/DataHaskell/numhask-linear-algebra ). This is something that's currently unavailable in the Haskell ecosystem and will be a highly relevant contribution to many. ---------------------------------------------------------------------------------------------------------------- * DataHaskell internal administration In addition to helping out with the survey blogpost, some help in the following areas would be much appreciated ** Knowledge base : testers wanted ! The knowledge base ( http://www.datahaskell.org/docs/community/current-environment.html ) is a growing curated collection of libraries for doing data science-related tasks; anything from manipulating storable data to statistical inference. It's pretty useful as it is, but I think the entries should be annotated with additional information, such as the degree of completeness and developer- and user-friendliness. For this, some help in testing out the libraries and reporting back would be very useful. Ultimately, it would be very useful to contribute such an assessment back to the "state of the ecosystem" document [9], which is highly visible but not so much up to date on the data-science related things. ** Maintainers wanted ! Currently, Nikita and I are the only owners of the github organization. It would be great if additional people stepped up for "tending the garden", e.g. keeping track of issue tickets, filling out the documentation, lending a hand on the gitter channel to address newcomers' issues, etc. Historically, data science and numerical computing have been underserved niches of the Haskell ecosystem, and while things are steadily improving, together we can bring about this change soon ! ---------------------------------------------------------------------------------------------------------------- That's it for now, I hope you enjoyed this Newsletter; don't hesitate to share your thoughts either here or on our Gitter chatroom! [1] Marco github.com/ocramz ---------------------------------------------------------------------------------------------------------------- References : [1] DH Gitter chatroom : https://gitter.im/dataHaskell/Lobby [2] DH user survey April 2018: https://www.surveymonkey.com/r/3FBBJWR [3] numhask : github.com/tonyday567/numhask [4] boke-hs , native Haskell bindings for Bokeh : https://github.com/ahaym/boke-hs [5] HaskellDO, the interactive Haskell editor: http://haskell.do/ , https://github.com/theam/haskell-do [6] hasktorch, the Haskell bindings to Torch : https://github.com/hasktorch/hasktorch [7] deanie , probabilistic programming language : https://github.com/jtobin/deanie [8] "Free for DSLs, co-free for interpreters" http://dlaing.org/cofun/posts/free_and_cofree.html [9] State of the Haskell ecosystem : https://github.com/Gabriel439/post-rfc/blob/master/sotu.md