August 2015 - Haskell-Cafe

Haskell Engineer Needed for Machine Learning Startup
by Charles Weitzer 30 Aug '15

30 Aug '15

Voleon Capital Management LP is a startup quantitative hedge fund located in Berkeley, California. We would like to hire a senior software engineer with strong functional programming experience as soon as possible. Voleon's founders previously worked together at one of the most successful quantitative hedge funds in the world. Our CEO has a PhD in Computer Science from Stanford and has been CEO and founder of a successful Internet infrastructure startup. Our Chief Investment Officer has a PhD in Statistics from Berkeley. Voleon's team includes PhD's from leading departments in statistics, computer science, and mathematics. We have made several unpublished advances in the field of machine learning and in other areas as well. Here is our formal job description: ********************************************************** * Senior Software Engineer * Technology-driven investment firm employing cutting-edge statistical machine learning techniques seeks an exceptionally capable software engineer. You will architect and implement new production trading systems, machine learning infrastructure, data integration pipelines, and large-scale storage systems. The firm researches and deploys systematic trading strategies designed to generate attractive returns without being dependent on the performance of the overall market. Join a team of under 30 people that includes a Berkeley statistics professor as well as over ten PhD's from Berkeley, Chicago, CMU, MIT, Princeton, Stanford, and UCLA, led by the founder and CEO of a successful Internet infrastructure technology firm. The firm's offices are walking distance from BART and the UC Berkeley campus in downtown Berkeley, California. We have a casual and collegial office environment, catered lunches, and competitive benefits packages. We seek candidates with a proven track record of writing correct, well-designed software, solving hard problems, and delivering complex projects on time. You should preferably have experience designing and implementing fault-tolerant distributed systems. Experience with building large-scale data infrastructure, stream processing systems, or latency-sensitive programs is a bonus. We are growing rapidly. Willingness to take initiative and a gritty determination to productize are essential. Required experience: - strong experience developing in a functional programming environment (Haskell, Erlang, others). - developing with C/C++/Python/Go in a Linux environment with a focus on performance, concurrency, and correctness. - working in TCP/IP networking, multi-threading, and server development. - working with common Internet protocols (IP, TCP/UDP, SSL/TLS, HTTP, SNMP, etc.). - architecting and designing highly available systems. - architecting and designing large-scale data management infrastructure. - working in large codebases and building modular, manageable code. Preferred experience.: - debugging and performance profiling, including the use of tools such as strace, valgrind, gdb, tcpdump, etc. - working with build and test automation tools. - working with well-defined change management processes. - diagnosing RDBMS performance problems, exploiting indexing, using EXPLAIN PLAN, optimizing at the code layer, etc. - working with messaging queues (RabbitMQ, Redis, etc.) as well as distributed caching systems. Interest in financial applications is essential, but experience in finance is not a primary factor in our hiring. Benefits and compensation are highly competitive. ********************************************************** The above job description is just a starting point in terms of possible duties and seniority. We can be very flexible for the right person. If you are interested or if you know of anyone who might be interested, let us know the best way to get in touch and we can discuss details. Thank you, Charles Weitzer Senior Recruiter Voleon Capital Management LP Charles(a)Voleon.com<mailto:Charles@Voleon.com> Office: 510.704.9870 x 7012 Mobile: (510) 558-9182 www.Voleon.com<http://www.voleon.com/>

1 0

Re: [Haskell-cafe] Why Haskell is beautiful to the novice
by Olaf Klinke 30 Aug '15

30 Aug '15

Dear cafe inhabitants, first of all thanks for all the elaborate answers. I did not imagine to trigger such an avalanche of opinions. Let me try to steer away from the Python/Haskell comparison (as comparison is what I wanted to avoid) and collect some points I found important in the past discussion. 1. There is a difference between teaching how the machines we call computers work and how computers are used to solve problems. What is computer science? The science about computers or science that can be executed on a computer? Both subjects are beautiful, in different ways. Haskell, I used to think, was designed explicitly to abstract away from how its programs are actually run on the hardware, to the point where one can not even tell (and care) what order certain computations are executed in, or whether they are executed at all. Therefore Haskell is certainly not the best language to teach about the mechanics of computing. Similarly, there are practical problems that may be more difficult to express in a functional language, because the problem may be already of a sequential, mutable, effect-full nature (e.g. writing an adventure game). 2. For teaching purposes, the remark that kids want something they can show off to others should not not be under-estimated. In order to quickly obtain showable results, one might choose a toy language (DSL?) or program robots. For this reason I considered HTML as a first introduction to syntax, even if it is not a programming language as such. But there exist syntax checkers and one can create and view HTML on every modern device. By the way, is there a live CD (not DVD) image that contains the Haskell platform and some other languages? 3. One skill/revelation that I want to install in the students of both math and CS, and which I believe Haskell is more suitable for than imperative languages, is the fact that in mathematics and computer science there are no hidden layers/aspects, no mysteries. Everything is composed of smaller, simpler parts. And if one wishes, one can unwrap one layer and look inside. How deeply one wishes to look inside is a question of audience, then. The layering might consist of recursion, classes, objects. It does not matter. The important point is that the path to a solution must descend the layers to the simple parts. This is in contrast with arts and humanities, where one can study the same piece of art, the same novel, several times in a lifetime and each time discover new things, and probably still miss others which require expert education. Mergesort, once grasped, is there in all its completeness, simplicity and beauty, there is no further interpretation. As others have remarked in this discussion, the `no mysteries' only applies to the denotational semantics, but not the operational semantics in case of Haskell. For assembler it might very well be the other way around. Thanks, Olaf

2 2

Haskell related talks at Curry On!
by Jurriaan Hage 30 Aug '15

30 Aug '15

Dear Haskell-minded, Curry On! in Prague was co-located with Ecoop and had some Haskell-oriented talks. In case you were not there, and had not noticed: video’s of the talks are available on YouTube. Of particular note are Julian Arny’s talk on Servant, url at https://www.youtube.com/watch?v=snOBI8PcbMQ Then there is the entertaining discussion by Thomas Arts of Quviq on awakening industry to the use of QuickCheck (it’s Erlang, I know, but coming from Haskell): https://www.youtube.com/watch?v=Ar0FPW2GSgo and finally my own on domain-specific type error diagnosis in Haskell: https://www.youtube.com/watch?v=bPrM1gONdII And when you’ve seen these, YouTube is bound to suggest many others. enjoy, Jurriaan Hage PS. Curry On! was in fact a great event with many interesting talks. It is going on repeat it seems, so catch it if you can.

1 0

Discussion: The CLOEXEC problem
by Niklas Hambüchen 30 Aug '15

30 Aug '15

Hello Cafe, I would like to point out a problem common to all programming languages, and that Haskell hasn't addressed yet while other languages have. It is about what happens to file descriptors when the `exec()` syscall is used (whenever you `readProcess`, `createProcess`, `system`, use any form of `popen()`, Shake's `cmd` etc.). (A Markdown-rendered version containing most of this email can be found at https://github.com/ndmitchell/shake/issues/253.) Take the following function f :: IO () f = do inSomeTemporaryDirectory $ do BS.writeFile "mybinary" binaryContents setPermissions "mybinary" (setOwnerExecutable True emptyPermissions) _ <- readProcess "./mybinary" [] "" return () If this is happening in parallel, e.g. using, forkIO f >> forkIO f >> forkIO f >> threadDelay 5000000` then on Linux the `readProcess` might often fail wit the error message mybinary: Text file busy This error means "Cannot execute the program 'mybinary' because it is open for writing by some process". How can this happen, given that we're writing all `mybinary` files in completely separate temporary directories, and given that `BS.writeFile` guarantees to close the file handle / file descriptor (`Fd`) before it returns? The answer is that by default, child processes on Unix (`fork()+exec()`) inherit all open file descriptors of the parent process. An ordering that leads to the problematic case could be: * Thread 1 writes its file completely (opens and closes an Fd 1) * Thread 2 starts writing its file (Fd 2 open for writing) * Thread 1 executes "myBinary" (which calls `fork()` and `exec()`). Fd 2 is inherited by the child process * Thread 2 finishes writing (closes its Fd 2) * Thread 2 executes "myBinary", which fails with `Text file busy` because an Fd is still open to it in the child of Process 1 The scope of this program is quite general unfortunately: It will happen for any program that uses parallel threads, and that runs two or more external processes at some time. It cannot be fixed by the part that starts the external process (e.g. you can't write a reliable `readProcess` function that doesn't have this problem, since the problem is rooted in the Fds, and there is no version of `exec()` that doesn't inherit parent Fds). This problem is a general problem in C on Unix, and was discovered quite late. Naive solutions to this use `fcntl`, e.g. `fcntl(fd, F_SETFD, FD_CLOEXEC)`: http://stackoverflow.com/questions/6125068/what-does-the-fd-cloexec-fcntl-f… which is the equivalent of Haskell's `setFdOption` to set the `CLOEXEC` flag to all Fds before `exec()`ing. Fds with this flag are not inherited by `exec()`ed child processes. However, these solutions are racy in multi-threaded programs (such as typical Haskell programs), where an `exec()` made by some thread can fall just in between the `int fd = open(...); exec(...)` of some other thread. For this reason, the `O_CLOEXEC` flag was added in Linux 2.6.23, see e.g. `man 2 open` http://man7.org/linux/man-pages/man2/open.2.html to the `open()` syscall to atomically open a file and set the Fd to CLOEXEC in a single step. This flag is not the default in Haskell - but maybe it should be. Other languages set it by default, for example Python. See PEP-433: https://www.python.org/dev/peps/pep-0433/ and the newer PEP-446: https://www.python.org/dev/peps/pep-0446/ for a very good description of the situation. Python >= 3.2 closes open Fds *after* the `exec()` when performed with its `subprocess` module. Python 3.4 uses O_CLOEXEC by default on all Fds opened by Python. It is also noted that "The programming languages Go, Perl and Ruby make newly created file descriptors non-inheritable by default: since Go 1.0 (2009), Perl 1.0 (1987) and Ruby 2.0 (2013)": https://www.python.org/dev/peps/pep-0446/#related-work A work-around for Haskell is to use `O_CLOEXEC` explicitly, as in this example module `System/Posix/IO/ExecSafe.hsc`: https://gist.github.com/nh2/4932ecf5ca919659ae51 Then we can implement a safe version of `BS.writeFile`: https://gist.github.com/nh2/4932ecf5ca919659ae51 Using this form of `writeFileExecSafe` helps in cases when your program is very small, when you can change all the code and you don't use any libraries that open files. However, this is a very rare case, and not a real solution. All multi-threaded Haskell programs that write and execute files will inherently trigger the `Text file busy` problem. We need to discuss what to do about this. Let us run this discussion on haskell-cafe and move to the libraries@ mailing list once we've got some ideas and opinions. My personal stance is that we should follow Python's example, and all functions in our standard libraries should open files with the O_CLOEXEC flag set. Niklas

6 12

Final CFP: FLOPS 2016, International Symposium on Functional and Logic Programming
by Oleg 30 Aug '15

30 Aug '15

NEW: revised submission deadlines (Sep 21 for abstracts, Sep 25 for papers) FLOPS 2016: March 3-6, 2016, Kochi, Japan Final Call For Papers http://www.info.kochi-tech.ac.jp/FLOPS2016/ Writing down detailed computational steps is not the only way of programming. The alternative, being used increasingly in practice, is to start by writing down the desired properties of the result. The computational steps are then (semi-)automatically derived from these higher-level specifications. Examples of this declarative style include functional and logic programming, program transformation and re-writing, and extracting programs from proofs of their correctness. FLOPS aims to bring together practitioners, researchers and implementors of the declarative programming, to discuss mutually interesting results and common problems: theoretical advances, their implementations in language systems and tools, and applications of these systems in practice. The scope includes all aspects of the design, semantics, theory, applications, implementations, and teaching of declarative programming. FLOPS specifically aims to promote cross-fertilization between theory and practice and among different styles of declarative programming. Scope FLOPS solicits original papers in all areas of the declarative programming: * functional, logic, functional-logic programming, re-writing systems, formal methods and model checking, program transformations and program refinements, developing programs with the help of theorem provers or SAT/SMT solvers; * foundations, language design, implementation issues (compilation techniques, memory management, run-time systems), applications and case studies. FLOPS promotes cross-fertilization among different styles of declarative programming. Therefore, submissions must be written to be understandable by the wide audience of declarative programmers and researchers. Submission of system descriptions and declarative pearls are especially encouraged. Submissions should fall into one of the following categories: * Regular research papers: they should describe new results and will be judged on originality, correctness, and significance. * System descriptions: they should contain a link to a working system and will be judged on originality, usefulness, and design. * Declarative pearls: new and excellent declarative programs or theories with illustrative applications. System descriptions and declarative pearls must be explicitly marked as such in the title. Submissions must be unpublished and not submitted for publication elsewhere. Work that already appeared in unpublished or informally published workshops proceedings may be submitted. See also ACM SIGPLAN Republication Policy. The proceedings will be published by Springer International Publishing in the Lecture Notes in Computer Science (LNCS) series, as a printed volume as well as online in the digital library SpringerLink. Post-proceedings: The authors of 4-7 best papers will be invited to submit the extended version of their FLOPS paper to a special issue of the journal Science of Computer Programming (SCP). Important dates Monday, September 21, 2015 (any time zone): Abstract Submission Friday, September 25, 2015 (any time zone): Submission deadline (FIRM) Monday, November 16, 2015: Author notification March 3-6, 2016: FLOPS Symposium March 7-9, 2016: PPL Workshop Invited Talks - Kazunori UEDA (Waseda University) The exciting time and hard-won lessons of the Fifth Generation Computer Project - Atze Dijkstra (Utrecht University) UHC: Coping with Compiler Complexity Submission Submissions must be written in English and can be up to 15 pages long including references, though pearls are typically shorter. The formatting has to conform to Springer's guidelines. Regular research papers should be supported by proofs and/or experimental results. In case of lack of space, this supporting information should be made accessible otherwise (e.g., a link to a Web page, or an appendix). Papers should be submitted electronically at https://easychair.org/conferences/?conf=flops2016 Program Committee Andreas Abel Gothenburg University, Sweden Lindsay Errington USA Makoto Hamana Gunma University, Japan Michael Hanus CAU Kiel, Germany Jacob Howe City University London, UK Makoto Kanazawa National Institute of Informatics, Japan Andy King University of Kent, UK (PC Co-Chair) Oleg Kiselyov Tohoku University, Japan (PC Co-Chair) Hsiang-Shang Ko National Institute of Informatics, Japan Julia Lawall Inria-Whisper, France Andres Loeh Well-Typed LLP, UK Anil Madhavapeddy Cambridge University, UK Jeff Polakow USA Marc Pouzet Ecole normale superieure, France Vitor Santos Costa Universidade do Porto, Portugal Tom Schrijvers KU Leuven, Belgium Zoltan Somogyi Australia Alwen Tiu Nanyang Technological University, Singapore Sam Tobin-Hochstadt Indiana University, USA Hongwei Xi Boston University, USA Neng-Fa Zhou CUNY Brooklyn College and Graduate Center, USA Organizers Andy King University of Kent, UK (PC Co-Chair) Oleg Kiselyov Tohoku University, Japan (PC Co-Chair) Yukiyoshi Kameyama University of Tsukuba, Japan (General Chair) Kiminori Matsuzaki Kochi University of Technology, Japan (Local Chair) flops2016 at logic.cs.tsukuba.ac dot jp

1 0

Question, re: type class syntax.
by David Banas 30 Aug '15

30 Aug '15

Hi all, I can’t figure out why this is invalid: class (Show a, Num a) => LogTree2 a where evaluator :: (LogTree2 a) t => t -> SizedVector a -> SizedVector a error| ‘LogTree2’ is applied to too many type arguments In the type ‘(LogTree2 a) t => t -> SizedVector a -> SizedVector a’ In the class declaration for ‘LogTree2’ Can anyone help me understand? Thanks, -db

3 5

General Functor and Traversable
by M Farkas-Dyck 29 Aug '15

29 Aug '15

Consider this: class (Category c, Category d) => Functor c d f where map :: c a b -> d (f a) (f b) Clearly mapM of Traversable is simply map :: Kleisli m a b -> Kleisli m (f a) (f b). But what is traverse? can it be so defined? I am thinking no, as Kleisli composition is not defined in general, but I know very little category theory so I may easily be wrong.

2 2

Re: [Haskell-cafe] List of nitpicks
by Johannes Waldmann 28 Aug '15

28 Aug '15

Re: http://repetae.net/recent/out/classalias.html Prelude> :set -XConstraintKinds Prelude> class C a; class D a; type E a = (C a, D a); instance E a => C (Maybe a) https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/constraint-… - J.W.

2 1

[ANN] dejafu-0.1.0.0: Overloadable primitives for testable, potentially non-deterministic, concurrency.
by Michael Walker 28 Aug '15

28 Aug '15

Hi cafe, For the past few months I've been working on a library for concurrency testing in Haskell, which I'll be giving a talk on at the Haskell Symposium next week. Now it's on Hackage! --- Déjà Fu is a library for developing and testing concurrent Haskell programs, it provides a typeclass-abstraction over GHC’s regular concurrency API, allowing the concrete implementation to be swapped out. Why do we need this? Well, concurrency is really hard to get right. Empirical studies have found that many real-world concurrency bugs can be exhibited with small test cases using as few as two threads: so it’s not just big concurrent programs that are hard, small ones are too. We as programmers just don’t seem to have a very good intuition for traditional threads-and-shared-memory-style concurrency. The typical approach to testing concurrent programs is to just run them lots of times, but that doesn’t provide any hard coverage guarantees, and then we need to wonder: how many runs do we need? Fortunately, there has been a lot of research into testing concurrency in the past several years. Systematic concurrency testing is an approach where the source of nondeterminism, the actual scheduler, is swapped out for one under the control of the testing framework. This allows possible schedules to be systematically explored, giving real coverage guarantees for our tests. This is a library implementing systematic concurrency testing. It provides two typeclasses, MonadConc to abstract over much of Control.Concurrent and related modules, and MonadSTM, to similarly abstract over Control.Monad.STM. ## How to use it: If you’re not making use of any IO in your code other than for concurrency, the transition to using MonadConc and MonadSTM will probably just be a textual substitution: - IO is replaced with MonadConc m => m - STM with MonadSTM m => m - *IORef with *CRef - *MVar with *CVar - *TVar with *CTVar - Most other things have the same name, and so can be replaced by just swapping imports around. If you are using other IO, you will need a gentle sprinkling of MonadIO and liftIO in your code as well. ## Is this really just a drop-in replacement for IO/STM? That’s the idea, yes. More specifically, the IO instance of MonadConc and the STM instance of MonadSTM just use the regular IO and STM functions, and so should have no noticeable change in behaviour, except for CRef, the IORef equivalent, where modifyCRef behaves like atomicModifyIORef, not modifyIORef. There are some other differences which can lead to incorrect results when testing, but which should not affect code when used in an IO or STM context. Specifically: Control.Monad.Conc.Class.getNumCapabilities can lie to encourage more concurrency when testing; and Control.Exception.catch can catch exceptions from pure code, but Control.Monad.Conc.Class.catch can’t (except for the IO instance). --- There is a blog post with references to prior posts and related papers here: http://www.barrucadu.co.uk/posts/2015-08-27-announce-dejafu.html -- Michael Walker (http://www.barrucadu.co.uk)

2 2

Re: [Haskell-cafe] Why Haskell is beautiful to the novice
by Nicola Gigante 28 Aug '15

28 Aug '15

> Il giorno 28/ago/2015, alle ore 15:37, Torsten Otto <torsten.otto(a)hamburg.de> ha scritto: > > What an interesting topic to discuss! > >> Am 28.08.2015 um 12:12 schrieb Nicola Gigante <nicola.gigante(a)gmail.com>: >> >> Using Haskell (specifically some kind of concepts, e.g. equational reasoning) >> will also make easier for students to see the connection between math >> and programming, making them more motivated to learn math as well. > > Can you elaborate on that? What would you do in class in terms of „equational reasoning“? > It depends on the age of the students and the math they already know. I don’t have specific examples of in-class exercises or things like that. However, the point is that writing Haskell code you can often come up with shorter or more elegant solutions by manipulating terms exactly as done in elementary algebra. Starting from a 10lines function and coming up with a simple and readable oneliner is fun and rewarding, and shows one of the possible connections to math, especially if you can make them reason about types, in which case the task becomes very similar to theorem proving than to algebraic manipulations alone. Another possible connection arise when you teach things such as the Monoid typeclass. Even if students have not been exposed to abstract mathematics, they can still appreciate the usefulness of abstracting similar operations (addition in this case) to being able to uniformly reason about apparently different concepts. Abstraction is the core of mathematics, but is also a fundamental concept in programming, of course, so you’re implicitly teaching two things at once. > Regards, > Torsten Greetings, Nicola

1 0