- Haskell-Cafe - Haskell.org

Thread-safe GHC
by Corentin Dupont 31 Aug '15

31 Aug '15

Hello, I am wondering if GHC itself is thread-safe. I am using Hint, and it reports that GHC is not thread-safe, and that I can't safely run two instances of the interpreter simultaneously. Is that still the case? Thanks! Corentin ---------- Forwarded message ---------- From: Daniel Gorín <jcpetruzza(a)gmail.com> Date: Thu, Aug 27, 2015 at 5:09 PM Subject: Re: Thread-safe Hint To: Corentin Dupont <corentin.dupont(a)gmail.com> Hi Corentin, sorry for the late reply. Until relatively recently, the problem was still on. But I too remember seeing something related to this issue being fixed (iirc, the problem was the runtime linker, which used global state), so perhaps it is already fixed in 7.10. If you can verify this, it shouldn’t be hard to show the error message only on old versions of ghc. I’ll be away for a couple of weeks, but if you want to look into this and send a patch, I’ll merge it when I return. Cheers, Daniel > On 24 Aug 2015, at 10:43 am, Corentin Dupont <corentin.dupont(a)gmail.com> wrote: > > Hello Daniel, > I noticed the following message in Hint: > This version of GHC is not thread-safe,can't safely run two instances of the interpreter simultaneously. > > Is it still the case with recent versions of GHC? > It would be neat to be able to launch several instances of the interpreter. In my game Nomyx I have several "match-up" going on and having one instance of the interpreter would be nicer. Otherwise I am obliged to reset the interpret each time I want to interpret something, which is time consuming (2-3 seconds). > > Thanks, > C

1 0

Re: [Haskell-cafe] Discussion: The CLOEXEC problem
by Donn Cave 31 Aug '15

31 Aug '15

Quoth David Turner <dave.c.turner(a)gmail.com>, > Could you be a bit more specific? Which bits of pre-existing software > didn't have a FD_CLOEXEC bit and would be broken by this proposal? Well, of course to be precise, the bit's always there, it's just normally not set - that's the normal environment that anything written up to now would expect. And of course, anything that depends on a GHC-opened file to stay open over an exec would be broken. I can't enumerate the software that meets that criterion. > Since Python recently decided to go through this exact transition, their > experience should be instructive. Do you know if there was negative fallout > from PEP 0466? I gave up on Python a long time ago and don't follow what goes on. If recently means less than a decade or so, though, it's not much to go on. If the problem addressed by the O_CLOEXEC proposal is obscure, the problems it may create are even more so - I'll certainly concede that - and it could take a lot of experience before those problems would be well known enough to show up if you went looking for them. > When thinking about FDs from outside the Haskell runtime (whether inherited > or simply opened in an external library), can you give an example of a case > where such a FD causes a problem if inherited and yet cannot be set as > FD_CLOEXEC at source? Sorry, I'm confused here. Files opened within GHC and externally have equal potential to work as intended or to cause problems, it seems to me. If we infer from the proposal that GHC-opened files with CLOEXEC unset may cause a problem, then it follows that other files with CLOEXEC unset also may cause the same problem. The proposal addresses only the former, and not the latter, and only for normal files - while the ordinary solution, as implemented in UNIX popen(3), deals with all - pipes, sockets, etc. Donn

6 10

Idiomatic way to parse bytecodes
by Gautier DI FOLCO 31 Aug '15

31 Aug '15

Hello all, I have some bytecodes to parse and I look for the most idiomatic way to do it. It's mostly JVM Bytecode, long story short, I have some embedded JVM Bytecodes: * one part is specification-defined, from 0x00 to 0xB8, differ from the regular JVM Bytecode * and the other part is proprietary and changes often I have two main goals: * print the signification of each bytecodes (a more explicit one than the hexadecimal value of the bytecode) * Do some analysis on it, spot patterns, apply markov model on it and so on. The issue with JVM Bytecode in general is that each bytecode hasn't been made equal. Some of them requires parameters, some of them modify the length of the next bytecode, etc. I'm not looking for a way to parse bytecode quickly, or for a library doing it for me. I look for a simple/composable/idiomatic way to do it. So if you have any thoughts, papers, articles or even code snippets (part of a library of not) that is done in the Haskell-way, please, share it :) Thanks in advance for your help.

4 4

Haskell Engineer Needed for Machine Learning Startup
by Charles Weitzer 30 Aug '15

30 Aug '15

Voleon Capital Management LP is a startup quantitative hedge fund located in Berkeley, California. We would like to hire a senior software engineer with strong functional programming experience as soon as possible. Voleon's founders previously worked together at one of the most successful quantitative hedge funds in the world. Our CEO has a PhD in Computer Science from Stanford and has been CEO and founder of a successful Internet infrastructure startup. Our Chief Investment Officer has a PhD in Statistics from Berkeley. Voleon's team includes PhD's from leading departments in statistics, computer science, and mathematics. We have made several unpublished advances in the field of machine learning and in other areas as well. Here is our formal job description: ********************************************************** * Senior Software Engineer * Technology-driven investment firm employing cutting-edge statistical machine learning techniques seeks an exceptionally capable software engineer. You will architect and implement new production trading systems, machine learning infrastructure, data integration pipelines, and large-scale storage systems. The firm researches and deploys systematic trading strategies designed to generate attractive returns without being dependent on the performance of the overall market. Join a team of under 30 people that includes a Berkeley statistics professor as well as over ten PhD's from Berkeley, Chicago, CMU, MIT, Princeton, Stanford, and UCLA, led by the founder and CEO of a successful Internet infrastructure technology firm. The firm's offices are walking distance from BART and the UC Berkeley campus in downtown Berkeley, California. We have a casual and collegial office environment, catered lunches, and competitive benefits packages. We seek candidates with a proven track record of writing correct, well-designed software, solving hard problems, and delivering complex projects on time. You should preferably have experience designing and implementing fault-tolerant distributed systems. Experience with building large-scale data infrastructure, stream processing systems, or latency-sensitive programs is a bonus. We are growing rapidly. Willingness to take initiative and a gritty determination to productize are essential. Required experience: - strong experience developing in a functional programming environment (Haskell, Erlang, others). - developing with C/C++/Python/Go in a Linux environment with a focus on performance, concurrency, and correctness. - working in TCP/IP networking, multi-threading, and server development. - working with common Internet protocols (IP, TCP/UDP, SSL/TLS, HTTP, SNMP, etc.). - architecting and designing highly available systems. - architecting and designing large-scale data management infrastructure. - working in large codebases and building modular, manageable code. Preferred experience.: - debugging and performance profiling, including the use of tools such as strace, valgrind, gdb, tcpdump, etc. - working with build and test automation tools. - working with well-defined change management processes. - diagnosing RDBMS performance problems, exploiting indexing, using EXPLAIN PLAN, optimizing at the code layer, etc. - working with messaging queues (RabbitMQ, Redis, etc.) as well as distributed caching systems. Interest in financial applications is essential, but experience in finance is not a primary factor in our hiring. Benefits and compensation are highly competitive. ********************************************************** The above job description is just a starting point in terms of possible duties and seniority. We can be very flexible for the right person. If you are interested or if you know of anyone who might be interested, let us know the best way to get in touch and we can discuss details. Thank you, Charles Weitzer Senior Recruiter Voleon Capital Management LP Charles(a)Voleon.com<mailto:Charles@Voleon.com> Office: 510.704.9870 x 7012 Mobile: (510) 558-9182 www.Voleon.com<http://www.voleon.com/>

1 0

Re: [Haskell-cafe] Why Haskell is beautiful to the novice
by Olaf Klinke 30 Aug '15

30 Aug '15

Dear cafe inhabitants, first of all thanks for all the elaborate answers. I did not imagine to trigger such an avalanche of opinions. Let me try to steer away from the Python/Haskell comparison (as comparison is what I wanted to avoid) and collect some points I found important in the past discussion. 1. There is a difference between teaching how the machines we call computers work and how computers are used to solve problems. What is computer science? The science about computers or science that can be executed on a computer? Both subjects are beautiful, in different ways. Haskell, I used to think, was designed explicitly to abstract away from how its programs are actually run on the hardware, to the point where one can not even tell (and care) what order certain computations are executed in, or whether they are executed at all. Therefore Haskell is certainly not the best language to teach about the mechanics of computing. Similarly, there are practical problems that may be more difficult to express in a functional language, because the problem may be already of a sequential, mutable, effect-full nature (e.g. writing an adventure game). 2. For teaching purposes, the remark that kids want something they can show off to others should not not be under-estimated. In order to quickly obtain showable results, one might choose a toy language (DSL?) or program robots. For this reason I considered HTML as a first introduction to syntax, even if it is not a programming language as such. But there exist syntax checkers and one can create and view HTML on every modern device. By the way, is there a live CD (not DVD) image that contains the Haskell platform and some other languages? 3. One skill/revelation that I want to install in the students of both math and CS, and which I believe Haskell is more suitable for than imperative languages, is the fact that in mathematics and computer science there are no hidden layers/aspects, no mysteries. Everything is composed of smaller, simpler parts. And if one wishes, one can unwrap one layer and look inside. How deeply one wishes to look inside is a question of audience, then. The layering might consist of recursion, classes, objects. It does not matter. The important point is that the path to a solution must descend the layers to the simple parts. This is in contrast with arts and humanities, where one can study the same piece of art, the same novel, several times in a lifetime and each time discover new things, and probably still miss others which require expert education. Mergesort, once grasped, is there in all its completeness, simplicity and beauty, there is no further interpretation. As others have remarked in this discussion, the `no mysteries' only applies to the denotational semantics, but not the operational semantics in case of Haskell. For assembler it might very well be the other way around. Thanks, Olaf

2 2

Haskell related talks at Curry On!
by Jurriaan Hage 30 Aug '15

30 Aug '15

Dear Haskell-minded, Curry On! in Prague was co-located with Ecoop and had some Haskell-oriented talks. In case you were not there, and had not noticed: video’s of the talks are available on YouTube. Of particular note are Julian Arny’s talk on Servant, url at https://www.youtube.com/watch?v=snOBI8PcbMQ Then there is the entertaining discussion by Thomas Arts of Quviq on awakening industry to the use of QuickCheck (it’s Erlang, I know, but coming from Haskell): https://www.youtube.com/watch?v=Ar0FPW2GSgo and finally my own on domain-specific type error diagnosis in Haskell: https://www.youtube.com/watch?v=bPrM1gONdII And when you’ve seen these, YouTube is bound to suggest many others. enjoy, Jurriaan Hage PS. Curry On! was in fact a great event with many interesting talks. It is going on repeat it seems, so catch it if you can.

1 0

Discussion: The CLOEXEC problem
by Niklas Hambüchen 30 Aug '15

30 Aug '15

Hello Cafe, I would like to point out a problem common to all programming languages, and that Haskell hasn't addressed yet while other languages have. It is about what happens to file descriptors when the `exec()` syscall is used (whenever you `readProcess`, `createProcess`, `system`, use any form of `popen()`, Shake's `cmd` etc.). (A Markdown-rendered version containing most of this email can be found at https://github.com/ndmitchell/shake/issues/253.) Take the following function f :: IO () f = do inSomeTemporaryDirectory $ do BS.writeFile "mybinary" binaryContents setPermissions "mybinary" (setOwnerExecutable True emptyPermissions) _ <- readProcess "./mybinary" [] "" return () If this is happening in parallel, e.g. using, forkIO f >> forkIO f >> forkIO f >> threadDelay 5000000` then on Linux the `readProcess` might often fail wit the error message mybinary: Text file busy This error means "Cannot execute the program 'mybinary' because it is open for writing by some process". How can this happen, given that we're writing all `mybinary` files in completely separate temporary directories, and given that `BS.writeFile` guarantees to close the file handle / file descriptor (`Fd`) before it returns? The answer is that by default, child processes on Unix (`fork()+exec()`) inherit all open file descriptors of the parent process. An ordering that leads to the problematic case could be: * Thread 1 writes its file completely (opens and closes an Fd 1) * Thread 2 starts writing its file (Fd 2 open for writing) * Thread 1 executes "myBinary" (which calls `fork()` and `exec()`). Fd 2 is inherited by the child process * Thread 2 finishes writing (closes its Fd 2) * Thread 2 executes "myBinary", which fails with `Text file busy` because an Fd is still open to it in the child of Process 1 The scope of this program is quite general unfortunately: It will happen for any program that uses parallel threads, and that runs two or more external processes at some time. It cannot be fixed by the part that starts the external process (e.g. you can't write a reliable `readProcess` function that doesn't have this problem, since the problem is rooted in the Fds, and there is no version of `exec()` that doesn't inherit parent Fds). This problem is a general problem in C on Unix, and was discovered quite late. Naive solutions to this use `fcntl`, e.g. `fcntl(fd, F_SETFD, FD_CLOEXEC)`: http://stackoverflow.com/questions/6125068/what-does-the-fd-cloexec-fcntl-f… which is the equivalent of Haskell's `setFdOption` to set the `CLOEXEC` flag to all Fds before `exec()`ing. Fds with this flag are not inherited by `exec()`ed child processes. However, these solutions are racy in multi-threaded programs (such as typical Haskell programs), where an `exec()` made by some thread can fall just in between the `int fd = open(...); exec(...)` of some other thread. For this reason, the `O_CLOEXEC` flag was added in Linux 2.6.23, see e.g. `man 2 open` http://man7.org/linux/man-pages/man2/open.2.html to the `open()` syscall to atomically open a file and set the Fd to CLOEXEC in a single step. This flag is not the default in Haskell - but maybe it should be. Other languages set it by default, for example Python. See PEP-433: https://www.python.org/dev/peps/pep-0433/ and the newer PEP-446: https://www.python.org/dev/peps/pep-0446/ for a very good description of the situation. Python >= 3.2 closes open Fds *after* the `exec()` when performed with its `subprocess` module. Python 3.4 uses O_CLOEXEC by default on all Fds opened by Python. It is also noted that "The programming languages Go, Perl and Ruby make newly created file descriptors non-inheritable by default: since Go 1.0 (2009), Perl 1.0 (1987) and Ruby 2.0 (2013)": https://www.python.org/dev/peps/pep-0446/#related-work A work-around for Haskell is to use `O_CLOEXEC` explicitly, as in this example module `System/Posix/IO/ExecSafe.hsc`: https://gist.github.com/nh2/4932ecf5ca919659ae51 Then we can implement a safe version of `BS.writeFile`: https://gist.github.com/nh2/4932ecf5ca919659ae51 Using this form of `writeFileExecSafe` helps in cases when your program is very small, when you can change all the code and you don't use any libraries that open files. However, this is a very rare case, and not a real solution. All multi-threaded Haskell programs that write and execute files will inherently trigger the `Text file busy` problem. We need to discuss what to do about this. Let us run this discussion on haskell-cafe and move to the libraries@ mailing list once we've got some ideas and opinions. My personal stance is that we should follow Python's example, and all functions in our standard libraries should open files with the O_CLOEXEC flag set. Niklas

6 12

Final CFP: FLOPS 2016, International Symposium on Functional and Logic Programming
by Oleg 30 Aug '15

30 Aug '15

NEW: revised submission deadlines (Sep 21 for abstracts, Sep 25 for papers) FLOPS 2016: March 3-6, 2016, Kochi, Japan Final Call For Papers http://www.info.kochi-tech.ac.jp/FLOPS2016/ Writing down detailed computational steps is not the only way of programming. The alternative, being used increasingly in practice, is to start by writing down the desired properties of the result. The computational steps are then (semi-)automatically derived from these higher-level specifications. Examples of this declarative style include functional and logic programming, program transformation and re-writing, and extracting programs from proofs of their correctness. FLOPS aims to bring together practitioners, researchers and implementors of the declarative programming, to discuss mutually interesting results and common problems: theoretical advances, their implementations in language systems and tools, and applications of these systems in practice. The scope includes all aspects of the design, semantics, theory, applications, implementations, and teaching of declarative programming. FLOPS specifically aims to promote cross-fertilization between theory and practice and among different styles of declarative programming. Scope FLOPS solicits original papers in all areas of the declarative programming: * functional, logic, functional-logic programming, re-writing systems, formal methods and model checking, program transformations and program refinements, developing programs with the help of theorem provers or SAT/SMT solvers; * foundations, language design, implementation issues (compilation techniques, memory management, run-time systems), applications and case studies. FLOPS promotes cross-fertilization among different styles of declarative programming. Therefore, submissions must be written to be understandable by the wide audience of declarative programmers and researchers. Submission of system descriptions and declarative pearls are especially encouraged. Submissions should fall into one of the following categories: * Regular research papers: they should describe new results and will be judged on originality, correctness, and significance. * System descriptions: they should contain a link to a working system and will be judged on originality, usefulness, and design. * Declarative pearls: new and excellent declarative programs or theories with illustrative applications. System descriptions and declarative pearls must be explicitly marked as such in the title. Submissions must be unpublished and not submitted for publication elsewhere. Work that already appeared in unpublished or informally published workshops proceedings may be submitted. See also ACM SIGPLAN Republication Policy. The proceedings will be published by Springer International Publishing in the Lecture Notes in Computer Science (LNCS) series, as a printed volume as well as online in the digital library SpringerLink. Post-proceedings: The authors of 4-7 best papers will be invited to submit the extended version of their FLOPS paper to a special issue of the journal Science of Computer Programming (SCP). Important dates Monday, September 21, 2015 (any time zone): Abstract Submission Friday, September 25, 2015 (any time zone): Submission deadline (FIRM) Monday, November 16, 2015: Author notification March 3-6, 2016: FLOPS Symposium March 7-9, 2016: PPL Workshop Invited Talks - Kazunori UEDA (Waseda University) The exciting time and hard-won lessons of the Fifth Generation Computer Project - Atze Dijkstra (Utrecht University) UHC: Coping with Compiler Complexity Submission Submissions must be written in English and can be up to 15 pages long including references, though pearls are typically shorter. The formatting has to conform to Springer's guidelines. Regular research papers should be supported by proofs and/or experimental results. In case of lack of space, this supporting information should be made accessible otherwise (e.g., a link to a Web page, or an appendix). Papers should be submitted electronically at https://easychair.org/conferences/?conf=flops2016 Program Committee Andreas Abel Gothenburg University, Sweden Lindsay Errington USA Makoto Hamana Gunma University, Japan Michael Hanus CAU Kiel, Germany Jacob Howe City University London, UK Makoto Kanazawa National Institute of Informatics, Japan Andy King University of Kent, UK (PC Co-Chair) Oleg Kiselyov Tohoku University, Japan (PC Co-Chair) Hsiang-Shang Ko National Institute of Informatics, Japan Julia Lawall Inria-Whisper, France Andres Loeh Well-Typed LLP, UK Anil Madhavapeddy Cambridge University, UK Jeff Polakow USA Marc Pouzet Ecole normale superieure, France Vitor Santos Costa Universidade do Porto, Portugal Tom Schrijvers KU Leuven, Belgium Zoltan Somogyi Australia Alwen Tiu Nanyang Technological University, Singapore Sam Tobin-Hochstadt Indiana University, USA Hongwei Xi Boston University, USA Neng-Fa Zhou CUNY Brooklyn College and Graduate Center, USA Organizers Andy King University of Kent, UK (PC Co-Chair) Oleg Kiselyov Tohoku University, Japan (PC Co-Chair) Yukiyoshi Kameyama University of Tsukuba, Japan (General Chair) Kiminori Matsuzaki Kochi University of Technology, Japan (Local Chair) flops2016 at logic.cs.tsukuba.ac dot jp

1 0

Question, re: type class syntax.
by David Banas 30 Aug '15

30 Aug '15

Hi all, I can’t figure out why this is invalid: class (Show a, Num a) => LogTree2 a where evaluator :: (LogTree2 a) t => t -> SizedVector a -> SizedVector a error| ‘LogTree2’ is applied to too many type arguments In the type ‘(LogTree2 a) t => t -> SizedVector a -> SizedVector a’ In the class declaration for ‘LogTree2’ Can anyone help me understand? Thanks, -db

3 5

General Functor and Traversable
by M Farkas-Dyck 29 Aug '15

29 Aug '15

Consider this: class (Category c, Category d) => Functor c d f where map :: c a b -> d (f a) (f b) Clearly mapM of Traversable is simply map :: Kleisli m a b -> Kleisli m (f a) (f b). But what is traverse? can it be so defined? I am thinking no, as Kleisli composition is not defined in general, but I know very little category theory so I may easily be wrong.

2 2