July 2020 - Haskell-Cafe

stack: pandoc fails to build as dependency
by Olaf Klinke 06 Aug '20

06 Aug '20

Dear cafe, I am baffled to learn that stack does not provide builds as reproducible as advertised. I have a project that depends on pandoc. It builds fine on my Debian 9 machine. Then I check out the project on a Debian 8 machine. Downloading and building pandoc-2.7.3 with the latest stack-2.1.3 on the Debian 8 machine runs fine. Now I create a new project, using the same stack.yaml as in the pandoc-2.7.3 package, except now pandoc-2.7.3 is an extra-dep. The project's only module contains nothing but: module Test where import Text.Pandoc Building this project with the same stack results in a linker error: pandoc > Preprocessing executable 'pandoc' for pandoc-2.7.3.. pandoc > Building executable 'pandoc' for pandoc-2.7.3.. pandoc > [1 of 2] Compiling Main pandoc > [2 of 2] Compiling Paths_pandoc pandoc > Linking .stack-work/dist/x86_64-linux/Cabal-2.4.0.1/build/pandoc/pandoc ... pandoc > collect2: error: ld returned 1 exit status pandoc > `gcc' failed in phase `Linker'. (Exit code: 1) This seems to be independent of the LTS version (the stack.yaml in pandoc-2.7.3 says lts-13.17 although lts-13.17 contains pandoc-2.5 and pandoc-2.7.3 is available only in 14.9). Can somebody hint what I am doing wrong? Should I file a bug report? Thanks Olaf

3 3

Best ways to achieve throughput, for large M:N ratio of STM threads, with hot TVar updates?
by YueCompl 06 Aug '20

06 Aug '20

Hello Cafe, I'm working on an in-memory database, in Client/Server mode I just let each connected client submit remote procedure call running in its dedicated lightweight thread, modifying TVars in RAM per its business needs, then in case many clients connected concurrently and trying to insert new data, if they are triggering global index (some TVar) update, the throughput would drop drastically. I reduced the shared state to a simple int counter by TVar, got same symptom. While the parallelism feels okay when there's no hot TVar conflicting, or M is not much greater than N. As an empirical test workload, I have a `+RTS -N10` server process, it handles 10 concurrent clients okay, got ~5x of single thread throughput; but in handling 20 concurrent clients, each of the 10 CPUs can only be driven to ~10% utilization, the throughput seems even worse than single thread. More clients can even drive it thrashing without much progressing. I can understand that pure STM doesn't scale well after reading [1], and I see it suggested [7] attractive and planned future work toward that direction. But I can't find certain libraries or frameworks addressing large M over small N scenarios, [1] experimented with designated N parallelism, and [7] is rather theoretical to my empirical needs. Can you direct me to some available library implementing the methodology proposed in [7] or other ways tackling this problem? I think the most difficult one is that a transaction should commit with global indices (with possibly unique constraints) atomically updated, and rollback with any violation of constraints, i.e. transactions have to cover global states like indices. Other problems seem more trivial than this. Specifically, [7] states: > It must be emphasized that all of the mechanisms we deploy originate, in one form or another, in the database literature from the 70s and 80s. Our contribution is to adapt these techniques to software transactional memory, providing more effective solutions to important STM problems than prior proposals. I wonder any STM based library has simplified those techniques to be composed right away? I don't really want to implement those mechanisms by myself, rebuilding many wheels from scratch. Best regards, Compl [1] Comparing the performance of concurrent linked-list implementations in Haskell https://simonmar.github.io/bib/papers/concurrent-data.pdf <https://simonmar.github.io/bib/papers/concurrent-data.pdf> [7] M. Herlihy and E. Koskinen. Transactional boosting: a methodology for highly-concurrent transactional objects. In Proc. of PPoPP ’08, pages 207–216. ACM Press, 2008. https://www.cs.stevens.edu/~ejk/papers/boosting-ppopp08.pdf <https://www.cs.stevens.edu/~ejk/papers/boosting-ppopp08.pdf>

9 28

[ANN] isbn - ISBN Validation and Manipulation
by Christian Charukiewicz 30 Jul '20

30 Jul '20

Hello Haskell Cafe, I wanted to share my first ever Haskell package: isbn https://hackage.haskell.org/package/isbn The package is motivated by my need to validate ISBNs (the unique identifier associated with every book published since 1970) in a Haskell application I am building. I published isbn as a back in May but yesterday I made some improvements the API and I think it is now ready to share as v1.1.0.0. I have been using Haskell commercially for a few years, and have made several contributions to various packages, but as mentioned, this is my first time authoring and publishing a package. If anyone has any feedback, I would be happy to hear it. Thank you, Christian Charukiewicz

2 2

Re: [Haskell-cafe] STM friendly TreeMap (or similar with range scan api) ? WAS: Best ways to achieve throughput, for large M:N ratio of STM threads, with hot TVar updates?
by Peter Robinson 30 Jul '20

30 Jul '20

Hi Compl, > >+ This package provides a proof-of-concept implementation of a skip list >> in STM >> >> This has to mean something but I can't figure out yet. >> >> Dear Peter Robinson, I hope you can see this message and get in the loop >> of discussion. >> > The reason for adding this sentence was that tskiplist hasn't been optimized for production use. Later on, I wrote an implementation of a concurrent skip list with atomic operations that performs significantly better, but it's operations work in the IO monad. I'm surprised to hear that you're getting poor performance even when using the stm-container package, which I believe was meant to be used in production. A while ago, I ran some benchmarks comparing concurrent dictionary data structures (such as stm-container) under various workloads. While STMContainers.Map wasn't as fast as the concurrent-hashtable package, the results indicate that the performance doesn't degrade too much under larger workloads. You can find these benchmark results here (10^6 randomly generated insertion/deletion/lookup requests distributed among 32 threads): https://lowerbound.io/blog/bench2-32.html And some explanations about the benchmarks are here: https://lowerbound.io/blog/2019-10-24_concurrent_hash_table_performance.html One issue that I came across when implementing the tskiplist package was this: If a thread wants to insert some item into the skip list, it needs to search for the entry point by performing readTVar operations starting at the list head. So, on average, a thread will read O(log n) TVars (assuming a skip list of n items) and, if any of these O(log n) TVars are modified by a simultaneously running thread, the STM runtime will observe a (false) conflict and rerun the transaction. It's not clear to me how to resolve this issue without access to something like unreadTVar (see [1]). Best, Peter [1] UnreadTVar: Extending Haskell Software Transactional Memory for Performance (2007) by Nehir Sonmez , Cristian Perfumo , Srdjan Stipic , Adrian Cristal , Osman S. Unsal , Mateo Valero.

3 3

Subtyping CoC
by james faure 28 Jul '20

28 Jul '20

Hello, I am working on a subtyping calculus of constructions: https://github.com/jfaure/Nimzo, based on Algebraic Subtyping [1] The goal is to leverage the synthesis of subtyping with the CoC for general purpose programming, both for the usual correctness guarantees, but additionally exceptional type inference and powerful optimizations by using subtyping relations on dependent types. I spent the last year researching the theory and practice for this, and with the compiler at now just over 3000 lines of Haskell, I feel like I am no longer advancing as quickly as I would like, so if anyone is interested in helping create this language of the future, please get in touch ! [1]: https://www.cs.tufts.edu/~nr/cs257/archive/stephen-dolan/thesis.pdf James Faure (Discord: J4#0303)

2 1

Job: Medical Solutions with Haskell in Freiburg (German speaker required)
by David Leuschner 28 Jul '20

28 Jul '20

Dear Haskellers, our team is very happily and successfully using Haskell in production for over 10 years. We're currently hiring german-speaking/learning, full-time, on-site Haskell developers to work on user-friendly web- and mobile applications for doctors, nurses, patients and all other people involved. We have a very experienced team and we're looking for new-comers that are eager to learn as well as we're looking for experienced Haskell developers. We'd love to hear from you! Please look at our full announcement on Reddit: https://www.reddit.com/r/haskell/comments/hy8tuv/medical_solutions_with_has… Kind regards, David -- David Leuschner Entwicklungsleitung Lohmann & Birkner Software Solutions GmbH Alt-Reinickendorf 25 13407 Berlin Email: leuschner(a)cp-med.com Web: http://www.checkpad.de Lohmann und Birkner Software Solutions GmbH Geschaeftsfuehrer: Dr. Ruediger Lohmann Handelsregister: Amtsgericht Berlin-Charlottenburg HRB 130806 B

1 0

How to understand the 320-byte heap footprint of UUID ?
by YueCompl 28 Jul '20

28 Jul '20

Hello Cafe, I'm about to introduce UUID into my code, and see https://github.com/haskell-hvr/uuid/issues/24 <https://github.com/haskell-hvr/uuid/issues/24> stating: > Currently, UUID is represented as data UUID = UUID {-# UNPACK #-} !Word32 {-# UNPACK #-} !Word32 {-# UNPACK #-} !Word32 {-# UNPACK #-} !Word32 > However, this suboptimal for 64bit archs (where GHC currently stores this a 320-byte Heap object); ... According to https://wiki.haskell.org/GHC/Memory_Footprint <https://wiki.haskell.org/GHC/Memory_Footprint> I can understand each evaluated `Word32` on 64-bit hardware can take 2 words - 16 bytes, and given they are unpacked and strict, I think one whole UUID record should just take 64 bytes plus a few words, which is far less than 320 bytes. So how comes the 320 bytes? Thanks with regards, Compl

3 2

how to get live info on thread scheduling?
by Johannes Waldmann 25 Jul '20

25 Jul '20

Dear Cafe, With Control.Concurrent, I would want to know, for a ThreadId, the total (up to now) (wallclock) time it was running/paused by the RTS scheduler. I guess that the scheduler's actions can be written to the eventlog for later analysis. Can I access them from inside a running program? I guess this is https://gitlab.haskell.org/ghc/ghc/-/wikis/event-log/live-monitoring , what is the status? - J.

1 0

GADT/Typeable/existential behaviour that I don't understand
by Tom Ellis 24 Jul '20

24 Jul '20

I can define the following import Data.Typeable data Foo where Foo :: Typeable x => x -> Foo eq = (\a b -> eqT) :: (Typeable a, Typeable b) => a -> b -> Maybe (a :~: b) and then these expressions work as expected > case Foo "Hello" of Foo x1 -> case eq x1 "" of { Nothing -> "It was not a string"; Just Refl -> "It was a string" } "It was a string" > case Foo 1 of Foo x1 -> case eq x1 "" of { Nothing -> "It was not a string"; Just Refl -> "It was a string" } "It was not a string" But if I omit the 'Nothing' branch (as below) I get "Couldn't match expected type ‘p’ with actual type ‘[Char]’ ‘p’ is untouchable". Can anyone explain why this happens? > case Foo "Hello" of Foo x1 -> case eq x1 "" of { Just Refl -> "It was not a string" } > case Foo 1 of Foo x1 -> case eq x1 "" of { Just Refl -> "It was not a string" }

3 5

Question about zippers on trees
by Dominik Schrempf 24 Jul '20

24 Jul '20

Hello Cafe! I am trying to modify a large 'Data.Tree.Tree'. I managed to modify node labels with specific indices in the form of @[Int]@ as they are defined in, for example, 'Control.Lens.At.Ixed' or 'Lens.Micro.GHC'. However, I also need to 1. modify the node label using information from nearby nodes (e.g., the children); 2. modify the tree structure itself; for example, I may want to change the sub-forest. Basically, I need a lens that focuses not on the node label, but on the node itself. I perceived that this is more difficult. I tried to use 'Control.Zipper'. I can use zippers to achieve point 1, albeit in a complicated way: (1) I need to go downwards to focus the specific node; (2) I need to traverse the children to collect data and save the data somewhere (how? in let bindings?); (3) I then go back upwards and change the node label using the collected data. Even so, I do not really manage to change the actual structure of the tree. I also briefly had a look at plates, but do not manage to use them in a proper way, maybe because the depth of my structures may be several hundred levels. Did you encounter similar problems in the past or could you point me to resources discussing these issues? Thank you! Dominik

5 7