- Haskell-Cafe - Haskell.org

Re: [Haskell-cafe] Adding Content-Addressable Storage to GHC
by Michal J Gajda 19 Mar '20

19 Mar '20

Great idea! We already have syntax for package imports: ``` import "bytestring" Data.ByteString ``` Source plugin could probably pull and build package by CAS: ``` import "CAS_6ffeacff768590" Data.ByteString ``` That would reuse existing syntax without confusion. And without inconvenience of having to import expressions one by one. Or declaring their types. One could even add CAS hashes to Hackage db. Probably the source plugin can add `-package` parameters with GHC_OPTIONS? On the expression side, it seems easy to use Generics to derive hashable for `Exp`, given a *particular compiler version*. Warning is, that one needs to be more careful when the `Exp`ression type changes. -- Cheers Michal

1 0

Adding Content-Addressable Storage to GHC
by Chris Done 19 Mar '20

19 Mar '20

Hi all, Is there any effort or designs ongoing to add CAS (content-addressable storage) to GHC, as in Unison? < https://www.unisonweb.org/docs/tour/> == The idea == The summary of the idea is simply that top-level declarations can be addressed by a hash of their contents. Recursive definitions are transformed into the worker/wrapper to eliminate the self-referencing issue of hashing. == Why I want this == There are lots of advantages to this, but the one that excites me the most is that we can move to running tests, especially property tests, at compile-time. The main downside to running tests at compile-time, as seen done with template-haskell is that you will re-run tests every time the module is recompiled, making your dev cycle slower. However, if your tests are keyed upon CAS hashes, then those hashes are only invalidated when individual declarations actually change. This means the re-running of tests becomes granular at the declaration-level. When a single test completes, either successfully or not, you can cache the result and lookup the result next time, using e.g. the SHA512 of the expression evaluated. Therefore you could change a single function in a library and it would only re-run the tests that are actually affected, rather than running all the tests in the whole module, and rather than the more typical approach which is running ALL tests in a test suite just because one thing changed. If you can couple tests with code then you can avoid the decoupling of code from the tests. == Implementation approaches == There are various ways to implement this with varying degrees of satisfaction: 1. Use TH: reify declarations, inspect the AST, and produce a SHA512. Use ambient values such as the GHC version, instances in scope, extensions, ghc options, etc. With TH, I'm confident that you can only achieve an imperfect hash because I doubt that all information is available to TH. Names that come from external packages could be treated as CAS'd at the scope of the package's installed hash. Ideally, you could have granularity into other packages. But it's not a necessity if you just want caching for your current development package. 2. Use a source plugin. A source plugin is already capable of accessing all GHC context information, so this might lead to more of a perfect hash. 3. Add it to GHC directly. Exposing a `expressionSHA512 :: Exp -> ByteString` could be one imaginary way to access this information. With such a function you could implement caching of fine-grained tests. A related discussion is the deterministic builds: https://gitlab.haskell.org/ghc/ghc/wikis/deterministic-builds Anyone else exploring this? Cheers, Chris

4 3

Re: [Haskell-cafe] Haskell-Cafe Digest, Vol 199, Issue 15
by Michal J Gajda 18 Mar '20

18 Mar '20

Great idea! We already have syntax for package imports: ``` import "bytestring" Data.ByteString ``` Source plugin could probably pull and build package by CAS: ``` import "CAS_6ffeacff768590" Data.ByteString ``` That would reuse existing syntax without confusion. And without inconvenience of having to import expressions one by one. Or declaring their types. One could even add CAS hashes to Hackage db. Wonder if source plugin can add `-package` parameters? On Wed, Mar 18, 2020, 13:01 <haskell-cafe-request(a)haskell.org> wrote: > Send Haskell-Cafe mailing list submissions to > haskell-cafe(a)haskell.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe > or, via email, send a message with subject or body 'help' to > haskell-cafe-request(a)haskell.org > > You can reach the person managing the list at > haskell-cafe-owner(a)haskell.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Haskell-Cafe digest..." > Today's Topics: > > 1. Re: Help with async library and killing child processes (YueCompl) > 2. Adding Content-Addressable Storage to GHC (Chris Done) > > > > ---------- Forwarded message ---------- > From: YueCompl <compl.yue(a)icloud.com> > To: Branimir Maksimovic <branimir.maksimovic(a)gmail.com> > Cc: haskell-cafe(a)haskell.org > Bcc: > Date: Wed, 18 Mar 2020 18:42:56 +0800 > Subject: Re: [Haskell-cafe] Help with async library and killing child > processes > Um, event processing applications by procedural programmers (part data > analyst personnels for some) is a thing in my org, so I'm baking an DSL > doing it. > > On 2020-03-18, at 18:18, Branimir Maksimovic < > branimir.maksimovic(a)gmail.com> wrote: > > That's just library, not built into language. You can do it with forkIO as > I do it per specifics > > of application. > > Greets, > Branimir. > On 3/18/20 11:06 AM, YueCompl wrote: > > No, not to cancel a thread, but the business task on that thread's way > ahead. That's what Go's Context is designed for. > > > - About Go context from: https://blog.golang.org/context > > At Google, we require that Go programmers pass a Context parameter as the > first argument to every function on the call path between incoming and > outgoing requests. This allows Go code developed by many different teams to > interoperate well. It provides simple control over timeouts and cancelation > and ensures that critical values like security credentials transit Go > programs properly. > Server frameworks that want to build on Context should provide > implementations of Context to bridge between their packages and those that > expect a Context parameter. Their client libraries would then accept a > Context from the calling code. By establishing a common interface for > request-scoped data and cancelation, Context makes it easier for package > developers to share code for creating scalable services. > > I (and seems Windows api too) strongly agree that cancelling a thread is > problematic: > > > https://docs.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-pro… > > TerminateThread is a dangerous function that should only be used in the > most extreme cases. You should call TerminateThread only if you know > exactly what the target thread is doing, and you control all of the code > that the target thread could possibly be running at the time of the > termination. For example, TerminateThread can result in the following > problems: > > - If the target thread owns a critical section, the critical section > will not be released. > - If the target thread is allocating memory from the heap, the heap > lock will not be released. > - If the target thread is executing certain kernel32 calls when it is > terminated, the kernel32 state for the thread's process could be > inconsistent. > - If the target thread is manipulating the global state of a shared > DLL, the state of the DLL could be destroyed, affecting other users of the > DLL. > > > On 2020-03-18, at 17:16, Branimir Maksimovic < > branimir.maksimovic(a)gmail.com> wrote: > > I'm pretty sure you can't cancel goroutine. Actually I have > > very bad experience with thread cancelation. Therefore flags > > passed via channels/mvars or shared flags are used to stop > > threads. > > Greets, > Branimir. > On 3/18/20 10:10 AM, YueCompl wrote: > > Idiomatic resource leakage prevention in short. > > An app wants to be as responsive to its users as practically feasible, it > would try multiple paths to load-balanced resources, and use the first > response that come back, then cancel other paths. Cancellation can help > reducing vain computation in such architectures, sometimes largely enough. > > On 2020-03-18, at 16:58, Branimir Maksimovic < > branimir.maksimovic(a)gmail.com> wrote: > > What's wrong with forkIO? > > Greets, > Branimir. > On 3/18/20 9:37 AM, YueCompl via Haskell-Cafe wrote: > > If not for tight loops or other CPU intensive tasks, you may be interested > in Edh, which introduces Go's goroutine to GHC runtime. But Edh threads add > much higher overhead on top of GHC threads, so there's a price to pay for > simplicity of end programmer's job. > > It's briefly described at > https://github.com/e-wrks/edh/tree/master/Tour#program--threading-model . > This is very new and I'm right now actively working on it for PoC of an STM > powered in-memory graph database implementation. > > Best regards, > Compl > > On 2020-03-18, at 11:15, Niklas Hambüchen <mail(a)nh2.me> wrote: > > I've you're new to async, also check out my recent rework of the docs that > talk about this topic specifically: > > https://github.com/simonmar/async/pull/105/files > > (To be available in the next release of async.) > _______________________________________________ > Haskell-Cafe mailing list > To (un)subscribe, modify options or view archives go to: > http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe > Only members subscribed via the mailman list are allowed to post. > > > > _______________________________________________ > Haskell-Cafe mailing list > To (un)subscribe, modify options or view archives go to:http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe > Only members subscribed via the mailman list are allowed to post. > > _______________________________________________ > Haskell-Cafe mailing list > To (un)subscribe, modify options or view archives go to: > http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe > Only members subscribed via the mailman list are allowed to post. > > > > > > > > ---------- Forwarded message ---------- > From: Chris Done <haskell-cafe(a)chrisdone.com> > To: haskell-cafe(a)haskell.org > Cc: > Bcc: > Date: Wed, 18 Mar 2020 11:21:29 +0000 > Subject: [Haskell-cafe] Adding Content-Addressable Storage to GHC > Hi all, > > Is there any effort or designs ongoing to add CAS (content-addressable > storage) to GHC, as in Unison? < > https://www.unisonweb.org/docs/tour/> > > == The idea == > > The summary of the idea is simply that top-level declarations can be > addressed by a hash of their contents. Recursive definitions are > transformed into the worker/wrapper to eliminate the self-referencing issue > of hashing. > > == Why I want this == > > There are lots of advantages to this, but the one that excites me the most > is that we can move to running tests, especially property tests, at > compile-time. > > The main downside to running tests at compile-time, as seen done with > template-haskell is that you will re-run tests every time the module is > recompiled, making your dev cycle slower. However, if your tests are keyed > upon CAS hashes, then those hashes are only invalidated when individual > declarations actually change. This means the re-running of tests becomes > granular at the declaration-level. When a single test completes, either > successfully or not, you can cache the result and lookup the result next > time, using e.g. the SHA512 of the expression evaluated. > > Therefore you could change a single function in a library and it would > only re-run the tests that are actually affected, rather than running all > the tests in the whole module, and rather than the more typical approach > which is running ALL tests in a test suite just because one thing changed. > > If you can couple tests with code then you can avoid the decoupling of > code from the tests. > > == Implementation approaches == > > There are various ways to implement this with varying degrees of > satisfaction: > > 1. Use TH: reify declarations, inspect the AST, and produce a SHA512. Use > ambient values such as the GHC version, instances in scope, extensions, ghc > options, etc. With TH, I'm confident that you can only achieve an imperfect > hash because I doubt that all information is available to TH. > > Names that come from external packages could be treated as CAS'd at the > scope of the package's installed hash. Ideally, you could have granularity > into other packages. But it's not a necessity if you just want caching for > your current development package. > > 2. Use a source plugin. A source plugin is already capable of accessing > all GHC context information, so this might lead to more of a perfect hash. > > 3. Add it to GHC directly. Exposing a `expressionSHA512 :: Exp -> > ByteString` could be one imaginary way to access this information. With > such a function you could implement caching of fine-grained tests. > > A related discussion is the deterministic builds: > https://gitlab.haskell.org/ghc/ghc/wikis/deterministic-builds > > Anyone else exploring this? > > Cheers, > > Chris > > > > > _______________________________________________ > Haskell-Cafe mailing list > Haskell-Cafe(a)haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe >

1 0

[HiW'20] Call for Talks
by Ben Gamari 16 Mar '20

16 Mar '20

Hello everyone, Haskell Implementors Workshop is calling for talk proposals. Co-located with ICFP, HiW is an ideal place to describe a Haskell library, a Haskell extension, compiler, works-in-progress, demo a new Haskell-related tool, or even propose future lines of Haskell development. The deadline for submissions is July 2nd 2020. Call for Talks ============== The 12th Haskell Implementors’ Workshop is to be held alongside ICFP 2020 this year in New Jersey. It is a forum for people involved in the design and development of Haskell implementations, tools, libraries, and supporting infrastructure, to share their work and discuss future directions and collaborations with others. Talks and/or demos are proposed by submitting an abstract, and selected by a small program committee. There will be no published proceedings. The workshop will be informal and interactive, with open spaces in the timetable and room for ad-hoc discussion, demos and lightning talks. Scope and Target Audience ------------------------- It is important to distinguish the Haskell Implementors’ Workshop from the Haskell Symposium which is also co-located with ICFP 2020. The Haskell Symposium is for the publication of Haskell-related research. In contrast, the Haskell Implementors’ Workshop will have no proceedings – although we will aim to make talk videos, slides and presented data available with the consent of the speakers. The Implementors’ Workshop is an ideal place to describe a Haskell extension, describe works-in-progress, demo a new Haskell-related tool, or even propose future lines of Haskell development. Members of the wider Haskell community encouraged to attend the workshop – we need your feedback to keep the Haskell ecosystem thriving. Students working with Haskell are specially encouraged to share their work. The scope covers any of the following topics. There may be some topics that people feel we’ve missed, so by all means submit a proposal even if it doesn’t fit exactly into one of these buckets: - Compilation techniques - Language features and extensions - Type system implementation - Concurrency and parallelism: language design and implementation - Performance, optimization and benchmarking - Virtual machines and run-time systems - Libraries and tools for development or deployment Talks ----- We invite proposals from potential speakers for talks and demonstrations. We are aiming for 20-minute talks with 5 minutes for questions and changeovers. We want to hear from people writing compilers, tools, or libraries, people with cool ideas for directions in which we should take the platform, proposals for new features to be implemented, and half-baked crazy ideas. Please submit a talk title and abstract of no more than 300 words. Submissions can be made via HotCRP at https://icfp-hiw20.hotcrp.com/ until July 2nd (anywhere on earth). We will also have lightning talks session. These have been very well received in recent years, and we aim to increase the time available to them. Lightning talks be ~7mins and are scheduled on the day of the workshop. Suggested topics for lightning talks are to present a single idea, a work-in-progress project, a problem to intrigue and perplex Haskell implementors, or simply to ask for feedback and collaborators. Logistics --------- We recognize the on-going threat that COVID-19 poses to our participants' safety. While August is nearly half a year away, we must account for the possibility that the virus continues to pose a significant threat into the summer. For this reason, we are investigating options that would allow remote presentations this year. In light of this, we urge potential presenters not to be discouraged from submitting and encourage participants to keep time open in their calendars, regardless of the on-going COVID situation. Rest assured that the conference organizers are working to ensure that the Implementors Workshop can be held safely and productively, regardless of how the COVID-19 situation evolves. Program Committee ----------------- - Andrey Mokhov (Newcastle University) - Ben Gamari (Well-Typed LLP) - Christian Baaij (QBayLogic) - George Karachalias (Tweag I/O) - Klara Marntirosian (KU Leuven) - Matthew Pickering (Univeristy of Bristol) - Ryan Scott (Indiana University Bloomington) Best wishes, Ben

1 0

Re: [Haskell-cafe] Data.Map - visiting tree nodes withi a given key range ?
by Olaf Klinke 16 Mar '20

16 Mar '20

You can roll your own indexKeyRange using the Data.Map.Internal module which exposes the (currently used) Map implementation. Also note that if the list of values in range is to be consumed immediately, you might want to go for a fold-based function: foldlWithRange :: (a -> k -> b -> a) -> (a,a) -> b -> Map k a -> b Olaf

2 5

Data.Map - visiting tree nodes within a given key range ?
by Compl Yue 15 Mar '20

15 Mar '20

Hello, I have a question regarding 'Data.Map' api, filed an issue https://github.com/haskell/containers/issues/708 And may be I can ask here at the same time? I'm not sure why|Data.Map|doesn't have a key range based visiting API, I figured out I can do it this way: |indexKeyRange :: IndexKey -> IndexKey -> Map IndexKey Object -> [(IndexKey, Object)] indexKeyRange !minKey !maxKey = toList . takeWhileAntitone (<= maxKey) . dropWhileAntitone (< minKey) | But wouldn't it save the computation needed to re-balance the intermediate tree generated ? Or that re-balancing can be optimized out actually ? I am creating an in-memory graph database, using|Data.Map.Strict.Map|as business object indices with specified object attributes. The typical scenario will be querying a small number of entries by key range, out of possibly all business objects of a certain class globally, so the implementation above would work, but not reasonable by far as it seems. I think a lazy list returned by mere node visiting (i.e. no new node creation) would satisfy my needs, or I missed something ? Thanks, Compl

2 1

Calling a State Monad recursively from within a let-block, via runState?
by David Banas 14 Mar '20

14 Mar '20

Can I call a State Monad recursively from within a let-block, via `runState`, or is that going way out of bounds? -db

2 1

ANNOUNCE: Save the date! Compose Conference 2020 colocated with ICFP
by MightyByte 13 Mar '20

13 Mar '20

We are excited to announce that the Compose Conference will be colocated with ICFP this year. Compose will be Thursday and Friday August 20 and 21 at the NJCU School of Business, a short subway / bus ride from Manhattan. The Compose Unconference will be Saturday and Sunday August 22 and 23 at Microsoft in Times Square. Regarding COVID-19, the Compose Team is closely monitoring the situation and are prepared to make responsible decisions if it comes to that. That being said, the situation will undoubtedly evolve a lot between now and August. We have been very excited about the collaboration between Compose and ICFP and wanted to get an announcement out so that if the conferences do happen as planned everyone is aware. A more detailed announcement will be coming when we have more confirmed details, but if you are already planning to attend ICFP, consider extending your trip by a few days and coming to Compose as well. To get a sense of Compose, you can check out the great talks from past conferences and the Compose Conference website: https://www.youtube.com/channel/UC0pEknZxL7Q1j0Ok8qImWdQ http://www.composeconference.org/ We look forward to seeing you! The Compose Conference Team Sent via Superhuman ( https://sprh.mn/?vip=mightybyte@gmail.com )

2 1

Using GHC option '-optl-static' causes segfault
by Nutr1t07 11 Mar '20

11 Mar '20

Hi, I added '-optl-static' in GHC option to make my program compiled statically. When I compiled the following code using 'stack build': -- Main.hs {-# LANGUAGE OverloadedStrings #-} module Main where import Web.Scotty as Scotty import Network.Wreq as Wreq import Control.Monad.IO.Class main :: IO () main = scotty 8443 $ do Scotty.get (literal "/") $ do _ <- liftIO $ Wreq.get "https://www.google.com/" html "Could not see this" -- Ran the compiled program and accessing "http://localhost:8443" would raise a segmentation fault. Here are what in my 'ghc-option': ghc-options: - -optl-static - -threaded - -rtsopts - -with-rtsopts=-N Then I removed the '-optl-static', compiled it, and the program ran well. It seems that it something has to do with glibc. What can I do with this problem? Regards PS. I'm using GHC 8.8.2, stack 2.1.3

6 10

[ANN] ghc-tag-plugin
by Marcin Szamotulski 10 Mar '20

10 Mar '20

Hi, I wrote a GHC plugin which writes tags files. I've been proof testing it on a large code base where parts of it is using various type extensions, or formats (literate haskell, haskell, hsc files) - it works very well. Here's a blog post about it: https://coot.me/posts/ghc-tags-plugin.html And here's a very fresh hackage version 0.1.3.0: https://hackage.haskell.org/package/ghc-tags-plugin Best regards, Marcin Szamotulski

2 2