- Haskell-Cafe - Haskell.org

Measuring memory usage
by Vlatko Basic 04 Jul '18

04 Jul '18

Hello, I've come to some strange results using Weigh package. It shows that HashMap inside 'data' is using much, much more memory. The strange thing is that I'm seeing too large mem usage in my app as well (several "MapData" like in records), and trying to figure out with 'weigh' what's keeping the mem. Noticed that when I change the code to use HashMap directly (not inside 'data', that's the only change), the mem usage observed with top drops down for ~60M, from 850M to 790M. These are the test results for 10K, 5K and 3.3K items for "data MapData k v = MapData (HashMap k v)" (at the end is the full runnable example.) Case Allocated GCs HashMap 262,824 0 HashMap half 58,536 0 HashMap third 17,064 0 MapData 4,242,208 4 I tested by changing the order, disabling all but one etc., and the results were the same. Same 'weigh' behaviour with IntMap and Map. So, if anyone knows and has some experience with such issues, my questions are: 1. Is 'weigh' package reliable/usable, at least to some extent? (the results do show diff between full, half and third) 2. How do you measure mem consumptions of your large data/records? 3. If the results are even approximately valid, what could cause such large discrepancies with 'data'? 4. Is there a way to see if some record has been freed from memory, GCed? module Main where import Prelude import Control.DeepSeq (NFData) import Data.HashMap.Strict (HashMap, fromList) import GHC.Generics (Generic) import Weigh (mainWith, value) data MapData k v = MapData (HashMap k v) deriving Generic instance (NFData k, NFData v) => NFData (MapData k v) full, half, third :: Int full = 10000 half = 5000 third = 3333 main :: IO () main = mainWith $ do value "HashMap" ( mkHMList full) value "HashMap half" ( mkHMList half) value "HashMap third" ( mkHMList third) value "MapData" (MapData $ mkHMList full) mkHMList :: Int -> HashMap Int String mkHMList n = fromList . zip [1..n] $ replicate n "some text"

4 8

Typechecker Plugin Constraint Solving
by Armin Wurzinger 04 Jul '18

04 Jul '18

Good day, I am currently developing a typechecker plugin for GHC for realizing an idea regarding types based on regular expressions to describe the contents of a string. This is done using type-level literals to denote a particular regex. Additionally there are additional constraints for working with such types. One of those constraints is called Intersectable and denotes whether an intersection between two regexes would be possible in the sense that there is some common alphabet between them. Furthermore there is a type-level function that calculates the actual intersection between two regex types and returns the resulting regex type: type family Intersectable (a :: Symbol) (b :: Symbol) :: Constraint type family Intersection (a :: Symbol) (b :: Symbol) :: Symbol Now i've created a typechecker plugin that resolves such constraints by using a finite-automata representation of the regexes to do the actual work. While my plugin is already able to proof progams where regexes are fully applied, i.e. they can reduce terms to a single regex and then verify the constraints accordingly or reject them, it fails to type "intermediate" functions. As a small example assume the following program: checkFour :: Intersectable a "[0-4]" => RgxString a -> RgxString (Intersection a "[0-4]") checkFour a = undefined checkEight :: Intersectable a "[0-8]" => RgxString a -> RgxString (Intersection a "[0-8]") checkEight a = undefined test = checkFour . checkEight -- this fails to type correctly test2 = checkFour $ checkEight $ (RgxString :: RegexString ".*") -- this works fine The constraint solver wants me to proof the following constraints for test: tcPluginSolve given = [] derived = [] wanted = [ [WD] irred_a2GV {0}:: Intersectable (Intersection a0 "[0-8]") "[0-4]" (CNonCanonical), [WD] irred_a2GY {0}:: Intersectable a0 "[0-8]" (CNonCanonical)] My plugin reduces the first constraint to Intersectable a0 [0-4] as regex intersection is commutative so i can intersect [0-8] with [0-4] first. But i haven't found out yet how i would tell the compiler to put this reduced constraint to the type signature instead of the current one where the constraint got dropped at all instead of at least remain unnormalized, and the return type remains unnormalized as well: RgxString a -> RgxString (Intersection (Intersection a "[0-8]") "[0-4]") I'd need to tell the typechecker somehow that (Intersection (Intersection a "[0-8]") "[0-4]") is equal to Intersection a "[0-4]" so it substitutes it accordingly. My idea is that i have to generate given type and function equality constraints during the typechecking telling it that those representations are equal, but i am really not sure what kind of constraints i need. CTyEqCan for instance seems to require a TyVar on one side while i though i could generate something like Intersection (Intersection a "[0-8]") "[0-4]" ~ Intersection a "[0-4]" I have already tried a lot but there is little documentation on the whole constraint solving process and i am slowly getting stuck. Thanks for helping!

1 0

ForeignPtrs, Finalizers and thread-local storage
by Justin Paston-Cooper 03 Jul '18

03 Jul '18

glpk-hs defines a binding to glpk, a linear programme library. A linear programme is created by calling `glp_create_prob` and deleted by calling `glp_delete_prob`. These two C functions manage memory themselves using thread-local storage. They are called by glpk-hs using the foreign function interface in the following code. Note that glpDelProb and glpCreateProb are imported foreign functions. runGLPK :: GLPK a -> IO a runGLPK m = do lp <- newForeignPtr glpDelProb =<< glpCreateProb withForeignPtr lp (execGLPK m) ( https://github.com/jyp/glpk-hs/blob/master/src/Data/LinearProgram/GLPK/Type… ) It seems that sometimes the finaliser is called in a thread different to the one where glp_create_prob is called. This leads to glpk aborting with a dynamic memory allocation error when tries to de-allocate the linear programme in the wrong thread-local storage, which has never seen that programme. Given that the ForeignPtr is used only in runGLPK, runGLPK could of course be fixed by using Control.Concurrent.bracket instead. However, I'm interested to know whether there is a solution to this problem. I'm only assuming that the finaliser sometimes runs in a different thread. If this is indeed the case, then is there any way to ensure that a finaliser runs in the thread where the pointer is created?

1 1

GHC's timing: Now solid enough for music?
by Jeffrey Brown 03 Jul '18

03 Jul '18

I just discovered Tom Murphy's Vivid-Synth[1], a Haskell library that lets you control SuperCollider. (A few very slight changes allowed me to get it to work in Stack instead of Cabal, as described here[2,3].) TidalCycles[4] also controls SuperCollider from Haskell. It does tortuous backflips to hand the timing job over to SuperCollider. If I recall correctly, that was because SC's timing was more solid than Haskell's. I modified the Vivid demo to go faster and have a few more voices[5], and tried running it at the same time as Spotify and Youtube and apt upgrade and loading LibreOffice and playing some music in VLC, and I detected (by ear) no timing errors. Can I expect GHC's maybe-new rhythmic stability to hold up? [1] http://www.vivid-synth.com/ [2] https://github.com/vivid-synth/vivid/issues/5 [3] https://github.com/JeffreyBenjaminBrown/vivid/commit/3d25266d4495d9dd8e573f… [4] https://tidalcycles.org/ [5] https://github.com/JeffreyBenjaminBrown/vivid/blob/jbb/jbb/sine.hs -- Jeff Brown | Jeffrey Benjamin Brown Website <https://msu.edu/~brown202/> | Facebook <https://www.facebook.com/mejeff.younotjeff> | LinkedIn <https://www.linkedin.com/in/jeffreybenjaminbrown>(spammy, so I often miss messages here) | Github <https://github.com/jeffreybenjaminbrown>

3 2

ANN: hledger-1.10
by Simon Michael 02 Jul '18

02 Jul '18

hledger (http://hledger.org) is a friendly, robust, cross-platform program for tracking money, time or other commodities, using double-entry accounting, simple plain text file formats, and command-line, curses or web UIs. I'm very pleased to announce hledger 1.10, including work by release contributors: Alex Chen, Everett Hildenbrandt, Jakub Zárybnický, Nolan Darilek, Dmitry Astapov, Jacob Weisz, Peter Simons, Stephen Morgan, Pavlo Kerestey, Trevor Riles, Léo Gaspard, Mykola Orliuk, Wad, Nana Amfo. You'll note this is not 2.0 as previously planned - we will spend a bit more time on that. The next major release is expected to be on 2018/09/30. Please see http://hledger.org/release-notes#hledger-1.10 for the changes (as soon as I can get to those; until then, the package changelogs): http://hackage.haskell.org/package/hledger-1.10/changelog http://hackage.haskell.org/package/hledger-ui-1.10/changelog http://hackage.haskell.org/package/hledger-web-1.10/changelog http://hackage.haskell.org/package/hledger-api-1.10/changelog http://hackage.haskell.org/package/hledger-lib-1.10/changelog How to get started: ------------------- See http://hledger.org/download for all install methods. One of the easiest is the hledger-install script, which requires only bash and will build and install the hledger tools in $HOME/.local/bin/: $ curl -O https://raw.githubusercontent.com/simonmichael/hledger/master/hledger-insta… $ less hledger-install.sh # (do security review) $ bash hledger-install.sh # (add -v for more detail; use bash -x to show commands being run) or (insecure): $ curl https://raw.githubusercontent.com/simonmichael/hledger/master/hledger-insta… | bash After installation, ensure $HOME/.local/bin is in your $PATH, and try some commands: $ hledger -h # quick help $ hledger help # list built-in manuals $ hledger add # record some transactions $ hledger # list available commands Next, I encourage you to at least skim the tutorials and docs at http://hledger.org. Say hello and ask questions in the #hledger IRC channel on Freenode: http://irc.hledger.org. New users and contributors are always welcome! Give feedback, report bugs, send pull requests, write, evangelise, donate. Best! -Simon

1 1

2nd Call for Talks -- Haskell Implementors' Workshop
by Joachim Breitner 02 Jul '18

02 Jul '18

Call for Contributions ACM SIGPLAN Haskell Implementors’ Workshop https://icfp18.sigplan.org/track/hiw-2018-papers Co-located with ICFP 2018 St. Louis, Missouri, US https://conf.researchr.org/home/icfp-2018 Important dates --------------- Proposal Deadline: Friday, 20 July, 2018 Notification: Friday, 3 August, 2018 Workshop: Sunday, 23 September, 2018 Keynote speaker --------------- This year, the Haskell Implementors’ Workshop is proud to present Rahul Muttineni as the keynote speaker. Rahul brough the joys of Haskell to the realm of Java by creating the Eta programming language. Title: Let's Go Mainstream with Eta! Eta is a fork of GHC that focuses on three core principles: user experience, performance, and safety. We'll discuss how we used these principles to guide the re-implementation of the GHC runtime and code generator on the JVM. Moreover, will also cover the inner workings of the FFI and the typechecker support we added for subtyping to make it smooth to interact with Java libraries. Finally, we'll round out with a look at where Eta is headed and how Eta and GHC can collaborate in the future. About the Haskell Implementors’ Workshop ---------------------------------------- The 10th Haskell Implementors’ Workshop is to be held alongside ICFP 2018 this year in St. Louis. It is a forum for people involved in the design and development of Haskell implementations, tools, libraries, and supporting infrastructure, to share their work and discuss future directions and collaborations with others. Talks and/or demos are proposed by submitting an abstract, and selected by a small program committee. There will be no published proceedings. The workshop will be informal and interactive, with open spaces in the timetable and room for ad-hoc discussion, demos and lightning talks. Scope and Target Audience ------------------------- It is important to distinguish the Haskell Implementors’ Workshop from the Haskell Symposium which is also co-located with ICFP 2018. The Haskell Symposium is for the publication of Haskell-related research. In contrast, the Haskell Implementors’ Workshop will have no proceedings – although we will aim to make talk videos, slides and presented data available with the consent of the speakers. The Implementors’ Workshop is an ideal place to describe a Haskell extension, describe works-in-progress, demo a new Haskell-related tool, or even propose future lines of Haskell development. Members of the wider Haskell community encouraged to attend the workshop – we need your feedback to keep the Haskell ecosystem thriving. Students working with Haskell are specially encouraged to share their work. The scope covers any of the following topics. There may be some topics that people feel we’ve missed, so by all means submit a proposal even if it doesn’t fit exactly into one of these buckets: * Compilation techniques * Language features and extensions * Type system implementation * Concurrency and parallelism: language design and implementation * Performance, optimisation and benchmarking * Virtual machines and run-time systems * Libraries and tools for development or deployment Talks ----- We invite proposals from potential speakers for talks and demonstrations. We are aiming for 20-minute talks with 5 minutes for questions and changeovers. We want to hear from people writing compilers, tools, or libraries, people with cool ideas for directions in which we should take the platform, proposals for new features to be implemented, and half-baked crazy ideas. Please submit a talk title and abstract of no more than 300 words. Submissions can be made via HotCRP at https://icfp-hiw18.hotcrp.com/ until July 20th (anywhere on earth). We will also have lightning talks session. These have been very well received in recent years, and we aim to increase the time available to them. Lightning talks be ~7mins and are scheduled on the day of the workshop. Suggested topics for lightning talks are to present a single idea, a work-in-progress project, a problem to intrigue and perplex Haskell implementors, or simply to ask for feedback and collaborators. Program Committee ----------------- * Edwin Brady (University of St. Andrews, UK) * Joachim Breitner – chair (University of Pennsylvania) * Ben Gamari (Well-Typed LLP) * Michael Hanus (Kiel University) * Roman Leshchinsky (Facebook) * Niki Vazou (University of Maryland) Contact ------- * Joachim Breitner <joachim(a)cis.upenn.edu> -- Joachim Breitner Post-Doctoral researcher http://cis.upenn.edu/~joachim

1 0

Re: [Haskell-cafe] [ANNOUNCE] GHC 8.6.1-alpha1 available
by Ben Gamari 02 Jul '18

02 Jul '18

George Colpitts <george.colpitts(a)gmail.com> writes: > I don't see an apple/darwin binary. Not sure if that is an oversight or was > planned. > Yikes! Thank you for mentioning this. This is indeed a mistake. I'm uploading the Darwin distribution right now; it will be done within 20 minutes. Cheers, - Ben

1 0

[ANNOUNCE] GHC 8.6.1-alpha1 available
by Ben Gamari 01 Jul '18

01 Jul '18

The GHC development team is pleased to announce the first alpha release leading up to GHC 8.6.1. The usual release artifacts are available from https://downloads.haskell.org/~ghc/8.6.1-alpha1 This is the first release (partially) generated using our new CI infrastructure. One known issue is that the haddock documentation is currently unavailable. This will be fixed in the next alpha release. Do let us know if you spot anything else amiss. As always, do let us know if you encounter any trouble in the course of testing. Thanks for your help! Cheers, - Ben

4 6

need help understanding how to specify constraints on monads
by Dennis Raddle 29 Jun '18

29 Jun '18

I'm writing a program with several functions, some of which depend on certain fields in a state monad, others of which depend on others, but no routine needs all the fields. So I thought I would declare a two classes, one for each type of data need that a function has: -- as an aside, here's an example of data which is parameterized by two types. data ReportData t1 t2 = ... -- this is rolling my own state monad with a random generator class Monad m => RandMonad m where getGen :: m StdGen putGen :: StdGen -> () -- this is a class of state monad which logs ReportData: class Monad m => LogMonad m where putReport :: ReportData t1 t2 -> m () For a particular use case, I declare a type of State monad: data MyStateData t1 t2 = MyStateData t1 t2 { theGen :: StdGen , theReports :: [StepReport t1 t2] } type MyState t1 t2 = State (MyStateData t1 t2) And I try to define my instances: instance RandMonad (MyState t1 t2) where getGen = gets theGen putGen g = modify (\s -> s { theGen = g}) instance LogMonad (MyState t1 t2) where putReport r = modify (\s -> s { theReports = r : theReports s}) I get an error on the LogMonad instance, saying that there's no instance for (MonadState (MyState t1 t2) (StateT (MyState t1 t2) Identity)) I guess I don't really understand typeclasses once you start using higher kinded types, so please enlighten me. Any reading on this subject would be helpful, too.

5 12

(New to Conduits) mixing lazy lists and Conduits?
by Jon Fairbairn 29 Jun '18

29 Jun '18

Suppose I’m writing some code using Conduits, but need to use some old function f::[a]->[b] (defined in a library somewhere) that transforms a lazy list. Is there a way of turning f into a Conduit without ending up with all of the list being in memory? ie something that looks like toConduit:: ([a]->[b]) -> ConduitT a b m () I’ve got nowhere with Hoogle or Hayoo -- Jón Fairbairn Jon.Fairbairn(a)cl.cam.ac.uk

3 7