Re: [Haskell-cafe] Free monad based EDSL for writing LLVM programs.

8 Aug 2014

      For transforming the code, have you checked out the uniplate package? It seems like it could fit your problem pretty well.

On 8 August, 2014 at 5:00:19 AM, haskell-cafe-request@haskell.org (haskell-cafe-request@haskell.org) wrote:

Send Haskell-Cafe mailing list submissions to 
haskell-cafe@haskell.org 

To subscribe or unsubscribe via the World Wide Web, visit 
http://www.haskell.org/mailman/listinfo/haskell-cafe 
or, via email, send a message with subject or body 'help' to 
haskell-cafe-request@haskell.org 

You can reach the person managing the list at 
haskell-cafe-owner@haskell.org 

When replying, please edit your Subject line so it is more specific 
than "Re: Contents of Haskell-Cafe digest..." 

Today's Topics: 

1. Re: Side-by-side pretty printing (J. Waldmann) 
2. Free monad based EDSL for writing LLVM programs. (arrowdodger) 
3. parsec: problem combining lookAhead with many1 (bug?) (silly8888) 
4. Re: parsec: problem combining lookAhead with many1	(bug?) 
(Andreas Reuleaux) 
5. Performance of StateT and best practices for	debugging 
(Kyle Hanson) 
6. [ANN] rtorrent-state 0.1.0.0 (Mateusz Kowalczyk) 
7. Re: Performance of StateT and best practices for	debugging 
(John Lato) 
8. How to improve the zipwith's performance (jun zhang) 
9. Re: Performance of StateT and best practices for	debugging 
(Bardur Arantsson) 
10. Visualising Haskell function execution (Jan Paul Posma) 
11. Re: Performance of StateT and best practices for	debugging 
(John Lato) 

---------------------------------------------------------------------- 

Message: 1 
Date: Thu, 7 Aug 2014 12:33:43 +0000 (UTC) 
From: J. Waldmann  
To: haskell-cafe@haskell.org 
Subject: Re: [Haskell-cafe] Side-by-side pretty printing 
Message-ID:  
Content-Type: text/plain; charset=us-ascii 

This is what I use 

http://autolat.imn.htwk-leipzig.de/gitweb/?p=autolib;a=blob;f=todoc/Autolib/... 

it's of the works-but-looks-ugly-and-is-terribly-inefficient variety 
but since it's applied to small Docs only (like, columns of matrices), 
I don't really care. 

- J.W. 

------------------------------ 

Message: 2 
Date: Thu, 7 Aug 2014 18:16:57 +0400 
From: arrowdodger <6yearold@gmail.com> 
To: haskell-cafe@haskell.org 
Subject: [Haskell-cafe] Free monad based EDSL for writing LLVM 
programs. 
Message-ID: 

Content-Type: text/plain; charset="utf-8" 

Hello. I'm new with Haskell and FP, so i wanted someone to give comments on 
the package i've made [1]. It's, actually, my first attempt to create 
something more or less real, so any feedback would be welcome. 

I've used Free monad to create EDSL that allows writing LLVM IR code. 
Afterwards it could be converted into pure AST structure provided by 
llvm-general-pure[2] package. Currently, it supports almost every 
instruction, but i haven't yet come up with sensible defaults for them. 

Another thing that bugs me is the ability to transform the code in syb way. 
I want take a user-supplied function that would pattern-match instruction 
and produce another code block and apply this function everywhere in the 
code, but still can't get my head around it. I've come up with extF 
function, that unlike extM, would resort to wrap instead of return, but 
that's all i've managed to do. 

Thanks in advance. 

[1] https://bitbucket.org/arrowdodger/llvm-general-edsl 
[2] http://hackage.haskell.org/package/llvm-general-pure 
-------------- next part -------------- 
An HTML attachment was scrubbed... 
URL: http://www.haskell.org/pipermail/haskell-cafe/attachments/20140807/6d973f16/... 

------------------------------ 

Message: 3 
Date: Thu, 7 Aug 2014 15:25:23 +0100 
From: silly8888  
To: haskell-cafe@haskell.org 
Subject: [Haskell-cafe] parsec: problem combining lookAhead with many1 
(bug?) 
Message-ID: 

Content-Type: text/plain; charset=UTF-8 

Suppose that we have the following parser: 

p = lookAhead (char 'a') >> char 'b' 

If we use it like so 

parse p "" "a" 

we get the following error: 

Left (line 1, column 1): 
unexpected "a" 
expecting "b" 

What happened is that char 'a' succeeded by consuming the 'a' from the 
input and then lookAhead rewinded the input stream (as it does on 
success). Then, char 'b' tries to parse (again) the first character of 
the input and fails. Everything works as expected. 

Now let's slightly modify our parser: 

p' = lookAhead (many1 $ char 'a') >> char 'b' 

I've only added a many1. I was expecting this parser to give the same 
error as the previous one: many1 $ char 'a' will succeed consuming one 
'a' and then lookAhead will rewind the input (as it does on success). 
Thus when we call char 'b' we are going to be in the beginning of the 
input again. Well, that doesn't happen: 

Left (line 1, column 2): 
unexpected end of input 
expecting "b" 

As you can see, lookAhead did not rewind the input as it was supposed to. 

------------------------------ 

Message: 4 
Date: Thu, 07 Aug 2014 17:32:11 +0100 
From: Andreas Reuleaux  
To: silly8888  
Cc: haskell-cafe@haskell.org 
Subject: Re: [Haskell-cafe] parsec: problem combining lookAhead with 
many1	(bug?) 
Message-ID: <87y4v0z2es.fsf@web.de> 
Content-Type: text/plain 

While I haven't tried out your example in parsec, I can at least confirm 
that in trifecta it does work that way you expect it, ie. there is no 
difference between the error messages in both of your cases: 
(parsec's many1 = trifecta's some) 

Prelude > :m +Text.Trifecta 
Prelude Text.Trifecta > :m +Text.Parser.LookAhead 
Prelude Text.Trifecta Text.Parser.LookAhead > 
... 
Prelude Text.Trifecta Text.Parser.LookAhead > parseTest (lookAhead (char 'a') >> char 'b') "a" 
... 
Loading package reducers-3.10.2.1 ... linking ... done. 
Loading package trifecta-1.5.1 ... linking ... done. 
(interactive):1:1: error: expected: "b" 
a<EOF> 
^ 
Prelude Text.Trifecta Text.Parser.LookAhead > parseTest (lookAhead (some $ char 'a') >> char 'b') "a" 
(interactive):1:1: error: expected: "b" 
a<EOF> 
^ 
Prelude Text.Trifecta Text.Parser.LookAhead > 

Hope this helps. 

-Andreas 

silly8888  writes:
...
Suppose that we have the following parser:
p = lookAhead (char 'a') >> char 'b'
If we use it like so
parse p "" "a"
we get the following error:
Left (line 1, column 1): 
unexpected "a" 
expecting "b"
What happened is that char 'a' succeeded by consuming the 'a' from the 
input and then lookAhead rewinded the input stream (as it does on 
success). Then, char 'b' tries to parse (again) the first character of 
the input and fails. Everything works as expected.
Now let's slightly modify our parser:
p' = lookAhead (many1 $ char 'a') >> char 'b'
I've only added a many1. I was expecting this parser to give the same 
error as the previous one: many1 $ char 'a' will succeed consuming one 
'a' and then lookAhead will rewind the input (as it does on success). 
Thus when we call char 'b' we are going to be in the beginning of the 
input again. Well, that doesn't happen:
Left (line 1, column 2): 
unexpected end of input 
expecting "b"
As you can see, lookAhead did not rewind the input as it was supposed to. 
_______________________________________________ 
Haskell-Cafe mailing list 
Haskell-Cafe@haskell.org 
http://www.haskell.org/mailman/listinfo/haskell-cafe
------------------------------ 

Message: 5 
Date: Thu, 7 Aug 2014 10:57:47 -0700 
From: Kyle Hanson  
To: haskell-cafe@haskell.org 
Subject: [Haskell-cafe] Performance of StateT and best practices for 
debugging 
Message-ID: 

Content-Type: text/plain; charset="utf-8" 

Hello, 

I was looking at cleaning up my refactoring a core loop of template 
rendering to go from a loop with many parameters 

loop :: RenderConfig -> BlockMap -> InputBucket m -> Builder -> [Pieces] -> 
ExceptT StrapError m Builder 

to a looped state monad transformer 

loop :: [Pieces] -> RenderT m Builder 

newtype RenderT m a = RenderT 
{ runRenderT :: ExceptT StrapError (StateT (RenderState m) m) a 
} deriving ( Functor, Applicative, Monad, MonadIO ) 

data RenderState m = RenderState 
{ position :: SourcePos 
, renderConfig :: RenderConfig 
, blocks :: BlockMap 
, bucket :: InputBucket m 
} 

however, there is a big slow down (about 6-10x) using a StateT. I think it 
might have something to do with laziness but I am not exactly sure of where 
to begin in tracking it down. Swapping out the Lazy State to a Strict State 
helps a little (only a 5x slow down) 

You can find some of the processing code here: 

https://github.com/hansonkd/StrappedTemplates/blob/321a88168d54943fc217553c8... 

With my old loop commented out. 

Its messy right now since I am just trying a number of different 
approaches. I did some more work factoring out the lifts, trying different 
iterations of foldlM and stuff but that didn't have that much of an effect 
on performance. 

After profiling I see in the StateT, the report has a lot more CAFs and 
garbage collecting. 

Here is the profiling report from my original version w/o StateT 
http://lpaste.net/108995 

Slow version with StateT 
http://lpaste.net/108997 

Here is the "makeBucket" function that is referenced (it is the same in 
both state and nonstate): 

https://github.com/hansonkd/StrappedTemplates/blob/321a88168d54943fc217553c8... 

Looking at stacked overflow and the official docs I have gotten an idea of 
what is going on. The heaps generated between them tells me that a lot more 
memory is being allocated to lists. These heaps were generated running my 
render function against a template with nested loops and a list of elements. 

http://imgur.com/a/2jOIf 

I am hoping that maybe someone could give me a hint at what to look at 
next. I've played around with Strictness and refactoring loops to no avail 
and now am kind of stuck. Any help would be appreciated. 

-- 
Kyle Hanson 
-------------- next part -------------- 
An HTML attachment was scrubbed... 
URL: http://www.haskell.org/pipermail/haskell-cafe/attachments/20140807/a82b99f5/... 

------------------------------ 

Message: 6 
Date: Thu, 07 Aug 2014 21:07:02 +0200 
From: Mateusz Kowalczyk  
To: haskell-cafe@haskell.org 
Subject: [Haskell-cafe] [ANN] rtorrent-state 0.1.0.0 
Message-ID: <53E3CE56.6090500@fuuzetsu.co.uk> 
Content-Type: text/plain; charset=windows-1252 

Hi, 

rtorrent-state is a library that allows working with rtorrent state 
files (SOMEHASH.torrent.rtorrent) placed in your session directory. 

If you're an rtorrent user and ever had to manually muck around with 
those files, you should be able to use this library to make your life 
easier. 

For example, you can stop all torrents in your session directory with 
just: ?overFilesIn "rtorrent/session/dir" stopTorrent? 

The way it works is by parsing the session files, modifying the 
resulting data type and serialising it back into the file. I did not do 
optimisation but I had no problem with test sample of 100,000 files. 

I need to add IOException handling and maybe extra utility functions but 
otherwise I consider the library finished. 

Thanks 
-- 
Mateusz K. 

------------------------------ 

Message: 7 
Date: Thu, 7 Aug 2014 15:39:53 -0700 
From: John Lato  
To: Kyle Hanson  
Cc: haskell-cafe  
Subject: Re: [Haskell-cafe] Performance of StateT and best practices 
for	debugging 
Message-ID: 

Content-Type: text/plain; charset="utf-8" 

I haven't looked very closely, but I'm suspicious of this code from 
"instance Block Piece" 

ListLike l -> forM l (\obj -> ...)
...
...
= (return . mconcat)
The "forM" means that "l" will be traversed once and create an output list, 
which will then be mconcat'd together. The list has to be created because 
of the monadic structure imposed by forM, but if the result of the mconcat 
isn't demanded right away it will be retained as a thunk that references 
the newly-created list. 

I'd suggest that you replace it with something like 

ListLike l -> foldM (\(!acc) obj -> ... >>= return . mappend acc) mempty l 

Here I've justed added a bang pattern to the accumulator. If whatever is 
being returned has some lazy fields, you may want to change that to use 
deepseq instead of a bang pattern. 

Also, "foo >>= return . bar" is often regarded as a bit of a code smell, it 
can be replaced with "bar <$> foo" or "bar `liftM` foo", or sometimes 
something even simpler depending on circumstances (but IMHO sometimes it's 
more clear to just leave it alone). 

The heap profile does look like a space leak. The line 

StrappedTemplates-0.1.1.0:Text.Strapped.Render.sat_sc1z 

is a thunk (you can tell because it's in '<>' brackets), so whatever is 
referencing that is not strict enough. Sometimes another heap profile 
report, e.g. "-hc" or maybe "-hy" will give more useful information that 
lets you identify what exactly "sat_sc1z" is. You could also try compiling 
with -ddump-stg, which will dump the intermediate STG output which usually 
shows those names. But then you'll probably also need to re-run the 
profile, since the names change between compilations. Also IIRC some of 
values aren't named until the cmm phase, but that's harder to map back to 
Haskell so if you can identify the code from stg it's simpler. 

If you haven't seen 
http://blog.ezyang.com/2011/06/pinpointing-space-leaks-in-big-programs/, 
I'd highly recommend it if you need to track down a space leak. 

John L. 

On Thu, Aug 7, 2014 at 10:57 AM, Kyle Hanson  wrote:
...
Hello,
I was looking at cleaning up my refactoring a core loop of template 
rendering to go from a loop with many parameters
loop :: RenderConfig -> BlockMap -> InputBucket m -> Builder -> [Pieces] 
-> ExceptT StrapError m Builder
to a looped state monad transformer
loop :: [Pieces] -> RenderT m Builder
newtype RenderT m a = RenderT 
{ runRenderT :: ExceptT StrapError (StateT (RenderState m) m) a 
} deriving ( Functor, Applicative, Monad, MonadIO )
data RenderState m = RenderState 
{ position :: SourcePos 
, renderConfig :: RenderConfig 
, blocks :: BlockMap 
, bucket :: InputBucket m 
}
however, there is a big slow down (about 6-10x) using a StateT. I think it 
might have something to do with laziness but I am not exactly sure of where 
to begin in tracking it down. Swapping out the Lazy State to a Strict State 
helps a little (only a 5x slow down)
You can find some of the processing code here:
https://github.com/hansonkd/StrappedTemplates/blob/321a88168d54943fc217553c8...
With my old loop commented out.
Its messy right now since I am just trying a number of different 
approaches. I did some more work factoring out the lifts, trying different 
iterations of foldlM and stuff but that didn't have that much of an effect 
on performance.
After profiling I see in the StateT, the report has a lot more CAFs and 
garbage collecting.
Here is the profiling report from my original version w/o StateT 
http://lpaste.net/108995
Slow version with StateT 
http://lpaste.net/108997
Here is the "makeBucket" function that is referenced (it is the same in 
both state and nonstate):
https://github.com/hansonkd/StrappedTemplates/blob/321a88168d54943fc217553c8...
Looking at stacked overflow and the official docs I have gotten an idea of 
what is going on. The heaps generated between them tells me that a lot more 
memory is being allocated to lists. These heaps were generated running my 
render function against a template with nested loops and a list of elements.
http://imgur.com/a/2jOIf
I am hoping that maybe someone could give me a hint at what to look at 
next. I've played around with Strictness and refactoring loops to no avail 
and now am kind of stuck. Any help would be appreciated.
-- 
Kyle Hanson
_______________________________________________ 
Haskell-Cafe mailing list 
Haskell-Cafe@haskell.org 
http://www.haskell.org/mailman/listinfo/haskell-cafe
-------------- next part -------------- 
An HTML attachment was scrubbed... 
URL: http://www.haskell.org/pipermail/haskell-cafe/attachments/20140807/b4c26366/... 

------------------------------ 

Message: 8 
Date: Fri, 8 Aug 2014 11:24:29 +0800 
From: jun zhang  
To: haskell-cafe@haskell.org 
Subject: [Haskell-cafe] How to improve the zipwith's performance 
Message-ID: 

Content-Type: text/plain; charset="utf-8" 

Dear All 

I write a code for Clustering with Data.Clustering.Hierarchical, but it's 
slow. 

I use the profiling and change some code, but I don't know why zipwith take 
so many time? (even I change list to vector) 

My code is as blow, Any one kindly give me some advices. 
====================== 
main = do 
.... 
let cluster = dendrogram SingleLinkage vectorList getVectorDistance 
.... 

getExp2 v1 v2 = d*d 
where 
d = v1 - v2 

getExp v1 v2 
| v1 == v2 = 0 
| otherwise = getExp2 v1 v2 

tfoldl d = DV.foldl1' (+) d 

changeDataType:: Int -> Double 
changeDataType d = fromIntegral d 

getVectorDistance::(a,DV.Vector Int)->(a, DV.Vector Int )->Double 
getVectorDistance v1 v2 = fromIntegral $ tfoldl dat 
where 
l1 = snd v1 
l2 = snd v2 
dat = DV.zipWith getExp l1 l2 

======================================= 

build with ghc -prof -fprof-auto -rtsopts -O2 log_cluster.hs 

run with log_cluster.exe +RTS -p 

profiling result is 

log_cluster.exe +RTS -p -RTS 

total time = 8.43 secs (8433 ticks @ 1000 us, 1 processor) 
total alloc = 1,614,252,224 bytes (excludes profiling overheads) 

COST CENTRE MODULE %time %alloc 

getVectorDistance.dat Main 49.4 37.8 
tfoldl Main 5.7 0.0 
getExp Main 4.5 0.0 
getExp2 Main 0.5 1.5 
-------------- next part -------------- 
An HTML attachment was scrubbed... 
URL: http://www.haskell.org/pipermail/haskell-cafe/attachments/20140808/d29aa298/... 

------------------------------ 

Message: 9 
Date: Fri, 08 Aug 2014 06:31:49 +0200 
From: Bardur Arantsson  
To: haskell-cafe@haskell.org 
Subject: Re: [Haskell-cafe] Performance of StateT and best practices 
for	debugging 
Message-ID:  
Content-Type: text/plain; charset=utf-8 

On 2014-08-07 19:57, Kyle Hanson wrote:
...
Hello,
Here is the "makeBucket" function that is referenced (it is the same in 
both state and nonstate):
https://github.com/hansonkd/StrappedTemplates/blob/321a88168d54943fc217553c8...
Just a shot in the dark, but I notice that you're using "modify" and not 
"modify'" which was added in a recent version of transformers. 

Strict.StateT is not always "strict enough" and you may need to use modify'. 

At any rate, it's worth a shot, I think. 

Regards, 

------------------------------ 

Message: 10 
Date: Thu, 7 Aug 2014 22:30:25 -0700 
From: Jan Paul Posma  
To: haskell-cafe@haskell.org 
Subject: [Haskell-cafe] Visualising Haskell function execution 
Message-ID: 

Content-Type: text/plain; charset="utf-8" 

Hey all, 

Last weekend my friend Steve and I did a small project for visualising 
Haskell function execution in the browser. It's meant to be used in 
education, and uses a tiny custom parser. I figured it could be of interest 
for anyone here learning or teaching Haskell: 
https://stevekrouse.github.io/hs.js/ 

To see it in action, scroll a bit down to the red bordered box, and click 
on "map", and then keep clicking on each new line. 

I hope it can be useful to someone. 

Cheers, JP 
-------------- next part -------------- 
An HTML attachment was scrubbed... 
URL: http://www.haskell.org/pipermail/haskell-cafe/attachments/20140807/3c4dabe9/... 

------------------------------ 

Message: 11 
Date: Thu, 7 Aug 2014 23:56:43 -0700 
From: John Lato  
To: Bardur Arantsson  
Cc: haskell-cafe  
Subject: Re: [Haskell-cafe] Performance of StateT and best practices 
for	debugging 
Message-ID: 

Content-Type: text/plain; charset="utf-8" 

On Thu, Aug 7, 2014 at 9:31 PM, Bardur Arantsson  
wrote:
...
On 2014-08-07 19:57, Kyle Hanson wrote:
...
Hello,
Here is the "makeBucket" function that is referenced (it is the same in 
both state and nonstate):
https://github.com/hansonkd/StrappedTemplates/blob/321a88168d54943fc217553c8...
...
Just a shot in the dark, but I notice that you're using "modify" and not 
"modify'" which was added in a recent version of transformers.
Strict.StateT is not always "strict enough" and you may need to use 
modify'.
At any rate, it's worth a shot, I think.
Good point. I think that even modify' will not be strict enough without 
adding strictness to RenderState as well. 

John L. 
-------------- next part -------------- 
An HTML attachment was scrubbed... 
URL: http://www.haskell.org/pipermail/haskell-cafe/attachments/20140807/ce617687/... 

------------------------------ 

Subject: Digest Footer 

_______________________________________________ 
Haskell-Cafe mailing list 
Haskell-Cafe@haskell.org 
http://www.haskell.org/mailman/listinfo/haskell-cafe 

------------------------------ 

End of Haskell-Cafe Digest, Vol 132, Issue 11 
*********************************************

Matt Giles

tags

participants (1)