Hi, Am Freitag, den 02.03.2007, 09:25 -0800 schrieb David Roundy:

echo hello world | hmap toUpper

$ echo hello world | hmap 'map Char.toUpper' HELLO WORLD (Whereas hmap is as defined in the original blog entry) Greetings, Joachim -- Joachim Breitner e-Mail: mail@joachim-breitner.de Homepage: http://www.joachim-breitner.de ICQ#: 74513189

On Fri, Mar 02, 2007 at 08:12:19PM +0100, Thomas Hartman wrote:

Great tip! One question though.

What condition is

if which ghc >/dev/null

checking?

What bad thing will happen if you don't do this check?

Sorry this is more a bash question than a haskell question.

It just checks whether ghc is in the path. -- David Roundy Department of Physics Oregon State University

There's some nice one liners bundled with h4sh: http://www.cse.unsw.edu.au/~dons/h4sh.html For example: http://www.cse.unsw.edu.au/~dons/h4sh.txt If you recall, h4sh is a set of unix wrappers to the list library. I still use them everyday, though probably should put out a new release soon. -- Don tphyahoo:

To answer my original question, here's a few ways to accomplish what I wanted with haskell

Perl is still a lot faster than ghc -e, but I guess if you wanted speed you could compile first.

********************************************************************

thartman@linodewhyou:~/learning/haskell/UnixTools$ ls -l total 16 -rw-r--r-- 1 thartman thartman 2726 Dec 20 07:56 UnixTools.hs -rw-r--r-- 1 thartman thartman 82 Jan 7 07:18 echo.hs -rwxr--r-- 1 thartman thartman 790 Mar 4 05:02 oneliners.sh -rwxr--r-- 1 thartman thartman 646 Mar 4 04:18 oneliners.sh~

thartman@linodewhyou:~/learning/haskell/UnixTools$ ./oneliners.sh haskell, ghc -e pipe 16

real 0m1.652s user 0m0.600s sys 0m0.030s ********** haskell, hmap pipe 16

real 0m1.549s user 0m0.410s sys 0m0.200s ********** haskell, two pipes 16

real 0m2.153s user 0m0.900s sys 0m0.370s ********** perl, two pipes 16

real 0m0.185s user 0m0.010s sys 0m0.100s

thartman@linodewhyou:~/learning/haskell/UnixTools$

thartman@linodewhyou:~/learning/haskell/UnixTools$ cat oneliners.sh hmap (){ ghc -e "interact ($*)"; } hmapl (){ hmap "unlines.($*).lines" ; } hmapw (){ hmapl "map (unwords.($*).words)" ; }

function filesizes () { find -maxdepth 1 -type f | xargs du }

echo haskell, ghc -e pipe time filesizes | ghc -e 'interact $ (++"\n") . show . sum . map ( ( read :: String -> Integer ) . head . words ) . lines ' echo "**********"

echo haskell, hmap pipe time filesizes | hmap '(++"\n") . show . sum . map ( ( read :: String -> Integer ) . head . words ) . lines' echo "**********"

echo haskell, two pipes time filesizes | hmapl "map ( head . words )" | hmap '(++"\n") . show . sum . map ( read :: String -> Integer ) . lines' echo "**********"

echo perl, two pipes time filesizes | perl -ane 'print "$F[0]\n"' | perl -e '$sum += $_ while <>; print "$sum\n"'

2007/3/2, Thomas Hartman :

...

Okay, I am aware of

http://haskell.org/haskellwiki/Simple_unix_tools

which gives some implementation of simple unix utilities in haskell.

But I couldn't figure out how to use them directly from the shell, and of course that's what most readers will probably wnat.

Or let me put it another way.

Is there a way to do

find -maxdepth 1 -type f | xargs du | perl -ane 'print "\$F[0]\n"' | perl -e '$sum += $_ while <>; print "$sum\n"'

as a shell command that idiomatically uses haskell?

For non-perlers, that sums up the disk usage of all files in the current directory, skipping subdirs.

print "\$F[0]\n

looks at the first (space delimited) collumn of output.

perl -e '$sum += $_ while <>; print "$sum\n"'

, which is I guess the meat of the program, sums up all the numbers spewed out of the first column, so in the end you get a total.

So, anyone out there want to establish a haskell one liner tradition?

:)

thomas.

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Yes, it definitely is a little lagged. It should be ported to use lazy bytestrings too. I wsa more suggesting the one liners as examples of haskell use in the shell. tphyahoo:

That seems like a really great thing to have. But I had troubles installing it.

h4sh depends on hs-plugins.

And... **************** thartman@linodewhyou:~/haskellInstalls/hs-plugins$ ./Setup.lhs configure Setup.lhs: Warning: The field "hs-source-dir" is deprecated, please use hs-source-dirs. Configuring plugins-1.0... configure: /usr/local/bin/ghc-pkg configure: Dependency base-any: using base-2.0 configure: Dependency Cabal-any: using Cabal-1.1.6 Setup.lhs: cannot satisfy dependency haskell-src-any thartman@linodewhyou:~/haskellInstalls/hs-plugins$ **************** Advice?

2007/3/4, Donald Bruce Stewart :

...

There's some nice one liners bundled with h4sh:

http://www.cse.unsw.edu.au/~dons/h4sh.html

For example:

http://www.cse.unsw.edu.au/~dons/h4sh.txt

If you recall, h4sh is a set of unix wrappers to the list library. I still use them everyday, though probably should put out a new release soon.

-- Don

tphyahoo:

...

To answer my original question, here's a few ways to accomplish what I wanted with haskell

Perl is still a lot faster than ghc -e, but I guess if you wanted speed you could compile first.

********************************************************************

thartman@linodewhyou:~/learning/haskell/UnixTools$ ls -l total 16 -rw-r--r-- 1 thartman thartman 2726 Dec 20 07:56 UnixTools.hs -rw-r--r-- 1 thartman thartman 82 Jan 7 07:18 echo.hs -rwxr--r-- 1 thartman thartman 790 Mar 4 05:02 oneliners.sh -rwxr--r-- 1 thartman thartman 646 Mar 4 04:18 oneliners.sh~

thartman@linodewhyou:~/learning/haskell/UnixTools$ ./oneliners.sh haskell, ghc -e pipe 16

real 0m1.652s user 0m0.600s sys 0m0.030s ********** haskell, hmap pipe 16

real 0m1.549s user 0m0.410s sys 0m0.200s ********** haskell, two pipes 16

real 0m2.153s user 0m0.900s sys 0m0.370s ********** perl, two pipes 16

real 0m0.185s user 0m0.010s sys 0m0.100s

thartman@linodewhyou:~/learning/haskell/UnixTools$

thartman@linodewhyou:~/learning/haskell/UnixTools$ cat oneliners.sh hmap (){ ghc -e "interact ($*)"; } hmapl (){ hmap "unlines.($*).lines" ; } hmapw (){ hmapl "map (unwords.($*).words)" ; }

function filesizes () { find -maxdepth 1 -type f | xargs du }

echo haskell, ghc -e pipe time filesizes | ghc -e 'interact $ (++"\n") . show . sum . map ( ( read :: String -> Integer ) . head . words ) . lines ' echo "**********"

echo haskell, hmap pipe time filesizes | hmap '(++"\n") . show . sum . map ( ( read :: String -> Integer ) . head . words ) . lines' echo "**********"

echo haskell, two pipes time filesizes | hmapl "map ( head . words )" | hmap '(++"\n") . show . sum . map ( read :: String -> Integer ) . lines' echo "**********"

echo perl, two pipes time filesizes | perl -ane 'print "$F[0]\n"' | perl -e '$sum += $_ while <>; print "$sum\n"'

2007/3/2, Thomas Hartman :

...

Okay, I am aware of

http://haskell.org/haskellwiki/Simple_unix_tools

which gives some implementation of simple unix utilities in haskell.

But I couldn't figure out how to use them directly from the shell, and of course that's what most readers will probably wnat.

Or let me put it another way.

Is there a way to do

find -maxdepth 1 -type f | xargs du | perl -ane 'print "\$F[0]\n"' | perl -e '$sum += $_ while <>; print "$sum\n"'

as a shell command that idiomatically uses haskell?

For non-perlers, that sums up the disk usage of all files in the current directory, skipping subdirs.

print "\$F[0]\n

looks at the first (space delimited) collumn of output.

perl -e '$sum += $_ while <>; print "$sum\n"'

, which is I guess the meat of the program, sums up all the numbers spewed out of the first column, so in the end you get a total.

So, anyone out there want to establish a haskell one liner tradition?

:)

thomas.

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

I think I could have most of the oneliner goodness of h4sh, without having to do the module install, if I could figure out a way to include modules with ghc -e.

i confess myself to be among those who underappreciated ghc -e, until this thread:) as Joachim said (thanks for starting this, btw;-), we can use qualified names $ echo hello world | hmap 'map Char.toUpper' HELLO WORLD and to get at your other methods, the question is not how to include modules with ghc -e; instead, recall that ghc -e supplies a command to be evaluated within the context of its parameter module (single input for a ghci session): $ cat Imports.hs import Debug.Trace helper x = trace "hi there" (x+1) $ ghc -e 'helper 41' Imports.hs hi there 42 as to the original question in this thread, my .bashrc now also has a few less symmetric entries: function hrunl { ghc -e "interact(show.($*).lines)"; } function hrunw { ghc -e "interact(show.($*).words)"; } function hrunwl { ghc -e "interact(show.($*).map words.lines)"; } using which the one-liner becomes something like find -maxdepth 1 -type f | xargs du | hrunwl "sum . map (read . head)" (the find/du is better left in shell tool land, it seems, and default Num is Integer) hth, claus

Can't seem to find these packages. Do I need to add another repo? Or build from source? thartman@linodewhyou:~/haskellInstalls/hs-plugins$ apt-cache search libghc6 | grep ^libghc6-haskell libghc6-haskelldb-dev - Haskell library for expressing database queries thartman@linodewhyou:~/haskellInstalls/hs-plugins$ cat /etc/apt/sources.list deb http://archive.ubuntu.com/ubuntu/ dapper main restricted deb-src http://archive.ubuntu.com/ubuntu/ dapper main restricted ## Uncomment the following two lines to add software from the 'universe' ## repository. ## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu ## team, and may not be under a free licence. Please satisfy yourself as to ## your rights to use the software. Also, please note that software in ## universe WILL NOT receive any review or updates from the Ubuntu security ## team. deb http://archive.ubuntu.com/ubuntu/ dapper universe deb-src http://archive.ubuntu.com/ubuntu/ dapper universe deb http://security.ubuntu.com/ubuntu dapper-security main restricted deb-src http://security.ubuntu.com/ubuntu dapper-security main restricted 2007/3/4, Brandon S. Allbery KF8NH :

On Mar 4, 2007, at 6:31 , Thomas Hartman wrote:

...

Setup.lhs: cannot satisfy dependency haskell-src-any

Used to be bundled, now unbundled. On debian/ubuntu check your libghc6-*-dev packages. (libghc6-haskell-src-dev?)

-- brandon s. allbery [linux,solaris,freebsd,perl] allbery@kf8nh.com system administrator [openafs,heimdal,too many hats] allbery@ece.cmu.edu electrical and computer engineering, carnegie mellon university KF8NH

with PCRE. now (and yep, it's a lot faster than posix): *************************************************************************** thartman@linodewhyou:~/learning/haskell/UnixTools$ ./q-words.sh q qua quack quacked quackery quackery's quacking quacks quad quad's real 0m6.691s user 0m2.460s sys 0m3.000s q qua quack quacked quackery quackery's quacking quacks quad quad's real 0m1.032s user 0m0.750s sys 0m0.100s thartman@linodewhyou:~/learning/haskell/UnixTools$ cat q-words.sh time cat /usr/share/dict/american-english | ghc -e 'interact $ unlines. take 10 . filter ( \x -> x =~ "^q" :: Bool ) . lines' ImportsRegexPosix.hs time cat /usr/share/dict/american-english | ghc -e 'interact $ unlines. take 10 . filter ( \x -> x =~ "^q" :: Bool ) . lines' ImportsRegexPCRE.hs thartman@linodewhyou:~/learning/haskell/UnixTools$ cat ImportsRegexPosix.hs import Text.Regex.Posix thartman@linodewhyou:~/learning/haskell/UnixTools$ cat ImportsRegexPCRE.hs import Text.Regex.PCRE thartman@linodewhyou:~/learning/haskell/UnixTools$ 2007/3/7, Thomas Hartman :

In the spirit of making easy things easy, here is a "haskell from shell" one-line grepper, that uses regexen.

Now, if only I could get pcre-regex installed I would be quite content. (Still stuck using posix RE for now.)

**************

thartman@linodewhyou:~/learning/haskell/UnixTools$ time ./q-words.sh q qua quack quacked quackery quackery's quacking quacks quad quad's

real 0m3.186s user 0m1.900s sys 0m0.810s

thartman@linodewhyou:~/learning/haskell/UnixTools$ cat q-words.sh cat /usr/share/dict/american-english | ghc -e 'interact $ unlines. take 10 . filter ( \x -> x =~ "^q" :: Bool ) . lines' Imports.hs

thartman@linodewhyou:~/learning/haskell/UnixTools$ cat Imports.hs import Text.Regex.Posix

****************************

2007/3/2, Thomas Hartman :

...

Okay, I am aware of

http://haskell.org/haskellwiki/Simple_unix_tools

which gives some implementation of simple unix utilities in haskell.

But I couldn't figure out how to use them directly from the shell, and of course that's what most readers will probably wnat.

Or let me put it another way.

Is there a way to do

find -maxdepth 1 -type f | xargs du | perl -ane 'print "\$F[0]\n"' | perl -e '$sum += $_ while <>; print "$sum\n"'

as a shell command that idiomatically uses haskell?

For non-perlers, that sums up the disk usage of all files in the current directory, skipping subdirs.

print "\$F[0]\n

looks at the first (space delimited) collumn of output.

perl -e '$sum += $_ while <>; print "$sum\n"'

, which is I guess the meat of the program, sums up all the numbers spewed out of the first column, so in the end you get a total.

So, anyone out there want to establish a haskell one liner tradition?

:)

thomas.

Just noticed a comment in http://www.serpentine.com/blog/2007/02/27/a-haskell-regular-expression-tutor... which says there's no perl-like regex replace in the library, and links to http://hpaste.org/697 which is an attempt at providing one. Not sure if this is useful or not. 2007/3/7, Thomas Hartman :

Okay, so much for PCRE match.

Can someone show me give me pointers on PCRE replace?

Ideal would be something with all the =~ s/// semantics from perl. (Not sure if this is included in Text.Regex.PCRE. is it?)

In other words, how to do this with (preferrably) Text.Regex.PCRE ?

Of course in this case a POSIX regex is fine, but since PCRE is faster and more expressive, and what I'm used to anyway, PCRE is probably is what I will tend to use by default, unless portability is paramount.

**************************** thartman@linodewhyou:~/learning/haskell/UnixTools$ cat lines.txt a b c a b c a b c

thartman@linodewhyou:~/learning/haskell/UnixTools$ cat lines.txt | perl -ne '$_ =~ s/(\w) (\w) (\w)/$1 z $3/; print $_' a z c a z c a z c thartman@linodewhyou:~/learning/haskell/UnixTools$

Just thought I'd add another potentially helpful bit to this oneliner / shell scripting thread. Though to be fair, this perhaps strains the definition of "one" in "one liner". This takes a long file containing mostly numerical data, filters out the numerical data, and sorts it numerically. (Not the same thing as sorting alphabetically, which is what you get by default, or using the unix sort utility). Maybe there's some flag to make unix sort util act like this? Enh, who cares, now we have haskell. :) Thanks to Thunder, Quicksilver, and whoever else it was on #haskell who helped me out with this. ****************************************************** thartman@linodewhyou:~/learning/haskell/UnixTools>cat sortNumeric.sh | head -n10 cat out_select_char_length_csv.out | ghc -e ' interact $ unlines . map show -- more efficient than -- reverse . take 10 . reverse . ( \s -> drop (length s - 10 ) s ) . Data.List.sort -- maybe not necessary? . map ( read :: String -> Integer ) . Data.Set.toAscList . Data.Set.fromList -- more efficient than prelude nub . filter ( all Data.Char.isDigit ) . lines' 2007/3/7, Chris Kuklewicz :

Thomas Hartman wrote:

...

Just noticed a comment in

http://www.serpentine.com/blog/2007/02/27/a-haskell-regular-expression-tutor...

which says there's no perl-like regex replace in the library, and links to

http://hpaste.org/697

which is an attempt at providing one.

Not sure if this is useful or not.

Any given replacement routine is less than 10 lines of code and will do exactly what you need.

A general replacement library has to contend with several things: 1a) What syntax/semantics? 1b) How do you supply a specification? Must it be the same type as the regular expression or the data? 1c) How do you report errors? 2) Which regex-* backends to support? 3) What types to work on? [Char], Seq Char, ByteString, Lazy ByteString. 4a) If the backend/type supports lazy matching then does the replacing? 4b) What if the backend/type does not support lazy match or strictness is desired? 5) If there is laziness then can it handle infinite streams of input? 6) Is anyone smart enough to design this API without actual users?

Note that some approaches allow for much more efficiency than others. Taking a normal ByteString and performing replacement to create a Lazy ByteString makes sense, but is a bit of wrinkle.

But as you pointed to on http://hpaste.org/697 any given example of a replacement routine will be very small, and easy to build on top of the regex-* API.

Same thing, but with better comments and whitespace. thartman@linodewhyou:~/learning/haskell/UnixTools>head -n30 sortNumeric.sh cat out_select_char_length_csv.out | ghc -e ' interact $ unlines -- take 10 from the end -- more efficient than -- reverse . take 10 . reverse . ( \s -> drop (length s - 10 ) s ) . map show -- convert Integer to String -- sort numerically . Data.List.sort . map ( read :: String -> Integer ) -- convert String to Integer -- Uniqify -- more efficient than prelude nub -- sorts too, but alphanumerically, whereas we want numerically . Data.Set.toAscList . Data.Set.fromList . filter ( all Data.Char.isDigit ) . lines ' 2007/3/20, Thomas Hartman :

To answer my own post, the Data.List.sort *is* necessary.

Otherwise, you get alphabetic sort.

2007/3/20, Thomas Hartman :

...

Just thought I'd add another potentially helpful bit to this oneliner / shell scripting thread. Though to be fair, this perhaps strains the definition of "one" in "one liner".

This takes a long file containing mostly numerical data, filters out the numerical data, and sorts it numerically. (Not the same thing as sorting alphabetically, which is what you get by default, or using the unix sort utility). Maybe there's some flag to make unix sort util act like this? Enh, who cares, now we have haskell. :)

Thanks to Thunder, Quicksilver, and whoever else it was on #haskell who helped me out with this.

******************************************************

thartman@linodewhyou:~/learning/haskell/UnixTools>cat sortNumeric.sh | head -n10 cat out_select_char_length_csv.out | ghc -e ' interact $ unlines . map show

-- more efficient than -- reverse . take 10 . reverse . ( \s -> drop (length s - 10 ) s )

. Data.List.sort -- maybe not necessary? . map ( read :: String -> Integer ) . Data.Set.toAscList . Data.Set.fromList -- more efficient than prelude nub . filter ( all Data.Char.isDigit ) . lines'

2007/3/7, Chris Kuklewicz :

...

Thomas Hartman wrote:

...

Just noticed a comment in

http://www.serpentine.com/blog/2007/02/27/a-haskell-regular-expression-tutor...

which says there's no perl-like regex replace in the library, and links to

http://hpaste.org/697

which is an attempt at providing one.

Not sure if this is useful or not.

Any given replacement routine is less than 10 lines of code and will do exactly what you need.

A general replacement library has to contend with several things: 1a) What syntax/semantics? 1b) How do you supply a specification? Must it be the same type as the regular expression or the data? 1c) How do you report errors? 2) Which regex-* backends to support? 3) What types to work on? [Char], Seq Char, ByteString, Lazy ByteString. 4a) If the backend/type supports lazy matching then does the replacing? 4b) What if the backend/type does not support lazy match or strictness is desired? 5) If there is laziness then can it handle infinite streams of input? 6) Is anyone smart enough to design this API without actual users?

Note that some approaches allow for much more efficiency than others. Taking a normal ByteString and performing replacement to create a Lazy ByteString makes sense, but is a bit of wrinkle.

But as you pointed to on http://hpaste.org/697 any given example of a replacement routine will be very small, and easy to build on top of the regex-* API.

Quoth Thomas Hartman, nevermore,

This takes a long file containing mostly numerical data, filters out the numerical data, and sorts it numerically. (Not the same thing as sorting alphabetically, which is what you get by default, or using the unix sort utility). Maybe there's some flag to make unix sort util act like this? Enh, who cares, now we have haskell. :)

Alas, foiled again! ;-) sort flag: -n, --numeric-sort D. -- Dougal Stanton