On 6/22/07, Jeremy Shaw wrote:

Hello,

Have you seen Tom Moertel's series on directory-tree printing in Haskell ?

No, I hadn't. Might be just the ticket. Thanks!

tittoassini:

Hi,

to load an Haskell symbol at run-time is still necessary to use the load functions from the hs-plugins library (System.Plugins.Load) or is there some function in the GHC API that does the same job?

yes, definitely possible. i think Lemmih put an example on the wiki a while ago. basically, ghc-api exposes the lower level api also used by hs-plugins -- a nice project would be to provide the hs-plugins api directly in ghc-api - avoiding the need for an external hs-plugins package. -- Don

tittoassini:

On Wednesday 27 June 2007 03:06:15 Donald Bruce Stewart wrote:

...

tittoassini:

...

Hi,

to load an Haskell symbol at run-time is still necessary to use the load functions from the hs-plugins library (System.Plugins.Load) or is there some function in the GHC API that does the same job?

yes, definitely possible. i think Lemmih put an example on the wiki a while ago. basically, ghc-api exposes the lower level api also used by hs-plugins -- a nice project would be to provide the hs-plugins api directly in ghc-api - avoiding the need for an external hs-plugins package.

-- Don

Hi Don, thanks for the answer.

I checked again the http://haskell.org/haskellwiki/GHC/As_a_library page on the wiki and there are examples of interactive evaluation but I cannot find an example of loading a symbol from a compiled module.

Yes, this is the part I meant about duplicating the hs-plugins API. Both ghc-apii and hs-plugins use the following api from the runtime: foreign import ccall unsafe "Linker.h lookupSymbol" pluginSym :: CString -> IO (Ptr a) foreign import ccall unsafe "Linker.h loadObj" pluginLoad :: CString -> IO Bool foreign import ccall unsafe "Linker.h initLinker" pluginInit :: IO () foreign import ccall unsafe "Linker.h resolveObjs" pluginResolve :: IO Bool hs-plugins duplicates some of the ghci internals to find, resolve and load packages and objects, and access their symbols. So that code must exist in the ghci parts of the ghc-api. You'll have to peek around in there. -- Don

On Saturday 23 June 2007 08:18, Chad Scherrer wrote:

Now, the UNIX command doesn't really cut it, because it complains there are too many files,

Sounds like a case for xargs(1)? Alexis.

Thanks, Bryan, this is much cleaner than the imperative hack I was throwing together. And aside from the imports, it even fits the "couple lines of code" criteria! Wonderful. I won't be able to try this out until I get back to work, but I'm wondering whether this will handle a few thousand files. As it is, even the "gunzip -c ./2*/*.z" I'm trying to emulate doesn't really work as is, because the OS complains there are too many pattern matches. Does "namesMatching" just feed the pattern to the OS, or does it match the pattern itself? Chad

...

What got me thinking about this is I'd like to be able to do something like this in just a couple lines of code:

gunzip -c ./2*/*.z

... and feed the result into a giant lazy ByteString.

Using my FileManip library, you'd do that like this.

import Codec.Compression.GZip import qualified Data.ByteString.Lazy as B import System.FilePath.Glob

foo :: IO B.ByteString foo = namesMatching "*/*.gz" >>= fmap B.concat . mapM (fmap decompress . B.readFile)

I also tried gzipping a different, smaller file, and I changed the string in "bar" accordingly. No error in that case. So it seems to be a problem with myData.z, but why would it gunzip from the command line with no trouble in that case?

Thanks, Chad

Because gunzip is smarter than your program in that he can decompress gzip format but Z format too (which is produced by the very old "compress" unix utility). -- Jedaï

On Mon, Jun 25, 2007 at 12:48:27PM -0700, Chad Scherrer wrote:

Jedaï,

Are you sure you're not confusing .z with .Z?

http://kb.iu.edu/data/afcc.html

And is it possible that gzip is smarter somehow? Doesn't Codec.Compression.GZip call the same C library used by gzip?

gzip: supports gzip, pack, compress zlib doesn't. http://www.zlib.net/zlib_faq.html#faq12 You ought to just use newpopen or similar (http://www.cse.unsw.edu.au/~dons/code/newpopen) Stefan

Bulat, I don't think I can. (1) (de)compress is defined for lazy bytestrings, and (2) my data comes to me compressed in order to fit it all on a single DVD. So even if I could uncompress each file strictly, I couldn't hold such a big strict bytestring in memory at once. On 6/25/07, Bulat Ziganshin wrote:

...

bar = fmap decompress $ B.readFile "myData.gz"

Monday, June 25, 2007, 10:47:11 PM, you wrote: try it with non-lazy bytestrings: import qualified Data.ByteString as B

On 6/25/07, Bryan O'Sullivan wrote:

Are you sure you really have gzip files? If you're on a Linux or similar box, what does "file myfile.z" report to you? It should say something like "gzip compressed data".

Aarrgh, that's the problem - it does use compress. Is the distinction between .z and .Z not an established standard? I'm guessing there's not a Haskell interface for compress. I could just tell the OS to start a gzip process, but I need to be able to build it here on my Linux box, and run it on various MS machines. Seems like the best approach at this point might be to require everyone (only 3 people) to uncompress the data onto the hard drive first, then go from there. Thanks for all the help! -Chad

On 6/25/07, Stefan O'Rear wrote:

.z : always pack .Z : always compress .gz : always gzip

gzip can handle all three, zlib only the last. (Are you *sure* your file is compress?)

This means it's compress, doesn't it? $ file myData.z myData.z: compress'd data 16 bits

...

I could just tell the OS to start a gzip process, but I need to be able to build it here on my Linux box, and run it on various MS machines. Seems like the best approach at this point might be to require everyone (only 3 people) to uncompress the data onto the hard drive first, then go from there.

Or could could reimplement compress in Haskell. The algorithm is shockingly simple, and there is a sample implementation (needs optimization and compress(1) header support, but the LZW engine is there) is already on the Wiki. Note that the patent expired in June '06, so you don't need to worry about that.

http://haskell.org/haskellwiki/Toy_compression_implementations

This looks like a lot of fun, but I've got too many other pieces of code to try to get running efficiently as it is. But I hadn't seen this link before, and it looks like interesting stuff. Thanks! Chad

On Jun 25, 2007, at 17:24 , Stefan O'Rear wrote:

.z : always pack .Z : always compress

...unless it's gone through a Windows system or a CD somewhere along the way. Note that gunzip accepts a wide variety of extensions but recognizes the files by magic number, *not* by the extension. -- brandon s. allbery [solaris,freebsd,perl,pugs,haskell] allbery@kf8nh.com system administrator [openafs,heimdal,too many hats] allbery@ece.cmu.edu electrical and computer engineering, carnegie mellon university KF8NH