- Libraries - Haskell.org

RE: Environment variable does not work (especially DARCS_EDITOR)
by Simon Marlow 26 Apr '05

26 Apr '05

On 26 April 2005 02:15, Benedikt Schmidt wrote: > Sorry for taking so long to reply. > > On Tue, 2005-04-19 at 11:36 +0100, Simon Marlow wrote: >> On 16 April 2005 20:41, Benedikt Schmidt wrote: >>> module Main where >>> import System.Cmd >>> main = rawSystem "nvi" [] > [ patch calling resetNonBlockingFD in runProcess ] >> If we did this, we would also need to re-enable non-blocking mode >> after the rawSystem completes. Also, this doesn't work well if >> runProcess is used and the Haskell program continues to run while >> the subprocess is executing, because the O_NONBLOCK flag is shared >> between parent and child. > > I didn't think about the flags beeing shared between parent and child. > > I tried using setFdOption first, but didn't see that rawSystem sets > stdin back to blocking mode when stdin is evaluated. > > This works for me now with ghc-6.4: > main = do stdin `seq` return () > bracket (setFdOption stdInput NonBlockingRead False) > (\_ -> setFdOption stdInput NonBlockingRead True) > (\_ -> rawSystem "nvi" []) > > >> So I suggest doing this for System.Cmd.{system,rawSystem} only. > > Would something like that be safe for system and rawSystem if all > arguments where forced before the setFdOption? If there are > several threads? You're right, it's not safe in general. Better to leave it up to the user - your fix is fine. I'll put an entry in the FAQ. Cheers, Simon

1 0

Cabal is a layer on Haskell
by Simon Marlow 26 Apr '05

26 Apr '05

On 26 April 2005 04:55, S. Alexander Jacobson wrote: > On Mon, 25 Apr 2005, John Meacham wrote: >> You can compare Cabal's way of doing things to what specific >> compilers have done, but not 'Haskell' because 'Haskell' doesn't >> concern itself with these issues. This is correct. > Yeah Simon Marlow corrected me on this one too. No I didn't :-p > Cabal should be thought of Cabal is part of the Haskell source not its > implementation: > > Me: My fundamental point here is that the meaning of code shouldn't > change depending on the location where it is interpreted. [...] > Concretely, the meaning of "import Data.List" in a > haskell module shouldn't depend on whether the module is on my > computer or yours. > > Simon: You can think of .cabal as part of the source if that helps. > It really is part of the source - but implemented and specified as > a layer on top of existing language and compiler technology, for > convenience and so that we can share the tools. You misinterpreted me. I'm simply saying that .cabal is part of the source of a Cabal package, not part of the Haskell source. Of course, since Cabal doesn't change the definition of Haskell. I'll state this clearly (again): Cabal is a layer on top of the Haskell language definition. It does not change Haskell, it is not "incompatible" with Haskell. It defines some things that are undefined in the Haskell definition. Cheers, Simon

1 0

The meaning of Cabal+Haskell
by Simon Marlow 26 Apr '05

26 Apr '05

On 26 April 2005 04:55, S. Alexander Jacobson wrote: > On Mon, 25 Apr 2005, John Meacham wrote: >> The haskell report (and hence the haskell language) specifies >> __nothing__ about how imported module names are actually mapped to >> code. This has always been left up to implementations. [...] >> This is why comparing haskell to cabal doesn't make sense, cabal is >> meerly one proposed implementation of behavior that is undefined by >> Haskell. > > That's what I thought too! But, Malcolm Wallace and Simon Marlow > corrected me a few posts back and they are the authors of the spec. > They say that, like Haskell, Cabal actually specifies __nothing__ > about how imported module names are mapped to code either. > > Me: My objection to Cabal is that it in fact DOES determine the > referent of an import statement (build-depends). > > Malcolm: I can see why you might object, if that were the case, but > Cabal does NOT in fact determine the referent! Cabal is just a > convenient way of grouping together the denotational /interface/ of > a large bunch of modules. The implementations of those modules can > be replaced at any time, provided they still meet the interface. > > Me: The Cabal spec says "build-depends: [...] a list of packages > [..] needed to build this one" > > Malcolm: "needed to build" tells me which interfaces must be > available at compile-time, no more and no less. > > Simon: Build-depends uses the source-code notion of interface, not > the object code. So, build-depends has the same meaning for all > compilers. Malcolm is pointing out that it's possible to substitute object code for a module as long as the replacement supports the same interface. The definition of "interface" in this case is compiler-dependent, so I think we should separate this observation from the discussion at hand - it's an extra complication (IMHO). You're interested in what the meaning of Cabal+Haskell is. Here's a succinct interpretation of the relationship: Haskell: "A program consists of a set of modules." Cabal+Haskell: "A program consists of a set of modules. I'm going to provide a way to construct that set of modules, by combining sets of modules called packages." Cheers, Simon

1 0

RE: Environment variable does not work (especially DARCS_EDITOR)
by Simon Marlow 26 Apr '05

26 Apr '05

On 16 April 2005 20:41, Benedikt Schmidt wrote: > [ crossposted to libraries(a)haskell.org ] > Tommy Pettersson <ptp(a)lysator.liu.se> writes: > >> A likely source of the problem can be that pico aborts if its >> stdin or stdout isn't a tty. I have that problem with nvi. >> If you run darcs 1.0.2 you can try and set DARCS_EDITOR to >> "pico </dev/tty >/dev/tty". If you run a later build you >> may have to write a wrapper script like >> >> #!/bin/sh >> pico </dev/tty >/dev/tty >> >> since the to-be-next stable darcs not always perform the >> redirection on the system call any longer. >> >> Or, if this turns out to be the problem, and if you know how >> to make darcs do this redirection of stdin/out in a portable >> way, fix it in the darcs source. > > The problem seems to be related to rawSystem and the ghc RTS (at > least GHC 6.4). GHC sets stdin/stdout/stderr to nonblocking mode > on startup and nvi doesn't like that. There is a comment in > ghc/rts/RtsUtils.c that resetNonBlockingFd is called before > exiting because leaving standard file descriptors in nonblocking > mode confuses some shells. > > module Main where > import System.Cmd > main = rawSystem "nvi" [] > > The test program works after applying the following patch to GHC > that disables nonblocking mode after the fork and before the exec. > > diff -u -r1.7 runProcess.c > --- libraries/base/cbits/runProcess.c 27 Mar 2005 13:41:19 -0000 > 1.7 +++ libraries/base/cbits/runProcess.c 16 Apr 2005 19:08:52 -0000 > @@ -62,6 +62,10 @@ > dfl.sa_handler = (void *)quithandler; > (void)sigaction(SIGQUIT, &dfl, NULL); > } > + > + resetNonBlockingFd(fdStdInput); > + resetNonBlockingFd(fdStdOutput); > + resetNonBlockingFd(fdStdError); > > dup2 (fdStdInput, STDIN_FILENO); > dup2 (fdStdOutput, STDOUT_FILENO); > > I'm not sure how to handle that in darcs without > copying/reimplementing the parts of System.Cmd we need. If we did this, we would also need to re-enable non-blocking mode after the rawSystem completes. Also, this doesn't work well if runProcess is used and the Haskell program continues to run while the subprocess is executing, because the O_NONBLOCK flag is shared between parent and child. So I suggest doing this for System.Cmd.{system,rawSystem} only. Cheers, Simon

3 2

RE: Nonblocking IO (was: Binary IO)
by Simon Marlow 25 Apr '05

25 Apr '05

On 25 April 2005 10:51, Peter Simons wrote: > Simon Marlow writes: > > >> hPutBufNonBlocking :: Handle -> Ptr a -> Int -> IO Int > >> hGetBufNonBlocking :: Handle -> Ptr a -> Int -> IO Int > > > I believe they don't have proper non-blocking behaviour > > on Windows, though. > > What exactly would happen when I'd try to use those on > Windows? Would they block until they had read/written > exactly 'Int' bytes? Or is there some other, more subtle > difference? I think they behave like the blocking variants on Windows (but the code is quite intricate.. I'm not 100% sure). This should really be fixed. Cheers, Simon

1 0

RE: Binary IO
by Simon Marlow 25 Apr '05

25 Apr '05

On 22 April 2005 12:05, Peter Simons wrote: > Simon Marlow writes: > > > Does anyone have any *objections* to introducing > > > System.IO.hGetWord8 :: Handle -> IO Word8 > > System.IO.hPutWord8 :: Word8 -> Handle -> IO Word8 > > I don't mind having these functions, but to be honest I > doubt that they would be useful for real life applications. > Reading a byte at a time is a performance nightmare, > buffering or not. IMHO, the much better API is this one: > > hPutBufNonBlocking :: Handle -> Ptr a -> Int -> IO Int > hGetBufNonBlocking :: Handle -> Ptr a -> Int -> IO Int You have these in GHC.IO at the moment. I beieve they don't have proper non-blocking behaviour on Windows, though. These are also rather more difficult to implement than the single-byte versions, so I can't see nhc98 or Hugs implementing them any time soon. Cheers, Simon

6 9

Announce: Haskell Cryptographic Library 2.0.1
by Dominic Steinitz 24 Apr '05

24 Apr '05

I would like to announce the release of a new version of the Haskell Cryptographic Library. It now uses darcs and cabal and provides the modules listed below. The most significant change is in the way ASN.1 is handled. This has been made generic enough to handle PKCS#8 private keys and X.509 certificates. For further information see http://www.haskell.org/crypto. Dominic. Codec.ASN1 Codec.ASN1.BER Codec.ASN1.PKCS8 Codec.ASN1.TLV Codec.ASN1.X509 Codec.Binary.Base64 Codec.Encryption.AES Codec.Encryption.Blowfish Codec.Encryption.DES Codec.Encryption.Modes Codec.Encryption.Padding Codec.Encryption.RSA Codec.Encryption.RSA.EMEOAEP Codec.Encryption.RSA.MGF Codec.Encryption.RSA.NumberTheory Codec.Utils Data.Digest.MD5 Data.Digest.SHA1 Data.LargeWord

1 0

Re: Binary IO
by Dominic Steinitz 24 Apr '05

24 Apr '05

On Friday 22 Apr 2005 9:48 pm, libraries-request(a)haskell.org wrote: > Message: 1 > Date: Fri, 22 Apr 2005 08:30:27 -0700 > From: "David Brown" <haskell(a)davidb.org> > Subject: Re: Binary IO > To: "Peter Simons" <simons(a)cryp.to>, libraries(a)haskell.org > Message-ID: <op.spm301mpl6e4hk(a)a64.davidb.org> > Content-Type: text/plain; format=flowed; delsp=yes; charset=iso-8859-1 > > On Fri, 22 Apr 2005 08:09:27 -0700, Peter Simons <simons(a)cryp.to> wrote: > >> And hGet/PutWord8 are fast enough for most situations. > > I haven't been following the discussion closely but I am just about to make a big release of the crypto library and I have been using "Hal Daume III's NewBinary now ghc6ized, cabalized, debianize, darcsized." available at http://www.n-heptane.com/nhlab/repos/NewBinary. Specifically I use: import qualified NewBinary.Binary as B (BinHandle, getBits, getByte) I hadn't see anything about getBits but would make a plea for its inclusion in any Binary IO package. > > I seem to run into this kind of thing a lot with Haskell. Dominic's ASN > library is useful to me, however I'm finding I'll probably just have to > use it as a template for new code. It works fine for what designed for, > parsing keys and such, but I'm looking to use it to represent gigabytes of > data. Processing data like that one Word8 at a time isn't practical. For information, the most significant change in the crypto library (as well darcsizing, cabalizing and restructuring it to fit the agreement reached on this list - http://www.haskell.org//pipermail/libraries/2005-January/002856.html) is the ASN.1 handling. If that is going to cause anyone problems, please let me know I will see what can be done. Dominic.

1 0

RE: Cabal Issues
by Simon Marlow 23 Apr '05

23 Apr '05

On 23 April 2005 21:03, Benjamin Franksen wrote: > On Saturday 16 April 2005 03:22, Isaac Jones wrote: >> Ashley Yakeley <ashley(a)semantic.org> writes: >>> I've just started using Cabal, for my time library implementation >>> (<http://semantic.org/TimeLib/>). I have some issues: >>> >>> 1. If I do "runghc Setup.hs haddock" and then "runghc Setup.hs >>> install", the documentation doesn't appear to get installed >>> anywhere. >> >> Where do folks think that the docs should go? > > This is highly system dependent and even linux distributions don't > agree here. So: > >> Or should we add a >> haddock-prefix flag to configure? > > Yes. For unix like systems, a reasonable default (if no option is > specified) could be /usr/local/share/doc. Sure - this is only the default, we expect RPMs and other package systems to override it to match the local policy. A subdirectory of /usr/local/share/doc is the right thing on Unix systems. Cheers, Simon

1 0

RE: GC trouble [was Re: Binary IO]
by Simon Marlow 23 Apr '05

23 Apr '05

On 23 April 2005 20:12, Jan-Willem Maessen wrote: > Hmm; this sounds like a lot of full-heap collections, which is exactly > what generational GC is trying to avoid. A very large old generation > (like, say, 500+Mb) might help a lot in this instance; I have no idea > how GHC decides generation sizes. It might also help to set a very > large allocation area to reduce promotion rate to the second > generation, and give the gobs of transient data some time to die---or, > similarly, to increase the number of generations to increase the time > it takes things to get to the old generation. Fundamentally, though, > when you run really close to your memory limits GC tends to be > unhappy. For a program with this much data, GHC's default allocation area size of 256k is pretty small. In fact, it's pretty small full stop - I recently bumped it to 512k. I suggest trying -A1m at least. I'd be interested to know whether you get better results by tweaking any of the other GC settings: more generatsions and more steps in particular. Cheers, Simon

1 0