July 2008 - Libraries

Adding split/split' to Data.List, and redefining words/lines with it; also, adding replace/replaceBy
by Gwern Branwen 21 Jul '08

21 Jul '08

Hi everyone. So recently while doing some shell scripting, I found myself redefining a 'split' function (take an item and a list, and make list of lists everywhere the item appears) *yet again*, and I got annoyed enough to resolve to fix the situation. While I was at it, I decided that since 'lines' and 'words' are conceptually specializations of a general split function, I would come up with rewrites for them too; much more aesthetically satisfying to me, as it's clearer in the code now that lines and words are essentially a specialization of split, but with pragmatic edge cases (which mean we don't get nice identities like 'unlines . lines == id', but makes them more useful with, say, getContents). Code: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > lines' :: String -> [String] > lines' s = removeTrailingNull (split' '\n' s) > where > removeTrailingNull :: [String] -> [String] > removeTrailingNull y = case y of > [] -> [] > [""] -> [] > (x:xs) -> x : removeTrailingNull xs > > linesProp :: String -> Bool > linesProp x = (Prelude.lines x == lines' x) > > words' :: String -> [String] > words' = filter (not . and . map isSpace) . split isSpace > > wordsProp :: String -> Bool > wordsProp x = (Prelude.words x == words' x) > > split :: (a -> Bool) -> [a] -> [[a]] > split _ [] = [] > split p s = let (l,s') = break p s in l : case s' of > [] -> [] > (r:s'') -> [r] : split p s'' > > splitUndoProp, splitUndoIdemProp, splitPreserveDelimsProp :: (Eq a) => a -> [a] > > -> Bool > splitUndoProp x y = (concat $ split (==x) y) == y > splitUndoIdemProp x y = (concat $ concat $ split (==[x]) $ split (==x) y) == y > splitPreserveDelimsProp x y = (length $ elemIndices [x] $ split (==x) y) == > > (length $ elemIndices x y) > > split' :: (Eq a) => a -> [a] -> [[a]] > split' a b = filter (/= [a]) $ split (\x -> x==a) b > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've run many many QuickChecks testing against the Prelude lines and words, and the definitions seem to be correct. What do people think of adding these? I know I'm not the only one who has wanted split or split' on more than one occasion, and they are not the funnest functions to rewrite every time you want them. (About all they're missing are Haddocks; and perhaps a better name for split' which reflects how it is lossy and can't be undone while split can be.) ------ On a secondary note, but less important than the foregoing, I'd like to add two functions: 'replace' and 'replaceBy'. They do basically what they sound like: given two items, change every occurrence in a given list of one item to another. These are two other functions I often have to redefine, which still surprises me - Data.List has a surfeit of obscure functions I've never used and which are kind of odd, but a basic search-and-replace function isn't there? I mean, I'm not saying let's add enough functions to Data.List to turn it into a mini-Perl, but it strikes me as a real gap. (As before, I've defined some sensible QC properties and checked, although the definitions look obviously right to me.) Code: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > replaceBy :: (a -> Bool) -> a -> [a] -> [a] > replaceBy a b = map (\x -> if a x then b else x) > > replace :: (Eq a) => a -> a -> [a] -> [a] > replace a = replaceBy (==a) > > replaceLengthProp :: (Eq a) => a -> a -> [a] -> Bool > replaceLengthProp x y z = (length $ replace x y z) == (length z) > replaceUndoableProp :: (Eq a) => a -> a -> [a] -> Bool > replaceUndoableProp x y z = if not (y `elem` z) then z == (replace y x $ replace > > x y z) else True > replaceIdempotentProp :: (Eq a) => a -> a -> [a] -> Bool > replaceIdempotentProp x y z = (replace x y $ replace x y z) == (replace x y z) > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -- gwern PRF fritz Lon News IG Keyhole advise VFCT SITOR MDA

10 18

Extending the GSoC model
by Andrew Wagner 21 Jul '08

21 Jul '08

All, Not sure if this is the best place to suggest this, but I was wondering about the possibility of extending the model that we currently use for Google's Summer of Code. The benefits: 1.) A place to make suggestions for code you would never be able to write, and somebody else might not think of 2.) A place to see what's being worked on, but not yet done 3.) A way for people who are less confident in their haskell-foo to find mentors 4.) A way for people who are more confident in their haskell-foo to contribute without spending a ton of time hacking up actual code 5.) Overall, more (and higher-quality) libraries Thoughts? What would it take to make this happen?

1 0

darcs patch: Fix #2005 [network package]
by Judah Jacobson 21 Jul '08

21 Jul '08

The attached patch fixes Trac #2005 (getnameinfo error on mac with ipv6 addresses). The problem was caused by a failure to initialize the field sockaddr_in6.sin6_len . The initialization was only happening on NetBSD; apparently OS X needs it too. This patch sets that field if the macro SIN6_LEN is defined (which is the standard way to detect that, according to my research online and RFC 3493 (http://tools.ietf.org/html/rfc3493) -Judah Sat Jul 19 10:18:41 PDT 2008 Judah Jacobson <judah.jacobson(a)gmail.com> * FIX #2005: Set the sockaddr_in.sin6_len field on all platforms that support it, not just NetBSD. M ./Network/Socket.hsc -1 +1

1 1

Proposal: overhaul System.Process
by Simon Marlow 18 Jul '08

18 Jul '08

I've made some improvements to System.Process that I'd like to get feedback on. Everything so far is backwards compatible in the sense that I've only added to the API - everything that was there before is still available, with the same semantics (except where bugs have been fixed). Haddock for the proposed new System.Process: http://darcs.haskell.org/~simonmar/process/System-Process.html Ticket: http://hackage.haskell.org/trac/ghc/ticket/2233 Discussion period: 4 weeks (20 May) Summary of changes: Tue Apr 22 15:02:16 PDT 2008 Simon Marlow <simonmarhaskell(a)gmail.com> * Overhall System.Process - fix #1780: pipes created by runInteractiveProcess are set close-on-exec by default - add a new, more general, form of process creation: createProcess Each of stdin, stdout and stderr may individually be taken from existing Handles or attached to new pipes. Also it has a nicer API. - add readProcess from Don Stewart's newpopen package. This function behaves like C's popen(). - Move System.Cmd.{system,rawSystem} into System.Process. Later we can depecate System.Cmd. - Don't use O_NONBLOCK for pipes, as it can confuse the process attached to the pipe (requires a fix to GHC.Handle in the base package). - move the tests from the GHC testsuite into the package itself, and add a couple more - bump the version to 2.0

13 75

Cabal: Building something from within a test hook
by Max Bolingbroke 18 Jul '08

18 Jul '08

Hi, I'd like to provide tests for my Cabal project, and would like to do so by having >another< Cabal project in a tests/ directory that is built and run by "Cabal test". I've got this far towards getting it working: """" main :: IO () main = defaultMainWithHooks (simpleUserHooks { runTests = buildAndRunTests }) buildAndRunTests :: Args -> Bool -> PackageDescription -> LocalBuildInfo -> IO () buildAndRunTests _ _ _ local_build_info = do test_package_description <- fmap flattenPackageDescription $ readPackageDescription normal "./tests/test-package.cabal" let [executable] = executables test_package_description build test_package_description local_build_info defaultBuildFlags [] system $ modulePath executable """ This kind of works, but the local_build_info is obviously wrong as it is for the package being compiled rather than the one I >want< to compile. What I'm really hoping Cabal provides or is able to provide is a way to configure the tests package using the old LBI such that the same tools (e.g. compiler, hc-pkg etc) that were used for this package are used for the tests one while leaving everything else at it's default value. I could then use the resulting LBI to build the tests as I desire. Any pointers or suggestions for alternative ways I should be going about this are gratefully recieved! Max

2 2

GHC API: monad and error handling
by Thomas Schilling 15 Jul '08

15 Jul '08

Hello librarians, as some may know, I am currently working on improvements to the GHC API. Many of the exported functions of the GHC module have a type of the form Session -> ... -> IO (Maybe X) where 'Session' is mutable and 'X' is the actual result type of the function. In order to enforce (the implicitly assumed) single- threaded use of a session and to provide richer error information I am restructuring all exported functions of the GHC API to return a computation in the 'Ghc' monad. This currently looks like this(*) newtype Ghc a = Ghc { unGhc :: Session -> IO (Either GhcError a } newtype Session = Session (IORef HscEnv) Functions that modify a session are now in this monad but behave mostly the same. A more difficult decision is how to deal with errors. The GhcError type currently looks like this: -- | An error annotated with the phase it happened in. data GhcError = GhcError HscPhase Messages -- ^ A "normal" compilation error. | ApiError HscPhase String -- ^ An error that violated some pre-condition/invariant of the API. | GhcIOException Exception -- some IO exception -- XXX: would (forall e. Typeable e => e) be better? Note that we have to wrap IO exceptions and propagate them separately in the Ghc monad.(**) -- | Lift an 'IO' action into the 'Ghc' monad. IO exceptions are wrapped and -- can be queried by matching on the 'GhcIOException' constructor in -- 'ghcCatch'. XXX: is there a cleaner way? io :: IO a -> Ghc a io action = Ghc $ \_ -> Exception.handle (return . Left . GhcIOException) $ do a <- action return (Right a) Now, what is the proper way to translate a function of type ... -> IO (Maybe a) into this monad? My suggestion would be to prefer ... - > Ghc a wherever possible. This would make things nicer with the upcoming split-up of more fine-grained control over the executed phases. We could write parsed_file <- parse file tc_rn_file <- typecheckRename parsed_file simpl_file <- simplify tc_rn_file ... `onCompileError` $ \GhcError phase msgs -> somehowHandleErrors phase msgs instead of mb_parsed_file <- parse file case mb_parsed_file of Nothing -> ... Just parsed_file -> do mb_tc_rn_file <- typecheckRename parsed_file case mb_tr_rn_file of ... etc ... where 'onCompileError' would only catch errors with the constructor 'GhcError', not 'ApiError' or 'GhcIOException'. For the latter It is, however, not always clear what is a compile error and what is not. Consider, for example, the following function: -- | Takes a 'ModuleName' and possibly a 'PackageId', and consults the -- filesystem and package database to find the corresponding 'Module', -- using the algorithm that is used for an @import@ declaration. findModule :: ModuleName -> Maybe PackageId -> Ghc Module findModule mod_name maybe_pkg = withSession $ \hsc_env -> let dflags = hsc_dflags hsc_env hpt = hsc_HPT hsc_env this_pkg = thisPackage dflags in case lookupUFM hpt mod_name of Just mod_info -> return (mi_module (hm_iface mod_info)) _not_a_home_module -> do -- XXX: should we really throw IO exceptions here? res <- io $ findImportedModule hsc_env mod_name maybe_pkg case res of Found _ m | modulePackageId m /= this_pkg -> return m | otherwise -> throwDyn (CmdLineError (showSDoc $ text "module" <+> pprModule m <+> text "is not loaded")) err -> let msg = cannotFindModule dflags mod_name err in throwDyn (CmdLineError (showSDoc msg)) This either returns a Module or throw an exception by looking it up in the home package table. Otherwise, it throws a CmdLineError exception, which is intended for reporting failure inside GHCi. I guess the proper way for this function would be to throw an 'ApiError', i.e., we expect the looked-up module to be existing in the home package table and to be loaded. My questions thus are: - Does this sound like a reasonable strategy? - Is such a monad a good or a bad idea? - Does anyone have an idea of a classification of errors, or guidelines/principles for one? - Any other comments? Thanks, / Thomas (*) an alternative implementation that would probably be a bit more efficient in case of errors could use continuation-passing style: newtype Ghc a = Ghc { unGhc :: forall ans. Session -> (GhcError -> IO ans) -- failure continuation -> (a -> IO ans) -- success continuation -> IO ans (**) instances for mtl classes will be provided in a separate package to avoid adding mtl as a build dependency. -- I was wrong. / This changes everything.

2 2

Trace and laziness
by Alexander Dunlap 14 Jul '08

14 Jul '08

Oops, forgot to CC the list ---------- Forwarded message ---------- From: Alexander Dunlap <alexander.dunlap(a)gmail.com> Date: Sun, Jul 13, 2008 at 11:52 AM Subject: Re: Trace and laziness To: Isaac Dupree <isaacdupree(a)charter.net> On Sun, Jul 13, 2008 at 3:24 AM, Isaac Dupree <isaacdupree(a)charter.net> wrote: > Alexander Dunlap wrote: >> >> Hi all, >> >> I've come to rely on Debug.Trace.trace as a tool to figure out what my >> programs are doing, so I was somewhat surprised to realize that it >> doesn't handle very long lists well. >> >> For example, if I do "trace (show [1..1e9]) 0", the program doesn't >> produce any output until the entire list is evaluated (and if the list >> is infinite, it just doesn't do anything). If I replace this with >> "unsafePerformIO $ hPutStrLn stderr (show [1..1e9]) >> return 0", the >> numbers from 1 to 1e9 start appearing on the screen immediately. >> >> Looking at the source of Debug.Trace, it looks like the lazy version >> (using hPutStrLn stderr) is implemented, but ifdef'd out for GHC, >> which uses a (presumably strict) C version. It would be nice if there >> was a way for GHC to do the lazy tracing as well. > > you seem to be right, although I hope that buffering effects don't change it > (ghc 6.8.2) : > $ ghci -e "Debug.Trace.trace ('a':'b':(Debug.Trace.trace ['C'] 'c'):'d':[]) > ()" > C > abcd > () > > I find this behavior important, because when it was the lazy version, one > thing's trace-'show' would always be interrupted in the middle of the line > by something else's trace-show, etc., so it was nearly impossible to > decipher. Strictness isn't the nicest solution but it works most of the > time... clearly not all the time... > > -Isaac > Perhaps we need a trace and a trace' (lazy version and strict version) then? Alex

1 0

Trace and laziness
by Alexander Dunlap 13 Jul '08

13 Jul '08

Hi all, I've come to rely on Debug.Trace.trace as a tool to figure out what my programs are doing, so I was somewhat surprised to realize that it doesn't handle very long lists well. For example, if I do "trace (show [1..1e9]) 0", the program doesn't produce any output until the entire list is evaluated (and if the list is infinite, it just doesn't do anything). If I replace this with "unsafePerformIO $ hPutStrLn stderr (show [1..1e9]) >> return 0", the numbers from 1 to 1e9 start appearing on the screen immediately. Looking at the source of Debug.Trace, it looks like the lazy version (using hPutStrLn stderr) is implemented, but ifdef'd out for GHC, which uses a (presumably strict) C version. It would be nice if there was a way for GHC to do the lazy tracing as well. I would have submitted a proposal on this, but I'm not sure about the design decisions that went into using the C version for GHC or about the best way to go about it. There are a number of workarounds (define my own function, don't print the whole list), so this probably isn't very high-priority if there are important other considerations. Alex

2 1

ANNOUNCE: vector 0.1 (efficient arrays with lots of fusion)
by Roman Leshchinskiy 12 Jul '08

12 Jul '08

Hi all, the vector library will eventually provide fast, Int-indexed arrays with a powerful fusion framework. It's very immature at the moment (I haven't tested most of the code) and implements just a few combinators but I thought releasing early wouldn't hurt. Use at your own risk and expect things to break horribly! What it provides: * Boxed and unboxed arrays with a generic interface and a very basic set of combinators. * A powerful loop fusion framework. It is based on stream fusion but already goes beyond that (in particular, it can do some things in-place) and will be significantly extended in the future. * Extensibility. * Rudimentary documentation. * Bugs. The code is based on the DPH libraries but is much more generic and a *lot* simpler. This has only been made possible by the tremendous progress in GHC's simplifier in the recent months/years (thanks Simon!). Consequently, you'll need a recent development version of GHC to build this, 6.8 won't work. It might be able to compile the library but you'll get terrible code, much worse than with lists. If you want to try it out, compile with -O2 -fno-spec-constr-count and please don't forget to let me know about things that didn't work (or, miraculously, did). Grab it from http://hackage.haskell.org/cgi-bin/hackage-scripts/package/vector or darcs get http://darcs.haskell.org/vector Again, special thanks to Simon for doing such a wonderful job with the optimiser. Enjoy, Roman

1 0

hscurses under wrong category
by Bas van Dijk 12 Jul '08

12 Jul '08

Hello, I noticed that the hscurses package[1] is listed under category "User-interface" while all other user interface packages are listed under "User Interfaces". Could someone please release hscurses under the "User Interfaces" category and remove the current one? That will make it more consistent. Thanks Bas [1] http://hackage.haskell.org/cgi-bin/hackage-scripts/package/hscurses

6 5