January 2004 - Haskell

High-level technique for program options handling
by Tomasz Zielonka 28 Jan '04

28 Jan '04

Hello! I've used Haskell to create various command-line utitities for unix-like systems. In the process I developed a simple yet powerful and flexible technique for processing program options. What you can read below is my unfinished attempt at writing an article about it. The current form is probably far from good, but I decided I'll rather release it as it is than waste my effort. I will highly appreciate your opinions and criticism for both technical and literary side of this text. High-level technique for program options handling ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Introduction ~~~~~~~~~~~~ Sven Panne's GetOpt module does a fine job at parsing command line options. It has a simple, easy to use interface, handles short and long options, abbreviated options and most needed option argument modes (no / optional / required argument). It can even produce a nice usage info from option descriptions. [Documentation and example code using this module can be found at http://www.haskell.org/ghc/docs/latest/html/libraries/base/System.Console.G…] However, I checked over half a dozen serious programs written in Haskell and saw that in most of them the code responsible for option handling is quite ugly, repetitious and error prone (of course there were exceptions, see below). Saying 'option handling' I don't mean the work of analyzing the command line, but rather how supplied options influence program's behaviour, how error situations are handled, etc. (The exception is Wolfgang Thaller's VOP program. Wolfgang used a nice, but a little lengthy technique there. There are probably other ,,exceptional'' programs out there). I think there are two major reasons of the current situation. The first reason is that option handling is rarely of primary concern to the programmer. Often there are not that much options to handle in the beginning and it seems that the simplest solution will do. Alas, if the program is evolving and its users ask for new functionality, new options appear, and the initial design starts to be an obstacle. The other reason is lack of good examples. The example delivered with GetOpt shows how to use the library, but it doesn't show that there are other, better ways to use it. I don't propose to introduce more advanced techniques to this example, because it would make it rather heavy. But it would be nice if there would be a pointer for interested users. About typical use of GetOpt ~~~~~~~~~~~~~~~~~~~~~~~~~~~ In a typical use of GetOpt module (like in the example) there is a definition of a sum datatype with data constructors corresponding to individual command-line options. Options with no arguments are represented by nullary constructors, and options that must or can have arguments - by unary constructors. For example, below is a slightly modified example from GetOpt's documentation: data Flag = Verbose -- this option has no arguments | Version -- no arguments | Input (Maybe String) -- optional argument | Output String -- mandatory argument | LibDir String -- mandatory argument GetOpt uses 'ArgDescr a' datatype to specify argument modes of options: data ArgDescr a = NoArg a | ReqArg (String -> a) String | OptArg (Maybe String -> a) String Because every constructor of Flag datatype has one of types Flag, (String -> Flag) or (Maybe String -> Flag), they can be given verbatim as first arguments to appropriate constructors of 'ArgDescr Flag' datatype. Used in this way, getOpt parses a list of strings to a list of Flag values. It also separates options from non option arguments, signals unrecoginized, ambiguous or improperly used options. But after all, it is not that big step. There is still much processing of this Flag list to do. One has to check if specific options are there in the list, some options have to be combined, etc, etc. Advertised technique ~~~~~~~~~~~~~~~~~~~~ ( Because I wanted it to be a fully functional Literate Haskell program, here come the required imports ) > module Main (main) where > > import System.Console.GetOpt > import System > import Control.Monad > import IO > import List > import Char Let's look at program options not from the command-line side, but rather from program(mer)'s side. How to encode them? What would be the best way to present them to the programmer. How should they influence program's behaviour? 'Verbose' could be an easily accessible Bool value, False by default. 'Version', if supplied, could just print the program's version and exit. 'Input' could just yield a String, either from stdin or from a specified file. 'Output' could just take a String and do something with it. We need some kind of a fixed-size polytypic dictionary of option values - a perfect application for Haskell's records. > data Options = Options { optVerbose :: Bool > , optInput :: IO String > , optOutput :: String -> IO () > } Note that I didn't place optVersion in the record. We will handle this option differently. There is one special combination of options, the start (or default) options: > startOptions :: Options > startOptions = Options { optVerbose = False > , optInput = getContents > , optOutput = putStr > } But in the end we would like to get the record reflecting the options given to the program. We do this by threading the Options record through functions processing single options. Each such function can change this record. Why not just put such functions in ArgDescr and OptDescr datatypes? Here we benefit from first-class citizenship of functions. I won't use a pure function with type (Options -> Options), but rather an effectful function in the IO Monad, because I want to easily perform side effects during option parsing (here only in 'verbose' and 'help' options, but I could also check validity of input and output files during option processing). You may prefer to use a pure function, a State+Error monad, a State+IO monad, or something different... > options :: [ OptDescr (Options -> IO Options) ] > options = > [ Option "i" ["input"] > (ReqArg > (\arg opt -> return opt { optInput = readFile arg }) > "FILE") > "Input file" > > , Option "o" ["output"] > (ReqArg > (\arg opt -> return opt { optOutput = writeFile arg }) > "FILE") > "Output file" > > , Option "s" ["string"] > (ReqArg > (\arg opt -> return opt { optInput = return arg }) > "FILE") > "Input string" > > , Option "V" ["version"] > (NoArg > (\_ -> do > hPutStrLn stderr "Version 0.01" > exitWith ExitSuccess)) > "Print version" > > , Option "h" ["help"] > (NoArg > (\_ -> do > prg <- getProgName > hPutStrLn stderr (usageInfo prg options) > exitWith ExitSuccess)) > "Show help" > ] Now we combine all this pieces: > main = do > args <- getArgs > > -- Parse options, getting a list of option actions > let (actions, nonOptions, errors) = getOpt RequireOrder options args > > -- Here we thread startOptions through all supplied option actions > opts <- foldl (>>=) (return startOptions) actions > > let Options { optVerbose = verbose > , optInput = input > , optOutput = output } = opts > > when verbose (hPutStrLn stderr "Hello!") > > input >>= output Voila! As you can see most of work is done in option descriptions. I have attached a longer example. It is a simple text file filter that has options for uppercasing characters, reversing characters and lines, dropping initial characters, etc. It also shows how easily you can handle the event of mandatory parameter omission with IO exceptions. Best regards, Tomasz -- .signature: Too many levels of symbolic links

4 6

POpen for Win32
by Graham Klyne 27 Jan '04

27 Jan '04

In case anyone is interested, my (as yet untested) implementation of POpen for Win32 can be found here: http://www.ninebynine.org/Software/HaskellUtils/Win32/ #g ------------ Graham Klyne For email: http://www.ninebynine.org/#Contact

1 0

Re-implementing Network.URI
by Graham Klyne 27 Jan '04

27 Jan '04

I have re-implemented the URI parser in Network.URI using Parsec rather than the regex library that isn't available in Hugs, and I've wired the result into part of my URI handling test suite. The parser is based on the very latest work-in progress that is intended to replace RFC 2396 (see source code for refs). The parsing is completely rewritten, and passes all my test cases. The relative URI calculation ("relativeTo") is still the original code, and I'm noting some discrepancies with my test suite (see below). There are some more discrepancies with test cases from RFC2396, that I haven't yet looked into. The source file and test suite can be found here: http://www.ninebynine.org/Software/HaskellUtils/Network/ It is not complete, but I'm hoping I have enough working to get the HXML toolbox running under Hugs/Windows. If there is any interest in adopting this as a more portable replacement for the current Network.URI module, I'll look into bringing it fully into line with the test suite. #g -- Test results: [[ ### Failure in: Test Relative URIs:20:1 testRelative21(abs) expected: "file:/ex/x/q/r#" but got: "file:/ex/x/q/r" ### Failure in: Test Relative URIs:30:1 testRelative31(abs) expected: "file:/some/dir/#" but got: "file:/some/dir/" ### Failure in: Test Relative URIs:36:1 testRelative37(abs) expected: "mailto:local/qual@domain.org#frag" but got: "mailto:local/local/qual@domain.org#frag" ### Failure in: Test Relative URIs:45:1 testRelative50(abs) expected: "http://example/a/b/../../c" but got: "http://example/c" ### Failure in: Test Relative URIs:46:1 testRelative51(abs) expected: "http://example/a/b/c/../../" but got: "http://example/a/" ### Failure in: Test Relative URIs:47:1 testRelative52(abs) expected: "http://example/a/b/c/./" but got: "http://example/a/b/c/" ### Failure in: Test Relative URIs:48:1 testRelative53(abs) expected: "http://example/a/b/c/.././" but got: "http://example/a/b/" ### Failure in: Test Relative URIs:49:1 testRelative54(abs) expected: "http://example/a/b/c/d/../../../../e" but got: "http://example/e" ### Failure in: Test Relative URIs:50:1 testRelative55(abs) expected: "http://example/a/b/c/d/../.././../../e" but got: "http://example/e" ### Failure in: Test Relative URIs:62 testRelative76 expected: "mid:m2@example.ord/c2@example.org" but got: "mid:m@example.ord/m2@example.ord/c2@example.org" Cases: 120 Tried: 120 Errors: 0 Failures: 10 ]] The relevant test cases are, stated as: [test-name: base-uri expected-absuri relative-uri] ... [testRelative21: "file:/ex/x/y" "file:/ex/x/q/r#" "q/r#"] (but got: "file:/ex/x/q/r") In this case, the empty fragment has been incorrectly discarded -- I think this may be a bug in 'uriToString' ... [testRelative31: "file:/some/dir/foo" "file:/some/dir/#" "./#" (similar problem) ... [testRelative37: "mailto:local/qual@domain.org" "mailto:local/qual@domain.org#frag" "local/qual@domain.org#frag"] (but got: "mailto:local/local/qual@domain.org#frag") In this case, it appears that relative path processing is being applied incorrectly to a non-hierarchical base URI. ... [testRelative50: "ftp://example/x/y" "http://example/a/b/../../c" "http://example/c"] [testRelative51: "ftp://example/x/y" "http://example/a/b/c/../../" "http://example/a/"] [testRelative52: "ftp://example/x/y" "http://example/a/b/c/./" "http://example/a/b/c/"] [testRelative53: "ftp://example/x/y" "http://example/a/b/c/.././" "http://example/a/b/"] [testRelative54: "ftp://example/x/y" "http://example/a/b/c/d/../../../../e" "http://example/e"] [testRelative55: "ftp://example/x/y" "http://example/a/b/c/d/../.././../../e" "http://example/e"] These exercise some URI path-normalization code that I have in my original implementation. As far as I can tell, these are not problems with the GHC implementation. ... [testRelative76: "mid:m@example.ord/c@example.org" "m2@example.ord/c2@example.org" "mid:m2@example.ord/c2@example.org"] (but got: "mid:m@example.ord/m2@example.ord/c2@example.org") This looks like another case of inappropriate relative path processing. ------------ Graham Klyne For email: http://www.ninebynine.org/#Contact

1 0

Typo in Network.URI
by Graham Klyne 27 Jan '04

27 Jan '04

I noticed a typo in Network.URI: [[ hexDigit c | c >= 'A' && c <= 'F' = ord c - ord 'A' + 10 | c >= 'a' && c <= 'F' = ord c - ord 'a' + 10 | otherwise = ord c - ord '0' ]] I think should be: [[ hexDigit c | c >= 'A' && c <= 'F' = ord c - ord 'A' + 10 | c >= 'a' && c <= 'f' = ord c - ord 'a' + 10 | otherwise = ord c - ord '0' ]] (lowercase 'f' in second line) #g ------------ Graham Klyne For email: http://www.ninebynine.org/#Contact

1 0

KDevelop & Haskell
by Gour 27 Jan '04

27 Jan '04

Hi! I've 'discovered' that latest KDevelop (3.0) has some sort of support for Haskell development. Although pretty new to Haskell, I'm looking for adequate IDE and would like to hear if someone can share his/her experiences in using KDevelop (or some other) IDE for Haskell development? (otherwise I'm light vim user, so pls. don't recommend me to learn emacs :-) Sincerely, Gour -- Gour gour(a)mail.inet.hr Registered Linux User #278493

2 4

Gallopping Tab characters
by George Russell 26 Jan '04

26 Jan '04

Andreas wrote: > The most flexible but safe solution is to simply define the indentation > as the sequence of indentation characters used. Two consecutive lines > are indented consistently whenever one indentation is a prefix of the > other. Hence you may freely mix different indentation characters, but > you must be consistent across lines. Any decent editor should be able to > ensure that. Well no they won't, because some editors might replace blocks of 8 spaces at the start of a line with TABs (or something like that), meaning that 8 and 7 spaces would go to "\t" and " ", which your algorithm would reject. On the other hand, so would mine, so perhaps your algorithm is better, being simpler Simon Marlow wrote: > As for the width of the tab character: tab stops are every 8 columns. > Period. The Haskell report says so Yes, true. I think it was Leslie Lamport who wrote in TeXHaX that anyone defining an input format which includes tabs should be sentenced to ten years programming COBOL in Nijny-Novgorod. Off you go, Simon ... :)

4 3

Simple Fudgets-question
by Henrik Berg 26 Jan '04

26 Jan '04

Hi! I can't find any function in the Fudgets-library to do this: (F a b) -> (F a (a, b)) That is, I have a fudget taking input of type a and giving output of type b, and I want to convert it to a fudget taking input of type a and giving output of type (a, b). All I want to do is to resend the input out on the output. I'm quite new to Fudgets (and Haskell in general), and feel quite stuck on this problem. Anyone here care to give me a hint? -- Henrik

1 0

More on HXML toolbox with Hugs/Windows: matchRegex
by Graham Klyne 26 Jan '04

26 Jan '04

I think I've managed to create a version of POpen to go with Hugs under Windows. (It all compiles/links, but I haven't yet got a stand-alone test case for it). So now I'm pushing ahead with trying to get the HXML toolbox running under Hugs/Windows, and I've hit a snag with the Regex library. The oldlibs version of Regex apparently exports a function matchRegex, but the Text.Regex module which it calls upon to supply that function does not appear to define it. The problem can be seen by trying to load module Network.URI that is supplied with Hugs (2003 edition). Any pointers, please? #g ------------ Graham Klyne For email: http://www.ninebynine.org/#Contact

1 0

Re: [Haskell] Re: Use of tab characters in indentation-sensitive code
by Jeff Dalton 26 Jan '04

26 Jan '04

> This just shows how deeply ingrained the ascii plain text mindset is > in the programming community. I don't expect anything like this to ever > fly, for this reason. You guys won't let it. :( (Sorry, but how did ascii get in there? Was the argument for Unicode or HTML?) As for mindset, the programming community was one of the first to try "structure editors" and other tools that gave a more abstract view of source code. The programming community has also been pretty quick to start using HTML and XML. That plain-text syntaxes for source code have survived suggests that there are good reasons for that and that the answer to the question in an earlier message Don't we have similar needs as other electronic document manipulators? is "no, not similar enough". I work with HTML and XML all the time, and I have never felt any desire to represent programs that way; and when I do have to deal with an XML syntax for anything like a program (such as ant acripts, xslt stylesheets), I find it very unpleasant. -- Jeff

1 0

RE: [Haskell] Re: Use of tab characters in indentation-sensitive code
by Simon Marlow 26 Jan '04

26 Jan '04

> Wolfgang Thaller writes (to the Haskell mailing list): > > > IMHO, there should only be warnings about tabs when their > > size makes a difference to the meaning of the program, > > I agree and would suggest an even more stringent test to > warn against > > > a = let x = 1 > > y = 2 -- OK > > in ... > > because the (visual) interpretation depends on the font > you use (fixed width vs. proportional). The Revised Haskell 98 Report contains this paragraph (sec. 9.3): For the purposes of the layout rule, Unicode characters in a source program are considered to be of the same, fixed, width as an ASCII character. However, to avoid visual confusion, programmers should avoid writing programs in which the meaning of implicit layout depends on the width of non-space characters. As for the width of the tab character: tab stops are every 8 columns. Period. The Haskell report says so :-P Cheers, Simon

1 0