- Haskell-Cafe - Haskell.org

Re: [Haskell] Simple IO Regions
by Keean Schupke 19 Jan '06

19 Jan '06

Hi Oleg, Just made a few modifications and thought it might be useful to people. I have rewritten the functions as "liftR" and "bracketR" over a "MonadIO" monad interface (allowing monad-transformers to be used). This is now usable as Region library, as you can lift any arbitrary IO function into a Region. I don't think I have overlooked anything, and I think it is as safe as the original... but perhaps you would like to check for stupid mistakes... The exported interface is intended to be "liftR", "bracketR" and "runR". See attached "MonadIORegion.hs"... Regards, Keean. oleg(a)pobox.com wrote: >This message shows a very simple implementation of Monadic Regions >(for the particular case of IO and reading the file). The technique >*statically* guarantees that neither a file handle nor any computation >involving the handle can leak outside of the region that created >it. Therefore, the handle can be safely closed (and its resources >disposed of) whenever control leaves the corresponding 'withFile' >block. Many handles can be open simultaneously, the type system >enforces the proper nesting of their regions. The technique has no >run-time overhead and induces no run-time errors. Unlike the previous >implementation of monadic regions, only the basic extensions >(higher-ranked types and one two-parameter type class) are used. No >undecidable instances, no functional dependencies (let alone >overlapping instances) are required. In fact, the implementation uses >only one trivial typeclass and one trivial instance. > >

3 7

(no subject)
by Andreas Bakurov 19 Jan '06

19 Jan '06

1 0

How to redirect a handle...
by Dimitry Golubovsky 19 Jan '06

19 Jan '06

Thanks everybody who answered. Indeed, forkProcess is something I completely overlooked... -- Dimitry Golubovsky Anywhere on the Web

1 0

Packed String parameters
by Scherrer, Chad 18 Jan '06

18 Jan '06

To read a file using a packed string, I need to use this function, right? hGetPS :: Handle -> Int -> IO PackedString What's the Int? Do you have to specify the length in advance? I don't know how packed strings are implemented, and I'm having trouble finding much documentation. I'm trying to speed up this horribly slow thing: dot :: [Double] -> [Double] -> Double dot x y = sum $ zipWith (*) x y main = getContents >>= sequence_ . map (print . dot weights . map read . words) . lines The idea is to take a dot product of each line of a file with a fixed vector of weights, and send the result to stdout. I thought PackedString's might help, but I'm not sure what to do with that parameter. The input file is over 1 million lines long. Any ideas? Thanks, Chad Scherrer Computational Mathematics Group Pacific Northwest National Laboratory "Time flies like an arrow; fruit flies like a banana." -- Groucho Marx

2 1

How to redirect a handle within a thread (GHC 6.4.1 runtime)?
by Dimitry Golubovsky 18 Jan '06

18 Jan '06

Is it possible to redirect a Handle (say stdout) somewhere only within a running thread (started with forkIO) not touching the same handle for the main and other threads? I have a lot of code written with putStr(Ln) which was used in a program acting as a filter, i. e. stdout was redirected by the invoking shell. Now I want to run this code within a thread in other program, but output must go to a file (or a pipe, or anywhere else a file descriptor may be opened for). So fdToHandle is not good because I need to modify the `stdout' only for that thread, not to create a new Handle. Rewriting the code is not a convenient way (but will be done if nothing else helps) because then I will need to pass that handle around. Running that code in a separate executable is just a bit more convenient because I need to pass many parameters via command line (a thread inherits values of them), and therefore is also desired to be avioded. Functions in System.Process deal with external executables. Does there exist anything like runInteractiveProcess, but just forking a new process (the whole memory is copied, so everything will be inherited), but executes a particular function within the parent program? I'm afraid that if I just make a foreign call to `fork', the RTS may not understand this correctly. -- Dimitry Golubovsky Anywhere on the Web

5 4

Avoiding name collisions by using value spaces instead of modules
by Brian Hulley 18 Jan '06

18 Jan '06

Hi - A main problem I've found with Haskell is that within a module, too many things are put into the same scope. For example data Tree a b = Leaf a | Node {elem::b, lhs::Tree a b, rhs::Tree a b} puts Leaf, Node, elem, lhs, and rhs, into the module's namespace, as well as Tree. This means that if you want another kind of tree in the same module you've got to try and think of another word for Leaf, or else prefix every value constructor with some prefix of the tycon to try and prevent name collisions. I propose that the above declaration should introduce a *new module* Tree, as a sub module of the containing module, and Leaf, Node, elem etc will be put into this module, and not the module containing the data declaration itself. Identifiers in this child module could then either be used qualified or unqualified as long as they don't conflict with anything in the parent (containing) module, thus in the parent module, we could have: a = Tree.Leaf 3 b = Node{elem = 6, lhs = Leaf 2, rhs = Leaf 3} c = Tree.elem b c = b^elem where ^ is just a helpful sugar which binds tighter than function application and can be used anywhere to allow an "object oriented" way of thinking where the first arg is put on the left of the function (more on this later). Also, we could generalise the GHC GADT style data declarations by allowing any declarations to appear in the where part, and allow modules to consist of a single data declaration (ie begin with the keyword data instead of module) so that the contents of the Data.Set module would instead be something like: data Ord a => Set a = .... where insert :: Set a -> a -> Set a ... This would allow Set to be imported qualified into other modules where we could refer to the Set type by the single word Set instead of Set.Set (which I always think looks very strange indeed) ie module M where import qualified Data.Set as Set f :: a -> Set a -- no longer need to write a -> Set.Set a In fact, the whole complexity asociated with hiding components of a module and allowing unqualified imports could be discarded, since the correct resolution for each identifier would be determined by its typed context, just as in OOP the resolution is determined by the type of the object. So to summarise so far, for any value f :: T0 -> T1 -> ... -> Tn, we construct the tuple (Q0, Q1, ... Qn) where for each i, Qi is obtained from Ti by ignoring all predicates and quantification and replacing each tyvar by the symbol '?'. For example, for module M where foo :: forall b. Eq a => Set (Tree a) -> Tree ([a],b) -> [Tree (a,b)] we ignore the forall and Eq, and replace tyvars and tuple the results to get: M.(Set (Tree ?), Tree ( (,) ( [] ?) ?), [] (Tree ( (,) ? ?))).foo as the fully qualified reference to the entity we've just declared. If we call the tuple (Set, ...) the "ty-tuple", which specifies a space of values (ie something that each value is considered to "belong" to - the analogue of the object in OOP), then we can define a projection from a space into a containing space as follows: 1) Let Qi == Ui Si0 Si1 ... Sik be any element of a ty-tuple P. Then we can form a new ty-tuple P' by replacing Qi by Ui. 2) Let Qi be any element of a ty-tuple P of arity n+1. Then we can form P' == (Q0, ..., Qi-1, Qi+1, ... Qn) of arity n In both cases, we can use P' to qualify the identifier to get a value reference as long as this still has enough information for disambiguating other uses of the identifier in M. Applying these transformations of the ty-tuple to the example above, any of the following could be used as a partial qualification of foo in M: (Set Tree, Tree ([],?), [Tree]).foo (Set, Tree, []).foo Tree.foo foo Rule 2 allows us to propagate identifiers back up to their ancestor modules as long as there are no conflicts along the way. The type inference algorithm could then be modified (I hope) to use the top-level annotation for a value declaration to determine the entities that identifiers refer to, by searching in the appropriate value spaces determined by the types of the args and return value. label :: Tree a -> Int -> (Tree a, Int) label (Leaf x) i = (Leaf i, i+1) The identifier Leaf is resolved in the current namespace augmented by the namespace for module Tree, thus we don't have to explicitly write Tree.Leaf even though the declaration of label occurs outside the Tree module itself. (I suggest that top-level type annotations should be mandatory since without them one just drowns in a sea of TI confusion when there is a type error. By making them mandatory the TI algorithm could make real use of them to resolve identifier bindings as in OOP) Similarly: foo :: a -> Set a foo x = singleton x There is no need to say Set.singleton x, because we know that the result of foo has type Set a therefore we search in the current module augmented by the contents of the Set module (every type is also a module) for a binding for "singleton" which maps from a to Set a. Finally (apologies for this long post), returning to the use of ^ to allow an object oriented way of thinking consider: insert :: a -> Set a -> Set a ps = singleton 3 qs = insert 4 ps rs = ps^insert 4 When resolving "insert" used in the binding for rs, the compiler should see that we are looking for some function Set Int -> Int -> Set Int, and hence will be looking in the current module augmented by the Set module. However the Set module only has a binding for insert with type a -> Set a -> Set a. So the compiler should generate a new function insert' from insert by moving the first Set a arg to the front. Summary: 1) Every value binding belongs to a value space represented by a ty-tuple which abstracts away from the details of the actual type 2) The space of ty-tuples forms a lattice, and in particular this means qualification is optional when there are no conflicts (we can use Leaf 3 instead of Tree.Leaf 3), and we can use partial qualification to supply just the disambiguation info needed. 3) The TI algorithm should use the type of the expression (generated top-down from the top-level annotation) to search in the correct value spaces to resolve identifiers 4) Record field selection doesn't need any special scoping rules, and is just one example of a general object-oriented way of thinking about function application. 5) We can get all the advantages of automatic namespace management the OOP programmers take for granted, in functional programming, by using value spaces as the analogue of objects, and can thereby get rid of complicated import/export directives and improve code readability with less typing (but making more use of typing - excuse the pun :-)) I'm in the process of trying to implement a pre-processor for Haskell which would use the algorithms sketched above (as well as fixing a few other things I think are wrong with Haskell such as the way the current layout rule allows one to write code that would break if you replaced an identifier with one that had a different number of characters in it etc), so any feedback on these ideas would be welcome. Thanks for reading so far! Brian Hulley PS Everything above is purely intended for resolving the kind of ad-hoc overloading that is not amenable to treatment by type classes (which transforms some subset of the ad-hoc-overloaded functions which share the same shape into a single function with implicit dictionary arguments (in the usual implementation)).

5 16

Yet another top-level IO proposal
by Robert Dockins 18 Jan '06

18 Jan '06

Fellow haskellers, I have a proposal I would like to enter into the eternal top-level IO debate. The proposal involves a minor language extension and some runtime support for thread local reference cells. I believe it has the potential to meet many of the needs of those requesting top level IO. My apologies for this rather lengthy message, but given the volatility of discussion on this matter in the past, it seemed best to lay out my thoughts as precisely as possible at the beginning. So without further ado, the proposal. The language extension: -- Add a new keyword 'threadlocal' and a top level declaration with the syntax threadlocal <identifier> <type expression> <initializer expression> -- The declaration consists of three parts 1) the name of the reference cell 2) the type of data stored in the cell and 3) an initializer action. -- The name of the cell declared with 'threadlocal' shares the function namespace and introduces a CAF with the type 'TLRef a', where 'a' stands for the type given in the declaration. -- The initializer action must be an expression with type 'TL a', where 'a' stands for the the type given in the 'threadlocal' declaration, and 'TL' is the thread-local initialization monad (a little like the ACIO monad, more on this below). The semantics: -- Each thread in a program (the main thread and threads sparked from forkIO and forkOS) has a "bank" of thread local variables. Writes to and reads from a thread- local cell are only written to/read from the bank of the thread performing the write/read. -- For any given bank, a thread-local cell may be "empty" (which means it holds no value) or "full" with a value of its declared type. -- There is a phantom bank of thread-local values belonging to no thread in which the value of all thread-local cells is "empty". This represents the state of thread local variables before program start. -- Whenever a thread is sparked (including the main thread) and before it begins executing, its thread-local variables are initialized. For each declared thread-local variable (in the transitive closure of imported modules), the declared initilzation action is run and the generated value initializes the thread-local cell for that thread. The initializer actions are run in an unspecified order. -- The primitives of the TL are strictly limited and include only actions which have no observable side effects (a proposed list of primitives is listed below). A TL action may read from (but NOT write to) thread-local cells in the bank of the sparking thread (the bank of the thread calling forkIO, or the special phantom bank for the main thread). -- Any exceptions generated during a thread-local initilization action are propigated to the thread which called forkIO/forkOS or, in the case of the main thread, directly to the runtime system just as though an uncaught exception bubbled off the main thread. -- New IO primitives are added to read from, write to and clear (set to empty) thread-local variables. Advantages: This proposal seems to hit most of the use cases that I recall having seen (including the very important allocate-a-top-level-concurrency-variable use case) and seems to provide a nice way to reinterpret some of the "magic" currently in the standard libraries. In addition, this proposal does not suffer from the module loading order problem that some previous proposals have; because thread local initializer actions depend only on the "previous" bank of values, the order in which they are run makes very little difference (only for the primitives that read clock time or some such). The value of a thread-local cell is always well-defined, even before the main thread starts. Values in a thread-local have a well defined lifetime that is tied to the owning thread. I think that efficient implementation is possible (maybe we can play some copy-on-write games?). I especially like that variables are only as "global" as desired for any given task; if a library writer uses thread-locals for some manner of shared "global" state, later users are always able to write programs that use more than one instance of the "global" state without needing to alter the library. Disadvantages: Requires a language extension (but I don't know of a serious alternate proposal that doesn't). Requires non-trivial runtime system support. Not sure what effect this has on garbage collection. Adds overhead to thread creation (this could perhaps be mitigated by introducing new primitives that distinguish heavyweight threads with their own thread-local banks from lightweight threads, which do not have separate thread-local banks). Its a bit complicated. You can shoot yourself in the foot (true of most of the other proposals). Some representative use cases: -- Implicit parameter style use case: You want to provide a default value that you expect will be rarely changed. Threading the parameter deeply through the code obsfucates meaning and the code is in the IO monad. Solution: * Define a thread-local variable for the value and set the initializer to set the variable to some default value if it was empty, or to copy the parent thread's value otherwise. * If desired, change the variable's value early in main (before any other threads are sparked); the new value will be propagated to all new threads and be available in main. * If desired, different threads can set different values of the parameter which will then be propagated to their sub-threads. -- Top level synchronization variable use case: You need an MVar to manage some "global" resource. Solution: * Define a thread-local variable to hold the MVar. Define the initializer to create a new MVar if the TLRef was empty, and to copy the parent thread's value otherwise. * Read the MVar from the thread-local var, and use it as usual. * Also, allows you to partition the program into sandboxes which use distinct MVars to manage distinct pools of the "global" resource without needing to change or complicate any underlying libraries. -- Running time statistics use case: You want to easily keep track of how long each thread in a program has been running in wall-clock time. Solution: * Define a thread-local variable with an initializer that reads the current wall-clock time. * Calculate the thread running time by taking the difference between the current wall-clock time and the time in the thread-local var. -- Give better semantics to standard handles use case: You want to make the handling of stdin, stdout and stderr in System.IO less "magic" and baked-in. Solution: * Define a thread-local variable for each of stdin, stdout, and stderr. The initializer action creates appropriate handles for each one in the case that the thread-local was empty, and copies the parent's value otherwise. * Nice feature: allows you to override the values of stdin, stdout and stderr for sub-threads, shell style. -- Give better semantics to getArgs use case: You want getArgs/withArgs and getProgName/withProgName to have better semantics. Solution: * Define a thread-local variable for the list of arguments and for the program name. Define an initializer which reads these values from C land when the thread-local value is empty and copies the parent thread's value otherwise. * Allows you to spark multiple threads in a single program with different "command line" arguments. The proposed list of new primitives: -- thread-locals maintained by the standard libs currentWorkingDirectory :: TLRef FilePath stdin :: TLRef Handle stdout :: TLRef Handle stderr :: TLRef Handle -- in the TL monad readTL :: TLRef a -> TL a -- ^ Reads a thread-local variable in the bank of the parent thread. -- Returns bottom if the cell is empty. tryReadTL :: TLRef a -> TL (Maybe a) -- ^ Reads a thread-local variable in the bank of the parent thread. -- Returns Nothing if the cell is empty. getClocktimeTL :: TL ClockTime getCPUTimeTL :: TL Integer getDefaultStdinHandle :: TL Handle getDefaultStdoutHandle :: TL Handle getDefaultStderrHandle :: TL Handle getDefaultArgs :: TL [String] getDefaultProgramName :: TL String newIORefTL :: a -> TL (IORef a) newMVarTL :: a -> TL (MVar a) newEmptyMVarTL :: TL (MVar a) newSTRefTL :: a -> TL (STRef a) -- in the IO monad readThreadLocal :: TLRef a -> IO a -- ^ bottom on empty tryReadThreadLocal :: TLRef a -> IO (Maybe a) -- ^ Nothing on empty writeThreadLocal :: a -> TLRef a -> IO () clearThreadLocal :: TLRef a -> IO () -- ^ Reset the thread-local to empty clearBank :: IO () -- ^ Clear all thread-local cells in the current thread ------------------------------------------------------------------------ ---------------- In order to get discussion flowing and experiment with the semantics I have put together a demonstration module which implements thread-local variables using the standard "unsafePerformIO" hacks. The module and an example usage are attached. If there is any interest in these ideas, I will post this proposal to the wiki. Please respond with thoughts and comments, Rob Dockins Speak softly and drive a Sherman tank. Laugh hard; it's a long way to the bank. -- TMBG

2 2

Space usage problems
by Ian Lynagh 17 Jan '06

17 Jan '06

Hi all, I am having space issues with some decompression code; I've attached a much simplified version as Test1.hs. At the bottom (foo/bar) is the equivalent of deflate. This should be a standalone module which doesn't know about the rest. In the middle (readChunks) is the equivalent of gunzip. It repeatedly calls foo until there is no more input left. At the top is a simple main function that calls them. If I do dd if=/dev/zero of=data bs=1000 count=3000" # making data around 3MB ghc --make Test1 -o Test1 -O -Wall ./Test1 then in top I see Test1 increasing memory usage to around 150MB. I think this is because the "let (ys, zs) = foo xs" means zs holds on to xs (it's hard to be sure as compiling for profiling is too happy to change the behaviour). I tried (Test2) changing foo to be a monad transformer over the calling monad, so the caller's remaining input was updated as we went along, but (as well as memory usage not obviously being fixed) this is giving me a stack overflow. Has anyone got any suggestions for making a constant space, constant stack version? Thanks Ian

5 11

libtiff
by Green Bryan - bgreen 17 Jan '06

17 Jan '06

Does anyone know of libtiff having been wrapped for Haskell? If not, where could I find a good tutorial on wrapping a c library with functions that have variable argument parameters? Thanks, Bryan Green *************************************************************************** The information contained in this communication is confidential, is intended only for the use of the recipient named above, and may be legally privileged. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please resend this communication to the sender and delete the original message or any copy of it from your computer system. Thank You. ****************************************************************************

2 1

Shootout favouring C
by Daniel Fischer 17 Jan '06

17 Jan '06

Hi, Is it only my machime, or can you confirm that for the Ackermann benchmark, it's very good for C that they chose 9 and not a larger value? For 10, we are significantly faster and for 11,12,13, we can run rings around the C-programme: dafis@linux:~/Documents/haskell/shootout> time ./cacker3 10; time ./acker 10; time ./cacker3 11; time ./acker 11; time ./cacker3 12; time ./acker 12 Ack(3,10): 8189 real 0m0.664s user 0m0.660s sys 0m0.010s Ack(3,10): 8189 real 0m0.405s user 0m0.400s sys 0m0.000s Ack(3,11): 16381 real 0m6.255s user 0m6.220s sys 0m0.000s Ack(3,11): 16381 real 0m1.731s user 0m1.710s sys 0m0.000s Ack(3,12): 32765 real 0m35.270s user 0m35.050s sys 0m0.000s Ack(3,12): 32765 real 0m10.673s user 0m10.590s sys 0m0.000s dafis@linux:~/Documents/haskell/shootout> time ./acker 13; time ./cacker3 13 Ack(3,13): 65533 real 1m4.476s user 1m4.050s sys 0m0.010s Ack(3,13): 65533 real 2m50.645s user 2m47.020s sys 0m0.020s Cheers, Daniel

8 19