Pure File Reading (was: Dealing with configuration data)
Dear all, At the moment, a discussion on haskell-cafe is going on about how to neatly program the fact that an entire program depends on a number of parameters that are read in once at the beginning of a program. The suggestion that many people came up with was using unsafePerformIO in the beginning to read the file. Here is my version of that: | data Configuration = ... -- config data | | getConfig :: Configuration | getConfig = unsafePerformIO $ | do ...read configuration from file... | return configuration | | main = | do doStuff It is quite disturbing that there is no other easy way to do this than using unsafePerformIO (except for using implicit parameters perhaps, but there are other reasons for not using those). I have been thinking a little bit more about this and here is what I found. Remember the Gofer days, when Gofer had a "function": openFile :: FilePath -> String This was of course a cheap and dirty way of implementing things like the getConfig above, but it is impure. However, one could imagine a functional version of this function: readFileOnce :: FilePath -> Maybe String This function will read the contents of the file (and return Nothing if something went wrong), but it is memoized, so that the second time you use this function you get the same result. So, it is a pure function. (Admittedly, it is somewhat unpredictable, but you will always get the same result for the same arguments.) It is no more strange than GHC's pure version of the getArgs function (I forgot what it was/is called). How about space behavior, you say? Reading a file, and memoizing the result means storing the whole contents of the file in memory! The point is that the use of this function will typically happen at the beginning of a program, when reading the configuration file(s). When all this has happened, the function readFileOnce, and its memo table, will be garabage collected. (Of course there is no guarantee that all calls to readFileOnce will be evaluated at the beginning of a program, and it is not required, but when you do, there are no space problems.) There could of course be pure "-Once" versions of other IO operations. Here is a list of possibilities: - reading a file - getting arguments - getting environment variables - downloading a webpage - ... What do you think? Regards, /Koen. -- Koen Claessen http://www.cs.chalmers.se/~koen Chalmers University, Gothenburg, Sweden.
There is another solution to the problem of configurational parameters. The main part of the solution is portable, does not depend on any pragmas, does not use unsafe operations, does not use implicit parameters, and does not require any modifications to the user code. I must warn that it is also potentially vomit-inducing. It seems that the problem at hand naturally splits into two phases: building the configuration environment, and executing some code in that environment. The phases are executed sequentially. The facts suggest the use of a SupedMonad. SuperMonad is very well known and often used, even by people who never heard of simpler monads. The following code is an illustration. Suppose file '/tmp/a.hs' contains the following user code, which is to run within the configuration environment provided by the module Config. For simplicity, our configuration is made of one Int datum, config_item:
File "/tmp/a.hs"
import Config (config_item)
foo = "foo shows: " ++ (show config_item)
bar = "bar shows: " ++ (show config_item)
main = do print foo print bar print foo
We specifically illustrate the reading of the config item several times. The following code runs the first phase: reads the configuration, build the SuperMonad and runs the SuperMonad.
import System (system, ExitCode(ExitSuccess))
myconfig_file = "/tmp/config"
phaseII_var = "/tmp/Config.hs" phaseII_const = "/tmp/a.hs"
nl = "\n"
writeConfig :: Int -> IO () writeConfig num = do writeFile phaseII_var $ concat ["module Config (config_item) where", nl, "config_item =", show num, nl]
runSuperIO () = system ("echo main | hugs " ++ phaseII_const) >>= \ExitSuccess -> print "Phase II done"
main = readFile myconfig_file >>= writeConfig . read >>= runSuperIO
I did warn you, didn't I? I have a hunch this solution will work with GHC even better than it works with Hugs. Perhaps we can even play with some dynamic linking tricks (like shared object initializers, etc). BTW, the solution above is similar in spirit to the following trick in C++: Config config; int main() { /* pure functional C++ code here -- yes, it exists*/} the constructor for 'config' is guaranteed to run before main(). Perhaps someone will implement Staged Haskell one day?
This message seems to have been lost and I'd like to try to breathe some life into it. First, a question: could such "readFilePure" functions be implemented on TOP of the current IO module (perhaps in IO.Pure or something). Of course, one could do something like: readFileOnce :: FilePath -> Maybe String readFileOnce = unsafePerformIO .... {-# NOINLINE readFilePure #-} but this is the sort of thing we're trying to get away from anyway. There doesn't (to me, at least) seem to be an obvious way to do this. It seems to be the sort of thing that requires compiler support. In this case, do any of the compiler implementers have a heart to tackle such a thing? On the other hand, if there's a way to do it on top of what already exists, I would be more than happy to implement it if someone were to point me in the right direction...
The point is that the use of this function will typically happen at the beginning of a program, when reading the configuration file(s). When all this has happened, the function readFileOnce, and its memo table, will be garabage collected.
I like this, and it works for configuration files, but I have another problem I would like to solve with this whole ...Once business which does not fit into this model. I have a large database-like-file which essentially contains an index at the beginning. When you want to look up something, you binary search for the term in the index, find the position of the entity you want, seek to that location and then read a specified amount. The way I have this currently set up is that everything in my program is embedded in the IO monad because 1) the database is huge and i cannot store it all in memory 2) usually only about 100 out of 250000 entries are queried per run, but which entries these are change from run to run Unfortunately, this means all my functions are monadic. However, there's no reason for them to be (in a sense): they are perfectly pure. In fact, I don't even have write access to the database :), but no one would ever change it anyway. So while I like the 'readFileOnce' and variants, I think that if someone is serious about this '...Once' stuff, we should have more or less the entire reading portion of the IO library in pure format for cases like this. Thoughts? - Hal
participants (3)
-
Hal Daume III -
Koen Claessen -
oleg＠pobox.com