Re: Q Monad State

On Fri, Mar 10, 2017 at 6:22 PM, Ethan Pailes wrote:

I had not considered the issue about incremental compilation. It certainly feels like it should cause problems. I wanted to confirm your suspicious, so I threw together an example project which is meant to cause the issues that you mentioned, but I was unable to produce any sort of error. I've thrown the code up at https://github.com/ethanpailes/th-ioref if you want to take a look. Basically there is a Ref module which makes an IORef, then a Quote module which reads it and a Lib module which uses the Quote module to add a top level value which depends on the contents of the IORef. I tried building it, then just modifying Quote and rebuilding. I was expecting exactly the sort of error you described, but everything works fine. When I added an executable which depends on the library, that also works fine. I feel like there should be an error though, so I would appreciate it if you could take a look at the code and try to figure out how to produce the sort of error that it seems like should occur. If it turns out that this does actually work, it would be great to know why because your point really seems like it should be right.

I don't see any writing of the IORef in this example. Indeed, if I add $(runIO (writeIORef ref 5) >> return []) To Lib.hs, it still just prints out 4 for both. You *really* won't be able to have the IORef transfer info from Lib.hs to Main.hs, since they are in different cabal components and so will be compiled by different invocations of ghc.

...

A third approach to consider is to just write out a file for each module, listing the metadata. When compiling other modules, you can use `reifyModules` to get the imports, so that you know which metadata files to load. One issue with this (as well as the IORef) is that it won't work across multiple packages.

Do you know if there is a way to learn where the build output is going in the Q Monad? It would be much better to write the tmp files out to some place in .stack_work rather than in /tmp. That way if /tmp disappears we don't loose the cached data.

Could use findCabalFile from https://hackage.haskell.org/package/th-utilities-0.2.0.1/docs/TH-RelativePat... to find the cabal file, take the dir that file is in, and add "/.stack-work" to it.

Ethan

On Fri, Mar 10, 2017 at 5:47 PM, Michael Sloan wrote:

...

Hi!

I don't think the IORef approach will work well, because GHC won't recompile modules unless they've changed (or if a file added via qAddDependentFile changes). So if you've done a compile of module A, and B depends on it, and then you just change B, only B will recompile and you won't have info from A.

Not sure if this applies to your circumstance, but one issue with the top level declaration approach is that TH can't enumerate the members of a module. One hacky way to write down such metadata in a retrievable way is to use typeclass instances. For example you might have

class TypeInfo (name :: Symbol) (typeInfo :: Symbol)

Then, you can reify the typeclass to get all its instances. As long as you only need to write out this info from declaration quasiquotes, this info ought to be available for typechecking later in the module.

A third approach to consider is to just write out a file for each module, listing the metadata. When compiling other modules, you can use `reifyModules` to get the imports, so that you know which metadata files to load. One issue with this (as well as the IORef) is that it won't work across multiple packages.

-Michael

On Fri, Mar 10, 2017 at 12:29 PM Ethan Pailes wrote:

...

Hi,

I've been working on PADS with Kathleen Fisher for the past couple of months, and we have some questions about the Q Monad.

In working on extending the PADS language, we have encountered a need to maintain a type environment across invocations of the [pads||] quasi quoter. This allows PADS types to be defined in one quote block and used in a later quote block. Unfortunately, the right way to do this is not obvious from the Q Monad’s interface[1]. The monad provides `qGetQ` and `qPutQ` methods to provide a place to store state across quotes, but unfortunately this state is local to each module. It is desirable from a usability perspective to be able to reuse PADS declarations across modules, but in the particular case of PADS the problem is actually worse than that. PADS ships with a standard library module which uses PADS to define several of the core types required to be productive, so without the ability to do type checking across modules the use of this standard library becomes impossible. There are two solutions to the problem that we have considered.

We can generate metadata about each type as it is defined and emit a top level declaration which contains this metadata.

This clutters the global namespace.

It lets you examine the metadata for each type easily in `ghci` which leads to a nice debugging workflow.

If you want to perform any typechecks or other calculations based on the metadata, they must happen after generated code is spliced in.

This means you lose a lot of type safety because everything must be represented as an `Exp` (even with typed template haskell there is no way to get a type for an `Exp` of the form `VarE <<name of metadata>>`)

It becomes impossible to report type errors at compile time. This is a dealbreaker because otherwise, what is the point?

Using `unsafePerformIO` to create an IORef and then using the Q Monand’s `qRunIO` function to access the state cell.

This works smoothly across modules

You get type safety back because everything happens in the Q Monad. You don’t have to generate code in order to get at the values.

It uses `unsafePerformIO` and `qRunIO` so you have to worry about how often everything gets run.

It seems like it should be safe with a {-# NONINLINE #-} pragma for the code which creates the IORef.

The IORef is hidden, and no one else should be able to modify it.

Adding stuff to a `Data.Map.Strict` (the environment implementation we are using) is idempotent, so the quote code getting run multiple times should not be an issue.

We have two main questions based on this.

Is the IO (ab)use outlined above really as safe as we think it is?

Given that environments are such a common requirement in compiling programming languages, and the goal of quasi quotation is allowing easy creation of EDSLs for Haskell, does it make sense to provide some state mechanism for the Q monad which is not restricted by modules.

[1]: https://hackage.haskell.org/package/template-haskell-2.11.1.0/docs/Language-...

Thanks,

Ethan Pailes

_______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries