#10871: Implement "buildable" interface files which can be directly compiled without source -------------------------------------+------------------------------------- Reporter: ezyang | Owner: ezyang Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.11 Resolution: | Keywords: backpack Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Description changed by ezyang: Old description:

A fat interface file is like a normal hi file, but with extra information so that GHC can *recompile* the associated module without having to parse, rename and typecheck the source code again. The proposed command- line interface is something like (subject to bikeshedding):

{{{ ghc -c A.hs -fno-code -fwrite-buildable-interface # produces a .hi-build file, then ghc -c A.hi-build # completes building, as if ghc -c A.hs had been called }}}

The primary motivation for this is to support partially compiling indefinite packages, which cannot be compiled to object code due to the fact that some of the dependencies haven't been provided yet. However, there are some other cases where this might be useful:

1. Supercompilation requires having the source of all bindings, buildable interfaces can make this information available, even beyond what storing inlinings might provide 2. A buildable interface file can be used to build variants of a source, e.g. profiling and optimization, even "on the fly" by GHC if necessary.

And maybe more we haven't thought of yet.

Note: we DO run the desugarer before we write out the buildable interface (so we are essentially serializing a `ModGuts`, not a `TcGblEnv`). This does mean that you can't rebuild a buildable interface with hpc (that's done during desugaring) but that seems like a small price to pay.

New description: A fat interface file is like a normal hi file, but with extra information so that GHC can *recompile* the associated module without having to parse, rename and typecheck the source code again. The proposed command-line interface is something like (subject to bikeshedding): {{{ ghc -c A.hs -fno-code -fwrite-buildable-interface # produces a .hi-build file, then ghc -c A.hi-build # completes building, as if ghc -c A.hs had been called }}} The primary motivation for this is to support partially compiling indefinite packages, which cannot be compiled to object code due to the fact that some of the dependencies haven't been provided yet. However, there are some other cases where this might be useful: 1. Supercompilation requires having the source of all bindings, buildable interfaces can make this information available, even beyond what storing inlinings might provide 2. A buildable interface file can be used to build variants of a source, e.g. profiling and optimization, even "on the fly" by GHC if necessary. 3. GHCi can take advantage of information from buildable interface files to give more detailed information about otherwise "compiled" modules And maybe more we haven't thought of yet. Note: we DO run the desugarer before we write out the buildable interface (so we are essentially serializing a `ModGuts`, not a `TcGblEnv`). This does mean that you can't rebuild a buildable interface with hpc (that's done during desugaring) but that seems like a small price to pay. -- -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10871#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10871: Implement "buildable" interface files which can be directly compiled without source -------------------------------------+------------------------------------- Reporter: ezyang | Owner: ezyang Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.11 Resolution: | Keywords: backpack Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by simonpj): Tidying does a number of things, listed at the top of `TidyPgm`. I have not thought this through fully, but my guess is that most of them aren't what we want for a fat interface file; e.g. dropping rules; the whole `findExternalIds` thing; injecting implicit bindings, etc. But we probably do need one aspect of tidying, namely to rename things. Consider `\x_12 \x_14. x_12`, where the `_n` part is the unique. When we serialise into an interface file we drop the unique, so it's important that the `OccNames` don't shadow. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10871#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10871: Implement "fat" interface files which can be directly compiled without source -------------------------------------+------------------------------------- Reporter: ezyang | Owner: ezyang Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.11 Resolution: | Keywords: backpack Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by ezyang): Here is a comment about the extra things that we seem to need to serialize to a `ModIface`, beyond `IfaceBinding`s: 1. `mg_rdr_env :: GlobalRdrEnv`. Originally, I claimed that this was only used by the vectorizor; however, I was slightly mistaken: it is ALSO used to generate error messages / debug information in the simplifier and interface generation code. For example, we don't warn about orphans until `MkIface`; and to give nice qualified names, we need a `GlobalRdrEnv`. I'm investigating if we can move orphan warnings earlier, but in general to not serialize `GlobalRdrEnv` we will pay a little bit in how good the error messages are (maybe this is OK for the simplifier monad, which I don't believe gives any user-visible messages using this). 2. `mg_hpc_info :: HpcInfo` and `mg_foreign :: ForeignStubs` are directly passed through to code generation during Tidy, so we can't get rid of them. So my inclination is to create a field in `ModIface` like `mi_cg_info :: Maybe CgIface`, which contains just these two bits. I managed to eliminate the usage fields in `ModGuts` by computing usage information earlier, during desugaring. This seems to work quite well. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10871#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10871: Implement "fat" interface files which can be directly compiled without source -------------------------------------+------------------------------------- Reporter: ezyang | Owner: ezyang Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.11 Resolution: | Keywords: backpack Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): -------------------------------------+------------------------------------- Comment (by mgsloan): This is a really cool idea. I'd like to encourage putting as much info as possible into these hi-fat files (or potentially make that an option). For example, type information for every sub expression, exact names for identifiers, and SrcLoc info. Why? Because this could also potentially allow for querying information about libraries without compiling them. For example, you might jump into some library code by using "go-to- definition" in an IDE. It would be excellent to then be able to immediately get info about the code you've jumped into, via these fat hi files. I don't know much about GHC internals, so maybe this doesn't make much sense. But, something to consider! -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10871#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10871: Implement "fat" interface files which can be directly compiled without source -------------------------------------+------------------------------------- Reporter: ezyang | Owner: ezyang Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.11 Resolution: | Keywords: backpack Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by ezyang): I would agree that the primary payoff of this feature is Backpack; without it, I think there is much less motivation. I think the "external core" use-case could be quite compelling, but there's more work to do extracting the interface file parser into its own library. Do fat interfaces violate cabal install "depending on its inputs, and nothing more"? When I try to answer this question, I find myself asking, "what distinguishes these from normal interface files which we save for packages that are already installed?" Let me recap Duncan's new plan for implementing cabal install so that it is minimally affected by the existing database of packages: 1. We first solve dependencies, without making any reference to the existing package database. This allows us to compute a number of IPIDs of configured packages. 2. Then, we *improve* these configured packages into installed packages simply by looking up their IPIDs in the package database. 3. When we compile, we begin as if we had already compiled the already installed packages, and finish up building and installing the rest of the packages in the plan. The emphasis seems to be removing side effects from step (1); but of course we do want to avoid rebuilding things that are already built, so (2) is "effectful" but in a benign way. I think this plan, which avoids side-effects during dependency resolution, works for Backpack fat interfaces too. Here's how it looks: 1. Once again, we solve dependencies. This gives a number of component IDs for all configured components. 2. We improve these configured components into installed components. Some of these will map to pre-compiled components, but others will map to *indefinite components* which were installed with fat interface file 3. When we compile, we begin as if we had compiled all the installed packages, and partially compiled all the indefinite ones, and just "finish" up the rest of the build according to the plan. There is extra complexity with ensuring that instantiated units get deduplicated correctly (arising from Backpack's applicativity) but I really think that fat interface files support the method that you are looking for. Let me also point out the other big difference between fat interface files and source files: GHC *always* knows how to compile fat interface files. You cannot have any sort of custom build system associated with them. They're just a big pile of inlinings; there's no external C code, no preprocessing, etc. This means, for example, that if you want a indefinite component with some C code, the C code gets built ONCE when you initially installed the indefinite package, not each time you instantiate the component. I'd really like to get you on board with this new plan, so let me know if you want to chat about it some more. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10871#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10871: Implement "fat" interface files which can be directly compiled without source -------------------------------------+------------------------------------- Reporter: ezyang | Owner: ezyang Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.11 Resolution: | Keywords: backpack Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonpj): Edward can correct me but I think it's about more than time/efficiency. To start again from source you need to have all your import paths, CPP settings, etc etc recorded so that you can replay them. You need to copy all the source files, including any C bits, into the installation tree. There's a LOT of front-end stuff to worry about. But this way all that is gone. We simply have Core; and any C bits are compiled to .o files. Nice! -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10871#comment:12 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

It feels like something we would have to consider in lots of places,

#10871: Implement "fat" interface files which can be directly compiled without source -------------------------------------+------------------------------------- Reporter: ezyang | Owner: ezyang Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.11 Resolution: | Keywords: backpack Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by ezyang): Aha! SM, your latest comment pinpointed a misunderstanding. You said that "cabal install A B" must work whether or not "cabal install A" was done previously, thus fat interface files are not necessary. But in fact, even in the case of "cabal install A B", we use fat interfaces to compile an instantiated version of A: you will have to compile to a fat interface first, and THEN finish instantiating and finishing the compilation. Why is it set up this way? There were a confluence of reasons that pushed us in this direction: 1. We want GHC(i) to be able to compile Backpack code directly, without requiring Cabal in the loop. 2. Conversely, we don't want to bake Backpack support into cabal-install; if we do, then alternate package managers like Stack also have to port their own support. (A cabal-install which knows, in "cabal install A B", that A is instantiated with B, knows too much about Backpack for its own good!) 3. We always need to typecheck A by itself *anyway*, because it may be ill-typed in a way that you won't discover when you instantiate it. The canonical example is: {{{ unit A where signature A where data T = T signature B where data T module M where import qualified A import qualified B x = A.T :: B.T }}} A shouldn't type-check, because there is no reason to believe A.T and B.T are the same type. But if B implements A and B with the same T, it is actually possible to compile A instantiated with B. OK, let me answer some of the other questions more directly: like hs-boot files, a kind of tax on people who want to write build systems and tools that work with packages. I agree, it does make things more complicated. But you never have to interact with fat interfaces unless you want to build a Backpack package: they all go away once you instantiate things. So I think the complexity here is somewhat opt-in. It's the same deal with some sort of external core format: you don't have to deal with it unless you want to generate external core or compile it.

I wouldn't be surprised if there were tricky technical problems with implementing it fully too. What about core-to-core compiler plugins? What is the equivalent of ghc -M for fat interface files?

Core-to-core plugins run when compiling from hi-fat to hi; that should just work out of the box. As for ghc -M, fat interface files are "essentially" external core input files, i.e. they come with full information about their dependency structure. So you could just write ghc -M which would tell you what order to build hi-fat files in the same way you can do it for hs. (I have not implemented ghc -M, but I have implemented --make's analysis to work on fat interface fiels.)

This would mean that we couldn't have C code in a package that depends on something provided by a hole too. Maybe you don't want to support that use case, but given that without fat interfaces it would "just work", it seems a shame to lose it.

We are giving this up. But I am not losing too much sleep over it; the preprocessor for the C code would already have to know how to mangle the Haskell symbols to point to the right location. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10871#comment:14 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10871: Implement "fat" interface files which can be directly compiled without source -------------------------------------+------------------------------------- Reporter: ezyang | Owner: ezyang Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.11 Resolution: | Keywords: backpack Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by ezyang): Let's think about it from a user perspective. Right now, if I want to compile a simple Haskell file, I don't need to futz around with Cabal: I can just ask GHC to build it for me `ghc --make A.hs`. So we want a similar experience with Backpack: if someone writes a simple Backpack file, they should be able to build it with `ghc --backpack p.bkp`, without futzing around with Cabal. This is MUCH more important for Backpack programming "in the small" (ala ML functors and C++ templates) rather than Backpack programming "in the large" (ala replacing package dependencies). I think it's important that we get a story that works well for both cases; even though the Backpack design is highly oriented towards large-scale modularity, it is much easier to get users using the small-scale features first. (Plus, there's interesting stuff to be done with alternate syntax building on Backpack but better suited for small-scale modularity.) But we can't ask them to use Cabal for that! Re (2), how can you let cabal-install work without knowing about Backpack? If it doesn't know about Backpack units, then it can only operate as it does now: building a graph of packages (packages that use Backpack are still obligated to state, in a build-depends, what external packages they use). So when we say "cabal install A B", we mean, "install package A" and "install package B". What does it mean to install an indefinite package? Typecheck it and install the fat interface files (so we can typecheck against it). What does it mean to install a package which instantiates an indefinite package? Compile it, and also compile anything that it instantiates. With fat interfaces, GHC can manage compiling the instantiated units, since it can just resume that compilation! How about installing the resutls? If Backpack were generative, every package which instantiates indefinite units would just get their own private copy of any other units it instantiated, and together they would constitute the package you would install. Fat interface files work perfectly for this case. But since Backpack is applicative, we also want identical instantiated units to be shared. Here is where you might object, since I am cheating here, but what seems easiest is to have GHC and Cabal (but not cabal-install) conspire to deduplicate instantiated units "behind the scenes". You might say that this will work really poorly for traditional distribution package managers. If C and D both instantiate A(B), where do the "install files" for A(B) live? The only reasonable place to put them in a distro is in their own package, implying that a distro packages you'd create for packages that use Backpack would have to know about how everything is instantiated. But traditional distribution package managers (e.g. deb, rpm) already can't deal with having both foo-0.1 and foo-0.2 installed at the same time. So it's no surprise that they'll have some trouble dealing with Backpack, which subsumes this mode of use. And if you do have a distro that can deal with it (e.g. Nix), you can either bundle private copies of the instantiated copies (and rely on GHC's linker to deduplicate them later, following on how C++ compilers handle this), or you can do as you suggest and run enough of Backpack to figure out how things are instantiated, and then make a build task for each instantiated unit. I think you are right that we will want to build this eventually; but I don't want it to be the default, and I want something simpler for casual use. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10871#comment:16 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10871: Implement "fat" interface files which can be directly compiled without source -------------------------------------+------------------------------------- Reporter: ezyang | Owner: ezyang Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.11 Resolution: | Keywords: backpack Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonpj):

Let me re-answer your questions: What does it mean to install an indefinite package? We install just the interface files, so that we can typecheck against it

We need more than that. We need to compile the indefinite package to code when we fill its holes. So we need access to its source code in some form. One way to do that is to keep the original Haskell source code (pre-cpp, pre-everything) and typecheck it from scratch, being (a) careful to keep a copy of the source code and (b) ever so careful to replay exactly the front-end compiler flags that were used the first time round. We could do this. But it's just easier to snapshot the Core bindings that we already have in our hand. With unfoldings in interface files we already do this; it's mainly a question of keeping all unfoldings, not just some. It's not an efficiency issue. If it's easier to compile again from source (remembering all the C bits etc) then let's do that. To me it looks harder. And that's the core issue to debate. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10871#comment:18 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10871: Implement "fat" interface files which can be directly compiled without source -------------------------------------+------------------------------------- Reporter: ezyang | Owner: ezyang Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.11 Resolution: | Keywords: backpack Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by ezyang): I think the difference in opinion stems from whether or not it's reasonable to ask the user to have built on the non-leaves prior to actually building the unit in question. Of course, the only way the user is going to do this in practice is with external tooling, e.g. cabal- install. But this is true for pre-Backpack packages too; no one really can install a package without help from Cabal. And since packages are the unit of distribution, and you want other things like the ability to download them from Hackage, this is just fine. In contrast, I can work on a multi-module Haskell program without using Cabal, and it would pretty horrible user-experience if I had to "install" every module before I could use it in another program. So here is what I think is the difference of opinion: I want to view Backpack units more as modules than as packages (in the old Cabal sense). And the reason for this is entirely tied up with small-scale use of Backpack. For example: today, if `containers` publishes `Data.Map` which has a map data type, I can use it immediately in an hs file, no fussing about with Cabal. If someone publishes a `data-map` Backpack unit, which has a hole for keys, it would be bad user experience to force a user to use Cabal to build and install all the instantiated copies of `data-map` they might want to use before they can build their application. To add insult to injury, if the key type in question is in the package I was working on, they would have separate it out into new unit containing just their type definition, so that could be installed first and the used to instantiated `data-map`, before they can go ahead and build the package they're thinking about. So, I think your intuition is right when holes are roughly "package" shaped. But I think it's quite important to have reasonable non-Cabal workflow when the holes are smaller, which forces us to have GHC instantiate things on the fly, which leads us to fat interfaces. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10871#comment:20 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10871: Implement "fat" interface files which can be directly compiled without source -------------------------------------+------------------------------------- Reporter: ezyang | Owner: ezyang Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.11 Resolution: | Keywords: backpack Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by ezyang): SM, I think you have convinced me that eventually, we will need a version of cabal-install that is Backpack-aware, and has a work item in its install plan for each instantiated Backpack package. I am also going to claim this implementation of `cabal-install` is not going to be "simple". Let me first state an principle which we have both agreed is desirable: **It should be possible to build a Backpack unit without Cabal** (...assuming that all external dependencies have been built already.) This principle forces a few choices: **We need a separate file format for Backpack.** Suppose that we don't have a separate file format for Backpack, e.g. that all the Backpack information is recorded in a Cabal file (as it was in our older design from a year ago.) GHC can't parse a Cabal file, but it needs to know specifically what unit provides any module it wants to import. How can you get this info to GHC? 1. In our old design, you could pass a pile of `-package` flags with renamings to make sure every module name points to the correct module from the correct instantiated unit. Obviously this is completely unusable by hand; you'd need a tool which knows about Backpack to generate these flags. So much for using Backpack without Cabal! My advisor complained a lot about this, and that was the impetus that lead us down the route to fat interface files. 2. You could write any Backpack-specific information to its own file and pass that to GHC. This is a lot better! Now the only extra information you might need to pass is how the holes are to be instantiated (in case the unit is indefinite but you want to compile it). **It is GHC, not Cabal, who knows how to instantiate units.** Simple corollary of the above. And I also assume that we don't want to duplicate code, so this code for instantiation should only live in one place. Notice that in the old design, it was Cabal that had the Backpack smarts! But I think this was the wrong choice: GHC should have the Backpack smarts. **Cabal needs to ask GHC for information about how it instantiated units.** Since the code for unit instantiation lives in GHC, but each instantiated unit needs to get installed to the database (a Cabal concern), it follows that Cabal needs to call GHC in order to get information about how units are instantiated. This is directly relevant to cabal-install. Traditionally, cabal-install takes the package to be built, resolves the dependency graph, and then builds the dependency graph (now an install plan) in topological order. But if GHC, not Cabal, does instantiation, we don't know what the full set of instantiated units a Backpack package needs until after we have configured it. Essentially, as we process packages in the install plan, after we configure, we have to ask GHC to tell us what other dependencies are needed, and then splice those dependencies into the install plan as extra targets that must be built. This is further complicated by the fact that the Cabal library may be embedded in a custom Setup script; in which case we have to define an interchange format between Cabal and cabal-install to communicate this information. I don't think this is the wrong way to do it, but it will take work to implement. Let me argue for fat interface files one more time. One of my top priorities at this stage is to get Backpack working "just enough" so that we can roll it out and let users (including me) play with it and use it for real applications. So if there is a way to get thing working today (maybe not entirely the Right(TM) way) in a way that doesn't impair our ability to make it work right later, I would quite like to take it. Fat interfaces are a way to get Backpack integrated with the Cabal ecosystem without having to rewrite cabal-install's install plan runner, make it easier to experiment with Backpack, have the property, "If you don't care about it, you don't have to", and most importantly, is mostly implemented at this point. This discussion has also reminded me that there is still an elephant in the room with regards to recompilation avoidance in Backpack. In a package model, GHC will not recompile if the package key of the depended upon package has not changed. Since we don't generally expect package keys to change on an edit/recompile cycle, it's unsound to keep around build files from an old compilation. This means that you end up doing a lot of recompiling with Cabal today when you change a package that is depended upon by other packages you are building. In fact, we want something like this: we don't *care* about recompilation avoidance when we are installing, but if we are edit/rebuilding, we want accurate recompilation avoidance for the entire set of packages which we may be editing and building. This sounds exactly like the "local package database" that we end up creating when compiling Backpack units using fat interfaces. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10871#comment:22 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10871: Implement "fat" interface files which can be directly compiled without source -------------------------------------+------------------------------------- Reporter: ezyang | Owner: ezyang Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.11 Resolution: | Keywords: backpack Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by ezyang):

In my (perhaps naive) view of the way this works, "instantiating a unit" is just compiling modules in dependency order, exactly as GHC does today.

No, more has to be done. For example, imagine package foo and bar set up this way: {{{ -- package foo unit foo where signature H module P }}} {{{ -- package bar unit a-impl where module A unit bar where include p include a-impl (A as H) }}} At the end of the day, we need to compile an instance of foo compiled against h-impl, e.g. `foo(H -> h-impl:A)`. How do we know that this is the case? Backpack specifies an ALGORITHM for figuring it out, which involves taking package `bar`, and then doing analysis on the includes (renames and all) to figure out what units are providing modules that other units require. If you support cross-package mutual recursion, this algorithm can be quite tricky indeed, since you need the business with infinite regular trees. So it seems quite desirable for this logic to live in GHC. So no, Cabal cannot just figure out how to instantiate units by looking at the package database, without also implementing a key part of the Backpack algorithms. (In the old design, Cabal DID implement this algorithm, but we really want to get away from that.) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10871#comment:24 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10871: Implement "fat" interface files which can be directly compiled without source -------------------------------------+------------------------------------- Reporter: ezyang | Owner: ezyang Type: feature request | Status: new Priority: lowest | Milestone: Component: Compiler | Version: 7.11 Resolution: | Keywords: backpack Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by ezyang): * priority: normal => lowest Comment: We have decided to can fat interfaces for Backpack. There might be other uses for this feature but this patchset will have to be picked up when there's another customer for the use-case. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10871#comment:26 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10871: Implement "fat" interface files which can be directly compiled without source -------------------------------------+------------------------------------- Reporter: ezyang | Owner: ezyang Type: feature request | Status: new Priority: lowest | Milestone: Component: Compiler | Version: 7.11 Resolution: | Keywords: backpack Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by ekmett): * cc: ekmett (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10871#comment:28 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler