lonetiger@gmail.com writes:
Hi Ben,
Thanks for the reply!
Sure.
Hi *,
I’ve been working the past 4 or so months on reviving dynamic linking support for Windows in a way that has the most chance of working.
My first patch in the series is up on Phabricator and with this patch dynamic linking work again, but only for the threaded RTS.
Thanks for all of your work on this, Tamar!
Home stretch :)
Yay!
Hmm, why? I thought recent Windows releases had a notion of "user local" installation, no? From what little I have heard it sounds like SxS assemblies is the right solution here.
Yes, so to be clear, SxS absolutely solve this problem. For final installs. The majority of the issue is that the testsuite won't have the assemblies in the SxS cache.
There *is* a sort of RPATH equivalent for SxS that can be used here, it however has two problems: 1) Even though the API has no such limit, the implementation in the Windows loader limits it per application to 5 entries.
(╯°□°）╯︵ ┻━┻
Obviously this won't be enough. So this is absolutely another option (maybe even preferable now that I think about it since it require almost no code change, mostly some build system changes.). Can do either one of two things: a) Copy all dll's to the lib folder when they're compiled instead of leaving them in place and add a single SxS search entry to find them. b) Turn of SxS in the testsuite for any Assemblies not the RTS, and add the inplace RTS directory to the SxS search path. Since it's only the RTS that's an issue.
2) The other problem is that the paths specified have to be relative to the application. (Of the top of my head) It doesn't support absolute paths. Which means I can't have GHC generate the entry because I have no idea where the testsuite intends to run the binary. One way around this is to have the testsuite generate the needed config file. That should be do-able.
I'll investigate this method first. I had discarded it for some reason before but now can't remember...
Right, it sounds like this is a workable option and the fact that it requires adding no further complexity to the compiler is quite a merit. The only question is what other use-cases might run into this same issue. For instance, what happens when I run `cabal build` and try to run an executable from `dist/build`. Then I run `cabal install` and run it from `.cabal/bin`. Surely Cabal will need to take some sort of action in this case. I suppose this means that using plain `ghc -dynamic` alone is probably out of the question.
This is a problem for running tests in the testsuite using the inplace GHC.
Typically on Windows the way you would do this is by delay loading the dll. This allows me to write some code on startup and manually load the runtime dll. The Windows loader would then just use the loaded dll. Unfortunately delay loading does not support const extern data. Such as const extern RtsConfig defaultRtsConfig;
Silly Windows.
Yeah, unfortunately this is because this isn't done by any OS functions. Lazy loading is purely something implemented by linkers on Windows and the appropriate runtimes. In essense it just creates a stub that replaces all functions, which first checks if the dll is loaded, if not load it and then call the real function. Which is why it only works for functions.
Ahhh. I see, that makes sense.
The RTS/GHC is full of such usage so it won’t be easy to change. Though I’d only have to change those exposed by Rts.h.
So I have two options: 1) Replace all const externs with a function call. This would work, but isn’t ideal because it would break if someone in the future adds a new data entry instead of a function. And we have an extra function call everywhere.
Right, I'm really not a fan of this option. Crippling the RTS's use of C on account of arbitrary limitations of the Windows dynamic linker doesn't seem very appealing.
I was not a fan of this either. Would imagine I would be chased around with flaming pitchforks were I to do this..
Yes, sharp, rusty, flaming pitchforks.
2) I could do some hacks on the Windows side, e.g. compile the program to a shared library, embed the shared library inside the exe and on startup after loading the propert rts, load the DLL from (mmapped) memory and run the code.
This sounds like it would get the job done, although it certainly adds complexity. Do you have any intuition for how much work it would be to implement this?
We sorta already do 80% of this. For Windows when making a dynamic version of GHC, the ghc-stage2.exe is a very thin shell, who's only purpose is to load the right libraries and change the search path to include the lib folder. The actual code is in a dll named e.g. ghc-stage2.exe.dll. The change needed would be to embed this dll into the exe (which is trivially done), and then load it from memory. This would require some work but there are enough wrapper code out there with appropriate licenses that we can use to accomplish this. Or at least get a running head start.
Alright, good to know. We can keep this one in our back pocket in case the option described about doesn't work out. Cheers, - Ben