Hi Tamar,
WINDOWS
=======
On Windows I did the following changes before running the 'plugin09' test:
1.) In the compiler (HscMain.hs), just before calling the plugin function 'parsedPlugin', I set BufferMode of the file stdout to LineBuffering.
2.) At the same place, I write a message to stdout with the text "COMPILER About to call plugin parse" and the result of the buffer-mode query.
3.) In the plugin, in the function parsedPlugin, I query and print (to stdout) the buffer mode.
4.) I added the heading PLUGIN to the normal parse message issued by the parsedPlugin function
5.) In the compiler (HscMain.hs) just after returning from the plugin, I print the line "COMPILER Returning from plugin parse" to stdout.
6.) In the plugin function interfaceLoadPlugin' that is called much later, I flush stdout, and add the heading "PLUGIN".
This gives the following interesting result:
COMPILER About to call plugin parse: LineBuffering
COMPILER Returning from plugin parse
PLUGIN Buffermode: BlockBuffering Nothing
PLUGIN parsePlugin(a,b)
PLUGIN interfacePlugin: Prelude
...
The output lines do not appear in the sequence they were produced!!
The plugin doesn't see/inherit the BlockBuffer mode (LineBuffering) set by the compiler!!
This is a strong indication, that there are two different buffers for stdout. One in the compiler and another one in the plugin.
At the end of the processing, the buffer in the compiler is automatically flushed, however the buffer in the plugin never gets flushed!
LINUX
=====
I did a similar test in Linux, however, here I set the buffer mode to 'Blockmode Nothing' and I didn't do a manual flush in the plugin. I got the following result:
COMPILER About to call plugin parse: Buffering mode: BlockBuffering Nothing
PLUGIN Buffering: BlockBuffering Nothing
PLUGIN parsePlugin(a,b)
COMPILER Returning from plugin parse
PLUGIN interfacePlugin: Prelude
...
Here the lines are in the same order as they were produced.
The setting of the Buffering mode is inherited by the plugin.
I think, on Linux the compiler and the plugin share the same buffer.
To fix the issue on Windows, the compiler and the plugin should use the same buffer for stdout.
However I don't know whether this is possible / difficult / easy?
What's your opinion?
Many thanks and kind regards
Roland
Here are my changes for Windows in code:
Change in HscMain the line "import System.IO (fixIO)" to "import System.IO"
Last lines of function HscMain.hs:hscParse'
-- apply parse transformation of plugins
let applyPluginAction p opts
= parsedResultAction p opts mod_summary
liftIO $ hSetBuffering stdout LineBuffering
mode <- liftIO $ hGetBuffering stdout
liftIO $ putStrLn ("COMPILER About to call plugin parse: " ++ show mode)
rsxresult <- withPlugins dflags applyPluginAction res
liftIO $ putStrLn "COMPILER Returning from plugin parse"
return rsxresult
New code for function SourcePlugin.hs:parsedPlugin
parsedPlugin opts _ pm
= do
mode <- liftIO $ hGetBuffering stdout
liftIO $ putStrLn $ "PLUGIN Buffermode: " ++ show mode
liftIO $ putStrLn $ "PLUGIN parsePlugin(" ++ intercalate "," opts ++ ")"
return pm
New code for function SourcePlugin.hs:interfaceLoadPlugin'
interfaceLoadPlugin' :: [CommandLineOption] -> ModIface -> IfM lcl ModIface
interfaceLoadPlugin' _ iface
= do liftIO $ putStrLn $ "PLUGIN interfacePlugin: "
++ (showSDocUnsafe $ ppr $ mi_module iface)
liftIO $ hFlush stdout
return iface
Am Dienstag, den 04.12.2018, 00:02 +0000 schrieb Phyx:
Hi Roland,
Thanks for looking into these.
>
I looked into the testcases 'plugins09', 'plugins10' and 'plugins11' and
found the following: GHC-Windows uses BufferMode 'BlockBuffering
Nothing', however, GHC-Linux uses 'LineBuffering'.
Ah, yes, this isn't technically a Linux vs Windows thing, GHC will always default to LineBuffering for terminals and BlockBuffering for anything else. The issue is that msys2 terminals have std handles that are backed by files instead of pipes. This is an artifact of how they try to emulate posix shells, See
https://github.com/msys2/msys2/wiki/Porting#standard-streams-in-mintty.
This means that when GHC runs inside an msys2 program such as bash it'll always default to BlockBuffering.
>
I don't know anything about the "Why" and "Where" in the GHC IO module on Windows, so I'm unable to come up with a patch.
The above said the handles should be getting flushed when the finalizers are run, but these aren't 100% guaranteed so if the tests rely on this then your solution (to force buffer mode to LineBuffering) sounds like perfectly adequate. Could you put a patch up with that?
My new I/O manager takes a different approach to determine the buffer mode, but I still have some kinks to work out before posting it.
>
PS: I can't say anything about the tests 'plugin-recomp-pure' and
'plugin-recomp-impure' as these tests run successfully on my (slow)
Windows box.
These don't fail for me or harbormaster either, so if Simon is able to consistently reproduce these then I'll have to ask him for a core dump so I can take a look.
Thanks again,
I appreciate the help!
Regards,
Tamar
Hi Tamar,
I looked into the testcases 'plugins09', 'plugins10' and 'plugins11' and found the following: GHC-Windows uses BufferMode 'BlockBuffering Nothing', however, GHC-Linux uses 'LineBuffering'.
If I add an 'import System.IO' and the line 'liftIO $ hSetBuffering stdout LineBuffer' as first line in the do block of the function '.../testuite/tests/plugins/simple-plugin/Simple/SourcePlugin.hs:parsedPlugin' then all 3 tests pass successfully on Windows!
I don't know anything about the "Why" and "Where" in the GHC IO module on Windows, so I'm unable to come up with a patch.
Regards
Roland
PS: I can't say anything about the tests 'plugin-recomp-pure' and 'plugin-recomp-impure' as these tests run successfully on my (slow) Windows box.