Weird "missing hi file" problem with a serializable Core patch

Hi all, Following a short chat in #ghc last week, I did a first attempt of reusing existing Iface logic to implement serialization for codegen-related Core. The implementation is included in the attached patch (~100 loc). As a quick and dirty validation of whether it works, I also modified the codegen pipeline logic to do a roundtrip: after CorePrep, the Core bits are converted to Iface, then we immediately convert it back and use it for later compiling. With the patch applied, stage-1 GHC would produce a "missing hi file" error like: : Bad interface file: _build/stage1/libraries/ghc-prim/build/GHC/Types.hi _build/stage1/libraries/ghc-prim/build/GHC/Types.hi: openBinaryFile: does not exist (No such file or directory) The error surprises me, since by the time we perform the Core-to-Core roundtrip, the .hi file should already have been written to disk. Is there anything obviously wrong with the implementation? I'd appreciate any pointers or further questions, thanks a lot! Best regards, Cheng

Without looking at the implementation, it looks to me like the filename is
doubled for some reason. This may suggest places to look.
On Wed, Sep 16, 2020 at 2:57 PM Cheng Shao
Hi all,
Following a short chat in #ghc last week, I did a first attempt of reusing existing Iface logic to implement serialization for codegen-related Core. The implementation is included in the attached patch (~100 loc). As a quick and dirty validation of whether it works, I also modified the codegen pipeline logic to do a roundtrip: after CorePrep, the Core bits are converted to Iface, then we immediately convert it back and use it for later compiling.
With the patch applied, stage-1 GHC would produce a "missing hi file" error like:
: Bad interface file: _build/stage1/libraries/ghc-prim/build/GHC/Types.hi _build/stage1/libraries/ghc-prim/build/GHC/Types.hi: openBinaryFile: does not exist (No such file or directory)
The error surprises me, since by the time we perform the Core-to-Core roundtrip, the .hi file should already have been written to disk. Is there anything obviously wrong with the implementation? I'd appreciate any pointers or further questions, thanks a lot!
Best regards, Cheng _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
-- brandon s allbery kf8nh allbery.b@gmail.com

Thanks Brandon,
I checked the strace log and before the error is written, there's a log entry:
openat(AT_FDCWD,
"_build/stage1/libraries/ghc-prim/build/GHC/Types.hi",
O_RDONLY|O_NOCTTY|O_NONBLOCK) = -1 ENOENT (No such file or directory)
So it looks like GHC is indeed looking at the correct hi path, not the
doubled path.
On Wed, Sep 16, 2020 at 9:03 PM Brandon Allbery
Without looking at the implementation, it looks to me like the filename is doubled for some reason. This may suggest places to look.
On Wed, Sep 16, 2020 at 2:57 PM Cheng Shao
wrote: Hi all,
Following a short chat in #ghc last week, I did a first attempt of reusing existing Iface logic to implement serialization for codegen-related Core. The implementation is included in the attached patch (~100 loc). As a quick and dirty validation of whether it works, I also modified the codegen pipeline logic to do a roundtrip: after CorePrep, the Core bits are converted to Iface, then we immediately convert it back and use it for later compiling.
With the patch applied, stage-1 GHC would produce a "missing hi file" error like:
: Bad interface file: _build/stage1/libraries/ghc-prim/build/GHC/Types.hi _build/stage1/libraries/ghc-prim/build/GHC/Types.hi: openBinaryFile: does not exist (No such file or directory)
The error surprises me, since by the time we perform the Core-to-Core roundtrip, the .hi file should already have been written to disk. Is there anything obviously wrong with the implementation? I'd appreciate any pointers or further questions, thanks a lot!
Best regards, Cheng _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
-- brandon s allbery kf8nh allbery.b@gmail.com

Cheng Shao
Hi all,
Following a short chat in #ghc last week, I did a first attempt of reusing existing Iface logic to implement serialization for codegen-related Core. The implementation is included in the attached patch (~100 loc). As a quick and dirty validation of whether it works, I also modified the codegen pipeline logic to do a roundtrip: after CorePrep, the Core bits are converted to Iface, then we immediately convert it back and use it for later compiling.
With the patch applied, stage-1 GHC would produce a "missing hi file" error like:
: Bad interface file: _build/stage1/libraries/ghc-prim/build/GHC/Types.hi _build/stage1/libraries/ghc-prim/build/GHC/Types.hi: openBinaryFile: does not exist (No such file or directory)
Hi Cheng, Which module is being compiled when this error is produced? Could you provide -ddump-if-trace output for the failing compilation?
The error surprises me, since by the time we perform the Core-to-Core roundtrip, the .hi file should already have been written to disk. Is there anything obviously wrong with the implementation? I'd appreciate any pointers or further questions, thanks a lot!
Note that interface files are written after the Core pipeline is run. Cheers, - Ben

Hi Ben, The -ddump-if-trace output is attached here. The error is produced when compiling GHC.Types in ghc-prim.
Note that interface files are written after the Core pipeline is run.
Sorry for the confusion, I didn't mean the Core simplifier pipeline. I
mean the "Core -> Iface -> Core" roundtrip I tried to perform using
the output of CorePrep. By the time we do CorePrep, the hi files
should already have been written.
On Wed, Sep 16, 2020 at 11:48 PM Ben Gamari
Cheng Shao
writes: Hi all,
Following a short chat in #ghc last week, I did a first attempt of reusing existing Iface logic to implement serialization for codegen-related Core. The implementation is included in the attached patch (~100 loc). As a quick and dirty validation of whether it works, I also modified the codegen pipeline logic to do a roundtrip: after CorePrep, the Core bits are converted to Iface, then we immediately convert it back and use it for later compiling.
With the patch applied, stage-1 GHC would produce a "missing hi file" error like:
: Bad interface file: _build/stage1/libraries/ghc-prim/build/GHC/Types.hi _build/stage1/libraries/ghc-prim/build/GHC/Types.hi: openBinaryFile: does not exist (No such file or directory)
Hi Cheng,
Which module is being compiled when this error is produced? Could you provide -ddump-if-trace output for the failing compilation?
The error surprises me, since by the time we perform the Core-to-Core roundtrip, the .hi file should already have been written to disk. Is there anything obviously wrong with the implementation? I'd appreciate any pointers or further questions, thanks a lot!
Note that interface files are written after the Core pipeline is run.
Cheers,
- Ben

By the time we do CorePrep, the hi files should already have been written.
I don't think so. When we generate real code we write the interface after the backend has generated the output object. See Note [Writing interface files] in GHC.Driver.Main Cheers, Sylvain On 17/09/2020 12:17, Cheng Shao wrote:
Hi Ben,
The -ddump-if-trace output is attached here. The error is produced when compiling GHC.Types in ghc-prim.
Note that interface files are written after the Core pipeline is run. Sorry for the confusion, I didn't mean the Core simplifier pipeline. I mean the "Core -> Iface -> Core" roundtrip I tried to perform using the output of CorePrep. By the time we do CorePrep, the hi files should already have been written.
On Wed, Sep 16, 2020 at 11:48 PM Ben Gamari
wrote: Cheng Shao
writes: Hi all,
Following a short chat in #ghc last week, I did a first attempt of reusing existing Iface logic to implement serialization for codegen-related Core. The implementation is included in the attached patch (~100 loc). As a quick and dirty validation of whether it works, I also modified the codegen pipeline logic to do a roundtrip: after CorePrep, the Core bits are converted to Iface, then we immediately convert it back and use it for later compiling.
With the patch applied, stage-1 GHC would produce a "missing hi file" error like:
: Bad interface file: _build/stage1/libraries/ghc-prim/build/GHC/Types.hi _build/stage1/libraries/ghc-prim/build/GHC/Types.hi: openBinaryFile: does not exist (No such file or directory)
Hi Cheng,
Which module is being compiled when this error is produced? Could you provide -ddump-if-trace output for the failing compilation?
The error surprises me, since by the time we perform the Core-to-Core roundtrip, the .hi file should already have been written to disk. Is there anything obviously wrong with the implementation? I'd appreciate any pointers or further questions, thanks a lot!
Note that interface files are written after the Core pipeline is run.
Cheers,
- Ben
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
participants (4)
-
Ben Gamari
-
Brandon Allbery
-
Cheng Shao
-
Sylvain Henry