GHC bug? Cabal bug? Haddock bug?
=== Short Story === If I build syb-with-class-0.6 via cabal (cabal configure; cabal build) in the unpacked tar directory, it builds correctly. If I build it via "cabal install" (either from the unpacked directory, or by letting cabal fetch it), then the resulting package is corrupted. In particular, the .hi interface file for Data.Generics.SYB.WithClass.Instances mentions symbols that aren't in the .a file. (Or rather, they have the wrong names.) I compared verbose logs of both builds and the differ only in temporary file names.... execpt that the "cabal install" version builds haddock, as my .cabal/conf file has documentation: True. Turns out that if turn documentation off, then then "cabal install" builds a .hi file that matches the .a file... and all is well. Is this a bug in cabal? cabal-install? ghc? haddock? I have saved logs of all this if anyone wants. - Mark (MtnViewMark) Lentczner markl@glyphic.com http://www.ozonehouse.com/mark/ === Versions === [2373] : cabal -V cabal-install version 0.8.0 using version 1.8.0.2 of the Cabal library [2374] : ghc -V The Glorious Glasgow Haskell Compilation System, version 6.10.4 [2376] : ghc-pkg describe haddock | grep version version: 2.4.2 === Background & Details === I was installing happstack on my Mac with my Haskell Platform (GHC 6.10.4) installation. I have successfully installed dozens of other packages in this environment before, and these results are annomalous. I kicked this off via: cabal install --user happstack This installs many packages, including syb-with-class-0.6, which compiled and installed just fine. When installing happstack-data, and compiling the file Happstack/Data/Proxy.hs, during the Template Haskell step (where things get loaded up in ghci), the build encounters this link error: [ 7 of 16] Compiling Happstack.Data.Proxy ( src/Happstack/Data/Proxy.hs, dist/build/Happstack/Data/Proxy.o ) Loading package ghc-prim ... linking ... done. Loading package integer ... linking ... done. Loading package base ... linking ... done. Loading package syb ... linking ... done. Loading package array-0.2.0.0 ... linking ... done. Loading package bytestring-0.9.1.5 ... linking ... done. Loading package containers-0.2.0.1 ... linking ... done. Loading package packedstring-0.1.0.1 ... linking ... done. Loading package pretty-1.0.1.0 ... linking ... done. Loading package template-haskell ... linking ... done. Loading package syb-with-class-0.6 ... linking ... done. (... many more loads elided...) Loading package HaXml-1.13.3 ... linking ... done. ghc: unknown symbol `_sybzmwithzmclasszm0zi6_DataziGenericsziSYBziWithClassziInstances_dataTypeZMabOQZN_closure' That symbol decodes to something referring to: package: syb-with-class-0.6 module: Data.Generics.SYB.WithClass.Instances reference: dataType[abOQ] The reference turns out to be from Loading Happstack.Data.Default, which in turn imports Data.Generics.SYB.WithClass.Instances. Poking around, I found that the interface (.hi) file for Data.Generics.SYB.WithClass.Instances does indeed export such an object: [2324] : ghc --show-iface Data/Generics/SYB/WithClass/Instances.hi | fgrep dataType[a Data.Generics.SYB.WithClass.Instances.dataType[abOQ]) -} Data.Generics.SYB.WithClass.Instances.dataType[abSm]) -} dataType[abOQ] :: Data.Generics.SYB.WithClass.Basics.DataType dataType[abSm] :: Data.Generics.SYB.WithClass.Basics.DataType But, the library doesn't export it: [2325] : nm libHSsyb-with-class-0.6.a | fgrep dataTypeZMa 0001854c D _sybzmwithzmclasszm0zi6_DataziGenericsziSYBziWithClassziInstances_dataTypeZMaeuiZN_closure 00018604 D _sybzmwithzmclasszm0zi6_DataziGenericsziSYBziWithClassziInstances_dataTypeZMaexTZN_closure These refer to dataType[aeui] and dataType[aexT], which don't exist in the interface file. Something seems amiss here: The interface file is exporting generated names that don't match what the library is exporting. If I do the same investigation with the profiling versions of this module, they match: [2326] : ghc --show-iface Data/Generics/SYB/WithClass/Instances.p_hi | fgrep dataType[a Data.Generics.SYB.WithClass.Instances.dataType[anmx]) -} Data.Generics.SYB.WithClass.Instances.dataType[anq8]) -} dataType[anmx] :: Data.Generics.SYB.WithClass.Basics.DataType dataType[anq8] :: Data.Generics.SYB.WithClass.Basics.DataType markl@mtree ~/Library/Haskell/packages/syb-with-class-0.6/lib/ghc-6.10.4 [2327] : nm libHSsyb-with-class-0.6_p.a | fgrep dataTypeZMa 00032704 D _sybzmwithzmclasszm0zi6_DataziGenericsziSYBziWithClassziInstances_dataTypeZManmxZN_closure 0003282c D _sybzmwithzmclasszm0zi6_DataziGenericsziSYBziWithClassziInstances_dataTypeZManq8ZN_closure These both export something dataType[anmx] and dataType[anq8] These two objects, by the way, stem from the last two lines of Data.Generics.SYB.WithClass.Instances.hs: $( deriveData [''ByteString] ) $( deriveData [''L.ByteString] )
This sounds similar to an issue I was seeing over here:
http://groups.google.com/group/happs/msg/04ecfe4fd6285c0d
The module being compiled also includes TH top-level statements, and
was only reproducible when building from Cabal.
Here's another occurance on a different platform:
http://groups.google.com/group/happs/msg/0f65ba6e4f7e822d
And the workaround:
http://groups.google.com/group/happs/msg/1e7761d421b0e5eb
Here's the GHC bug report: http://hackage.haskell.org/trac/ghc/ticket/3799
Antoine
2010/1/16 Mark Lentczner
=== Short Story ===
If I build syb-with-class-0.6 via cabal (cabal configure; cabal build) in the unpacked tar directory, it builds correctly.
If I build it via "cabal install" (either from the unpacked directory, or by letting cabal fetch it), then the resulting package is corrupted. In particular, the .hi interface file for Data.Generics.SYB.WithClass.Instances mentions symbols that aren't in the .a file. (Or rather, they have the wrong names.)
I compared verbose logs of both builds and the differ only in temporary file names.... execpt that the "cabal install" version builds haddock, as my .cabal/conf file has documentation: True. Turns out that if turn documentation off, then then "cabal install" builds a .hi file that matches the .a file... and all is well.
Is this a bug in cabal? cabal-install? ghc? haddock?
I have saved logs of all this if anyone wants.
- Mark (MtnViewMark) Lentczner markl@glyphic.com http://www.ozonehouse.com/mark/
=== Versions ===
[2373] : cabal -V cabal-install version 0.8.0 using version 1.8.0.2 of the Cabal library
[2374] : ghc -V The Glorious Glasgow Haskell Compilation System, version 6.10.4
[2376] : ghc-pkg describe haddock | grep version version: 2.4.2
=== Background & Details ===
I was installing happstack on my Mac with my Haskell Platform (GHC 6.10.4) installation. I have successfully installed dozens of other packages in this environment before, and these results are annomalous.
I kicked this off via: cabal install --user happstack
This installs many packages, including syb-with-class-0.6, which compiled and installed just fine.
When installing happstack-data, and compiling the file Happstack/Data/Proxy.hs, during the Template Haskell step (where things get loaded up in ghci), the build encounters this link error:
[ 7 of 16] Compiling Happstack.Data.Proxy ( src/Happstack/Data/Proxy.hs, dist/build/Happstack/Data/Proxy.o ) Loading package ghc-prim ... linking ... done. Loading package integer ... linking ... done. Loading package base ... linking ... done. Loading package syb ... linking ... done. Loading package array-0.2.0.0 ... linking ... done. Loading package bytestring-0.9.1.5 ... linking ... done. Loading package containers-0.2.0.1 ... linking ... done. Loading package packedstring-0.1.0.1 ... linking ... done. Loading package pretty-1.0.1.0 ... linking ... done. Loading package template-haskell ... linking ... done. Loading package syb-with-class-0.6 ... linking ... done. (... many more loads elided...) Loading package HaXml-1.13.3 ... linking ... done. ghc: unknown symbol `_sybzmwithzmclasszm0zi6_DataziGenericsziSYBziWithClassziInstances_dataTypeZMabOQZN_closure'
That symbol decodes to something referring to: package: syb-with-class-0.6 module: Data.Generics.SYB.WithClass.Instances reference: dataType[abOQ]
The reference turns out to be from Loading Happstack.Data.Default, which in turn imports Data.Generics.SYB.WithClass.Instances.
Poking around, I found that the interface (.hi) file for Data.Generics.SYB.WithClass.Instances does indeed export such an object:
[2324] : ghc --show-iface Data/Generics/SYB/WithClass/Instances.hi | fgrep dataType[a Data.Generics.SYB.WithClass.Instances.dataType[abOQ]) -} Data.Generics.SYB.WithClass.Instances.dataType[abSm]) -} dataType[abOQ] :: Data.Generics.SYB.WithClass.Basics.DataType dataType[abSm] :: Data.Generics.SYB.WithClass.Basics.DataType
But, the library doesn't export it:
[2325] : nm libHSsyb-with-class-0.6.a | fgrep dataTypeZMa 0001854c D _sybzmwithzmclasszm0zi6_DataziGenericsziSYBziWithClassziInstances_dataTypeZMaeuiZN_closure 00018604 D _sybzmwithzmclasszm0zi6_DataziGenericsziSYBziWithClassziInstances_dataTypeZMaexTZN_closure
These refer to dataType[aeui] and dataType[aexT], which don't exist in the interface file.
Something seems amiss here: The interface file is exporting generated names that don't match what the library is exporting.
If I do the same investigation with the profiling versions of this module, they match:
[2326] : ghc --show-iface Data/Generics/SYB/WithClass/Instances.p_hi | fgrep dataType[a Data.Generics.SYB.WithClass.Instances.dataType[anmx]) -} Data.Generics.SYB.WithClass.Instances.dataType[anq8]) -} dataType[anmx] :: Data.Generics.SYB.WithClass.Basics.DataType dataType[anq8] :: Data.Generics.SYB.WithClass.Basics.DataType
markl@mtree ~/Library/Haskell/packages/syb-with-class-0.6/lib/ghc-6.10.4 [2327] : nm libHSsyb-with-class-0.6_p.a | fgrep dataTypeZMa 00032704 D _sybzmwithzmclasszm0zi6_DataziGenericsziSYBziWithClassziInstances_dataTypeZManmxZN_closure 0003282c D _sybzmwithzmclasszm0zi6_DataziGenericsziSYBziWithClassziInstances_dataTypeZManq8ZN_closure
These both export something dataType[anmx] and dataType[anq8]
These two objects, by the way, stem from the last two lines of Data.Generics.SYB.WithClass.Instances.hs:
$( deriveData [''ByteString] ) $( deriveData [''L.ByteString] )
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Indeed - all those look exactly like the same issue.
And the workaround: http://groups.google.com/group/happs/msg/1e7761d421b0e5eb
That doesn't fix the real issue: It causes happstack-data to not need the thing that is built wrong in syb-with-class. I believe my work-around (build syb-with-class w/o documentation) will be a more universal workaround, as at that point the syb-with-class install is no longer broken.
Here's the GHC bug report: http://hackage.haskell.org/trac/ghc/ticket/3799
Registration for that trac is broken, so I couldn't update that ticket. I see that you've added pointers to my post, thank you! Perhaps you can also add a note that the issue seems to have to do with something that cabal does differently when there is a documentation step enabled. - Mark
2010/1/16 Mark Lentczner
Indeed - all those look exactly like the same issue.
And the workaround: http://groups.google.com/group/happs/msg/1e7761d421b0e5eb
That doesn't fix the real issue: It causes happstack-data to not need the thing that is built wrong in syb-with-class. I believe my work-around (build syb-with-class w/o documentation) will be a more universal workaround, as at that point the syb-with-class install is no longer broken.
Here's the GHC bug report: http://hackage.haskell.org/trac/ghc/ticket/3799
Registration for that trac is broken, so I couldn't update that ticket. I see that you've added pointers to my post, thank you! Perhaps you can also add a note that the issue seems to have to do with something that cabal does differently when there is a documentation step enabled.
antoine-latters-macbook:syb-with-class-0.6.1 alatter$ find . | grep \\.o$ | xargs ls -l -rw-r--r-- 1 alatter staff 121496 Jan 16 20:04 ./dist/build/Data/Generics/SYB/WithClass/Basics.o -rw-r--r-- 1 alatter staff 5888 Jan 16 20:04 ./dist/build/Data/Generics/SYB/WithClass/Context.o -rw-r--r-- 1 alatter staff 157220 Jan 16 20:04 ./dist/build/Data/Generics/SYB/WithClass/Derive.o -rw-r--r-- 1 alatter staff 403216 Jan 16 20:04 ./dist/build/Data/Generics/SYB/WithClass/Instances.o -rw-r--r-- 1 alatter staff 602752 Jan 16 20:04 ./dist/build/HSsyb-with-class-0.6.1.o antoine-latters-macbook:syb-with-class-0.6.1 alatter$ find . | grep \\.hi$ | xargs ls -l -rw-r--r-- 1 alatter staff 33600 Jan 16 20:04 ./dist/build/Data/Generics/SYB/WithClass/Basics.hi -rw-r--r-- 1 alatter staff 2367 Jan 16 20:04 ./dist/build/Data/Generics/SYB/WithClass/Context.hi -rw-r--r-- 1 alatter staff 13960 Jan 16 20:04 ./dist/build/Data/Generics/SYB/WithClass/Derive.hi -rw-r--r-- 1 alatter staff 166039 Jan 16 20:04 ./dist/build/Data/Generics/SYB/WithClass/Instances.hi antoine-latters-macbook:syb-with-class-0.6.1 alatter$ cabal haddock Running Haddock for syb-with-class-0.6.1... Preprocessing library syb-with-class-0.6.1... Warning: The documentation for the following packages are not installed. No
Here's the good stuff. First, I did a "cabal configure && cabal build" for syb-with-class. And then: links will be generated to these packages: ffi-1.0, rts-1.0 Documentation created: dist/doc/html/syb-with-class/index.html antoine-latters-macbook:syb-with-class-0.6.1 alatter$ find . | grep \\.o$ | xargs ls -l -rw-r--r-- 1 alatter staff 121496 Jan 16 20:05 ./dist/build/Data/Generics/SYB/WithClass/Basics.o -rw-r--r-- 1 alatter staff 5896 Jan 16 20:05 ./dist/build/Data/Generics/SYB/WithClass/Context.o -rw-r--r-- 1 alatter staff 157220 Jan 16 20:05 ./dist/build/Data/Generics/SYB/WithClass/Derive.o -rw-r--r-- 1 alatter staff 403216 Jan 16 20:05 ./dist/build/Data/Generics/SYB/WithClass/Instances.o -rw-r--r-- 1 alatter staff 602752 Jan 16 20:04 ./dist/build/HSsyb-with-class-0.6.1.o antoine-latters-macbook:syb-with-class-0.6.1 alatter$ find . | grep \\.hi$ | xargs ls -l -rw-r--r-- 1 alatter staff 33600 Jan 16 20:05 ./dist/build/Data/Generics/SYB/WithClass/Basics.hi -rw-r--r-- 1 alatter staff 2367 Jan 16 20:05 ./dist/build/Data/Generics/SYB/WithClass/Context.hi -rw-r--r-- 1 alatter staff 13960 Jan 16 20:05 ./dist/build/Data/Generics/SYB/WithClass/Derive.hi -rw-r--r-- 1 alatter staff 166039 Jan 16 20:05 ./dist/build/Data/Generics/SYB/WithClass/Instances.hi <<<<< Note that after running "cabal haddock" we re-build all of our .hi and .o files EXCEPT ./dist/build/HSsyb-with-class-0.6.1.o And now, since TH generates random symbols, we have symbols in the new .hi files that aren't in the old (and only) HSsyb-with-class-0.6.1.o. Antoine
AHA!
Note that after running "cabal haddock" we re-build all of our .hi and .o files EXCEPT ./dist/build/HSsyb-with-class-0.6.1.o
And now, since TH generates random symbols, we have symbols in the new .hi files that aren't in the old (and only) HSsyb-with-class-0.6.1.o.
So, this leaves us with two questions: 1) Why does "cabal haddock" rebuild the .hi and .o files? On the face of it, this seems odd: Build documentation and your library gets rebuilt? 2) Why doesn't Instances.o get rebuilt? Surely this has something to do with the fact that Instances.hs contains only orphan instances. But any answer here just leads to a raft of other questions: Surely this problem would plague other modules have have similar source files? What is Haddock doing? If Haddock needs the .hi files, why not just use them? If it just "builds them to be sure", why in the dist tree and not some place temporary? If is going to build .o files, why not all? Curiouser..... - Mark Mark Lentczner http://www.ozonehouse.com/mark/ IRC: mtnviewmark
2010/1/17 Mark Lentczner
AHA!
Note that after running "cabal haddock" we re-build all of our .hi and .o files EXCEPT ./dist/build/HSsyb-with-class-0.6.1.o
And now, since TH generates random symbols, we have symbols in the new .hi files that aren't in the old (and only) HSsyb-with-class-0.6.1.o.
So, this leaves us with two questions:
1) Why does "cabal haddock" rebuild the .hi and .o files? On the face of it, this seems odd: Build documentation and your library gets rebuilt?
Maybe this has already been answered elsewhere: when TH is involved Haddock may rebuild your library. In this case Cabal is directing Haddock's output to the same place that is used for normal builds. This bug has been fixed in Cabal.
2) Why doesn't Instances.o get rebuilt? Surely this has something to do with the fact that Instances.hs contains only orphan instances. But any answer here just leads to a raft of other questions: Surely this problem would plague other modules have have similar source files? What is Haddock doing? If Haddock needs the .hi files, why not just use them? If it just "builds them to be sure", why in the dist tree and not some place temporary? If is going to build .o files, why not all?
Not sure why Instances.o isn't rebuilt. David
participants (3)
-
Antoine Latter -
David Waern -
Mark Lentczner