[GHC] #14030: Implement the "Derive Lift instances for data types in template-haskell" proposal
#14030: Implement the "Derive Lift instances for data types in template-haskell" proposal -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: (none) Type: task | Status: new Priority: normal | Milestone: 8.4.1 Component: Template | Version: 8.3 Haskell | Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- Back in September 2015, I [https://mail.haskell.org/pipermail/libraries/2015-September/026117.html proposed] using the `DeriveLift` extension to, well, derive `Lift` instance for data types in the `template-haskell` library. The proposal was well received, but I was unable to implement it at the time due to `DeriveLift`'s newness (having only been introduced in GHC 8.0). Now that GHC 8.0 is the oldest version of GHC that we support bootstrapping with, this is no longer an obstacle. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14030 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#14030: Implement the "Derive Lift instances for data types in template-haskell" proposal -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: RyanGlScott Type: task | Status: new Priority: normal | Milestone: 8.4.1 Component: Template Haskell | Version: 8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by RyanGlScott): * owner: (none) => RyanGlScott -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14030#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#14030: Implement the "Derive Lift instances for data types in template-haskell" proposal -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: RyanGlScott Type: task | Status: new Priority: normal | Milestone: 8.4.1 Component: Template Haskell | Version: 8.3 Resolution: | Keywords: deriving Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by RyanGlScott): * keywords: => deriving -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14030#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#14030: Implement the "Derive Lift instances for data types in template-haskell" proposal -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: RyanGlScott Type: task | Status: new Priority: normal | Milestone: 8.8.1 Component: Template Haskell | Version: 8.3 Resolution: | Keywords: deriving Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by harpocrates): * cc: harpocrates (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14030#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#14030: Implement the "Derive Lift instances for data types in template-haskell" proposal -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: RyanGlScott Type: task | Status: new Priority: normal | Milestone: 8.8.1 Component: Template Haskell | Version: 8.3 Resolution: | Keywords: deriving Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by RyanGlScott): At one point in time (a release of GHC or two ago), I attempted to implement this, but ran into interface file-related issues. Unfortunately, I no longer have the exact errors in front of me, but I recall that there was some difficulty due to the fact that: 1. We were deriving `Lift` instances in `Language.Haskell.TH.Syntax` 2. The derived `Lift` code references functions from `Language.Haskell.TH.Lib.Internal` 3. Because `Language.Haskell.TH.Lib.Internal` imports `Language.Haskell.TH.Syntax`, there were (seemingly) some issues with import cycles That being said, I just tried implementing this afresh with GHC 8.6.2 as my bootstrapping compiler, and I did not hit any issues at all! This is quite strange, and I'm somewhat paranoid that there //is// an issue lurking underneath the surface that I just haven't exposed yet. While typing this out, I realized one thing. I think the last time I tried this, I might have been using GHC 8.2.2 as my bootstrapping compiler. There is a notable difference between 8.2.2 and future major releases: 8.2.2 does //not// have `Language.Haskell.TH.Lib.Internal` (it wouldn't be until 8.4 that that would come into existence). I think GHC might have been confused because the `DeriveLift`-generated code that the bootstrapping compiler (8.2.2) generated mentioned things from `Language.Haskell.TH.Lib`, but the stage-1 compiler generated `DeriveLift` code that mentioned `Language.Haskell.TH.Lib.Internal`. If that's the case, then that would explain why I can't reproduce the issue now (with 8.6.2). Still, this has me a bit worried that what we're doing here is a bit fragile, and that perhaps moving functions from `Language.Haskell.TH.Lib.Internal` to some other, hypothetical module in the future might cause this issue to surface once more... -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14030#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#14030: Implement the "Derive Lift instances for data types in template-haskell" proposal -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: RyanGlScott Type: task | Status: new Priority: normal | Milestone: 8.8.1 Component: Template Haskell | Version: 8.3 Resolution: | Keywords: deriving Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by RyanGlScott): OK, I decided that I wanted to remember exactly what the error message was, so I compiled GHC 8.4.4 using 8.2.2 as the bootstrapping compiler. Here is the error that I got: {{{ $ PATH=/opt/ghc/8.2.2/bin:$PATH make -j1 ===--- building phase 0 make --no-print-directory -f ghc.mk phase=0 phase_0_builds make[1]: Nothing to be done for 'phase_0_builds'. ===--- building phase 1 make --no-print-directory -f ghc.mk phase=1 phase_1_builds "/opt/ghc/8.2.2/bin/ghc" -hisuf hi -osuf o -hcsuf hc -static -O0 -H64m -Wall -package-db libraries/bootstrapping.conf -this-unit-id template- haskell-2.13.0.0 -hide-all-packages -i -ilibraries/template-haskell/. -ilibraries/template-haskell/dist-boot/build -Ilibraries/template-haskell /dist-boot/build -ilibraries/template-haskell/dist-boot/build/./autogen -Ilibraries/template-haskell/dist-boot/build/./autogen -Ilibraries /template-haskell/. -optP-include -optPlibraries/template-haskell/dist- boot/build/./autogen/cabal_macros.h -package-id base-4.10.1.0 -package-id ghc-boot-th-8.4.4 -package-id pretty-1.1.3.3 -Wall -this-unit-id template- haskell -XHaskell2010 -no-user-package-db -rtsopts -fno-warn- deprecated-flags -odir libraries/template-haskell/dist-boot/build -hidir libraries/template-haskell/dist-boot/build -stubdir libraries /template-haskell/dist-boot/build -c libraries/template- haskell/./Language/Haskell/TH/Syntax.hs -o libraries/template-haskell /dist-boot/build/Language/Haskell/TH/Syntax.o libraries/template-haskell/Language/Haskell/TH/Syntax.hs:2002:43: error: • Failed to load interface for ‘Language.Haskell.TH.Lib’ Use -v to see a list of the files searched for. • In the expression: Language.Haskell.TH.Lib.conE (mkNameG_d "template-haskell" "Language.Haskell.TH.Syntax" "NominalR") In an equation for ‘lift’: lift NominalR = Language.Haskell.TH.Lib.conE (mkNameG_d "template-haskell" "Language.Haskell.TH.Syntax" "NominalR") When typechecking the code for ‘lift’ in a derived instance for ‘Lift Role’: To see the code I am typechecking, use -ddump-deriv In the instance declaration for ‘Lift Role’ | 2002 | deriving( Show, Eq, Ord, Data, Generic, Lift ) | ^^^^ libraries/template-haskell/ghc.mk:3: recipe for target 'libraries /template-haskell/dist-boot/build/Language/Haskell/TH/Syntax.o' failed make[1]: *** [libraries/template-haskell/dist- boot/build/Language/Haskell/TH/Syntax.o] Error 1 Makefile:122: recipe for target 'all' failed make: *** [all] Error 2 }}} If you pass `-v`, you'll see what the culprit is: {{{ $ "/opt/ghc/8.2.2/bin/ghc" -hisuf hi -osuf o -hcsuf hc -static -O0 -H64m -Wall -package-db libraries/bootstrapping.conf -this-unit-id template- haskell-2.13.0.0 -hide-all-packages -i -ilibraries/template-haskell/. -ilibraries/template-haskell/dist-boot/build -Ilibraries/template-haskell /dist-boot/build -ilibraries/template-haskell/dist-boot/build/./autogen -Ilibraries/template-haskell/dist-boot/build/./autogen -Ilibraries /template-haskell/. -optP-include -optPlibraries/template-haskell/dist- boot/build/./autogen/cabal_macros.h -package-id base-4.10.1.0 -package-id ghc-boot-th-8.4.4 -package-id pretty-1.1.3.3 -Wall -this-unit-id template- haskell -XHaskell2010 -no-user-package-db -rtsopts -fno-warn- deprecated-flags -odir libraries/template-haskell/dist-boot/build -hidir libraries/template-haskell/dist-boot/build -stubdir libraries /template-haskell/dist-boot/build -c libraries/template- haskell/./Language/Haskell/TH/Syntax.hs -o libraries/template-haskell /dist-boot/build/Language/Haskell/TH/Syntax.o -v Glasgow Haskell Compiler, Version 8.2.2, stage 2 booted by GHC version 8.0.2 Using binary package database: /opt/ghc/8.2.2/lib/ghc-8.2.2/package.conf.d/package.cache Using binary package database: libraries/bootstrapping.conf/package.cache package flags [-package-id base-4.10.1.0{unit base-4.10.1.0 True ([])}, -package-id ghc-boot-th-8.4.4{unit ghc-boot-th-8.4.4 True ([])}, -package-id pretty-1.1.3.3{unit pretty-1.1.3.3 True ([])}] loading package database /opt/ghc/8.2.2/lib/ghc-8.2.2/package.conf.d loading package database libraries/bootstrapping.conf package binary-0.8.5.1 overrides a previously defined package package hpc-0.6.0.3 overrides a previously defined package package Cabal-2.0.1.0 is unusable due to shadowed dependencies: binary-0.8.5.1 package ghc-8.2.2 is unusable due to shadowed dependencies: binary-0.8.5.1 hpc-0.6.0.3 ghc-boot-8.2.2 ghci-8.2.2 package ghc-boot-8.2.2 is unusable due to shadowed dependencies: binary-0.8.5.1 package ghci-8.2.2 is unusable due to shadowed dependencies: binary-0.8.5.1 ghc-boot-8.2.2 wired-in package ghc-prim mapped to ghc-prim-0.5.1.1 wired-in package integer-gmp mapped to integer-gmp-1.0.1.0 wired-in package base mapped to base-4.10.1.0 wired-in package rts mapped to rts wired-in package template-haskell mapped to template-haskell-2.13.0.0 wired-in package ghc mapped to ghc-8.4.4 wired-in package dph-seq not found. wired-in package dph-par not found. *** Checking old interface for Language.Haskell.TH.Syntax (use -ddump-hi- diffs for more details): *** Parser [Language.Haskell.TH.Syntax]: !!! Parser [Language.Haskell.TH.Syntax]: finished in 25.95 milliseconds, allocated 36.318 megabytes *** Renamer/typechecker [Language.Haskell.TH.Syntax]: !!! Renamer/typechecker [Language.Haskell.TH.Syntax]: finished in 775.89 milliseconds, allocated 669.498 megabytes libraries/template-haskell/Language/Haskell/TH/Syntax.hs:2002:43: error: • Failed to load interface for ‘Language.Haskell.TH.Lib’ Locations searched: libraries/template-haskell/./Language/Haskell/TH/Lib.hi libraries/template-haskell/./Language/Haskell/TH/Lib.hi-boot libraries/template-haskell/dist- boot/build/Language/Haskell/TH/Lib.hi libraries/template-haskell/dist-boot/build/Language/Haskell/TH/Lib .hi-boot libraries/template-haskell/dist- boot/build/./autogen/Language/Haskell/TH/Lib.hi libraries/template-haskell/dist- boot/build/./autogen/Language/Haskell/TH/Lib.hi-boot • In the expression: Language.Haskell.TH.Lib.conE (mkNameG_d "template-haskell" "Language.Haskell.TH.Syntax" "NominalR") In an equation for ‘lift’: lift NominalR = Language.Haskell.TH.Lib.conE (mkNameG_d "template-haskell" "Language.Haskell.TH.Syntax" "NominalR") When typechecking the code for ‘lift’ in a derived instance for ‘Lift Role’: To see the code I am typechecking, use -ddump-deriv In the instance declaration for ‘Lift Role’ | 2002 | deriving( Show, Eq, Ord, Data, Generic, Lift ) | ^^^^ *** Deleting temp files: Deleting: *** Deleting temp dirs: Deleting: }}} It appears as though GHC is searching in through the interface files of the stage-1 GHC (8.4.4), but using the bootstrapping compiler's (8.2.2's) `DeriveLift` name information to determine where to search! This adds more evidence to my belief that this is quite fragile. I'm not sure how to do better, however. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14030#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#14030: Implement the "Derive Lift instances for data types in template-haskell" proposal -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: RyanGlScott Type: task | Status: new Priority: normal | Milestone: 8.8.1 Component: Template Haskell | Version: 8.3 Resolution: | Keywords: deriving Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by harpocrates): Thanks for figuring out what the issue was in 8.2.2! Isn't this actually great news? Once https://phabricator.haskell.org/D5220 makes it through, the code that `Lift` will generate won't involve `conE`, `mkNameG_d`, etc. - it'll just be generating syntax for a bracket! I'm gonna try this out just to make sure I'm not missing anything. Another sanity check will be to see where we still use `THNames`. What do you think? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14030#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#14030: Implement the "Derive Lift instances for data types in template-haskell" proposal -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: RyanGlScott Type: task | Status: new Priority: normal | Milestone: 8.8.1 Component: Template Haskell | Version: 8.3 Resolution: | Keywords: deriving Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by RyanGlScott): Replying to [comment:8 harpocrates]:
the code that `Lift` will generate won't involve `conE`, `mkNameG_d`, etc. - it'll just be generating syntax for a bracket!
I'm not so sure. Nominally, yes, the generated code will only use the bracket syntax. But one must ask: what does the bracket syntax desugar down to? The answer may surprise you: {{{ λ> :set -ddump-simpl λ> data MyBool = F | T λ> deriving instance Lift MyBool ==================== Tidy Core ==================== Result size of Tidy Core = {terms: 38, types: 12, coercions: 3, joins: 0/0} -- RHS size: {terms: 21, types: 2, coercions: 0, joins: 0/0} $clift_r4ul :: MyBool -> Q Exp [GblId, Arity=1, Unf=OtherCon []] $clift_r4ul = \ (ds_d4um :: MyBool) -> case ds_d4um of { F -> Language.Haskell.TH.Lib.Internal.conE (mkNameG_d (GHC.CString.unpackCString# "interactive"#) (GHC.CString.unpackCString# "Ghci4"#) (GHC.CString.unpackCString# "F"#)); T -> Language.Haskell.TH.Lib.Internal.conE (mkNameG_d (GHC.CString.unpackCString# "interactive"#) (GHC.CString.unpackCString# "Ghci4"#) (GHC.CString.unpackCString# "T"#)) } -- RHS size: {terms: 1, types: 0, coercions: 3, joins: 0/0} interactive:Ghci5.$fLiftMyBool [InlPrag=INLINE (sat-args=0)] :: Lift MyBool [GblId[DFunId(nt)], Arity=1, Unf=OtherCon []] interactive:Ghci5.$fLiftMyBool = $clift_r4ul `cast` (Sym (Language.Haskell.TH.Syntax.N:Lift[0] <MyBool>_N) :: (MyBool -> Q Exp) ~R# Lift MyBool) }}} It turns out that bracket syntax desugars down to code that, in fact, uses `conE` and `mkNameG_d`! It's a bit more indirect, but I worry that there's the same potential for fragility. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14030#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#14030: Implement the "Derive Lift instances for data types in template-haskell" proposal -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: RyanGlScott Type: task | Status: new Priority: normal | Milestone: 8.10.1 Component: Template Haskell | Version: 8.3 Resolution: | Keywords: deriving Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by int-index): * cc: int-index (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14030#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#14030: Implement the "Derive Lift instances for data types in template-haskell" proposal -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: RyanGlScott Type: task | Status: new Priority: normal | Milestone: 8.10.1 Component: Template Haskell | Version: 8.3 Resolution: | Keywords: deriving Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by RyanGlScott): To confirm my hunch that even generating code with bracket syntax would still be fragile, I tried compiling GHC 8.4.4 with 8.2.2 as the bootstrapping compiler again, but instead of attempting to derive a `Lift` instance for `Role`, I defined one manually using bracket syntax: {{{#!hs instance Lift Role where lift NominalR = [| NominalR |] ... }}} Alas, that also suffers from the same issues observed in comment:7: {{{ libraries/template-haskell/Language/Haskell/TH/Syntax.hs:2005:28: error: • Failed to load interface for ‘Language.Haskell.TH.Lib’ Use -v to see a list of the files searched for. • In the expression: [| NominalR |] In an equation for ‘lift’: lift NominalR = [| NominalR |] In the instance declaration for ‘Lift Role’ | 2005 | lift NominalR = [| NominalR |] | ^^^^^^^^^^^^^^ }}} This all seems horribly delicate. There has to be a way to ensure that subsequent changes to `template-haskell` won't break the build in similar ways, but I can't think of how to orchestrate this. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14030#comment:12 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC