GitLab

Alan Zimmerman pushed to branch wip/az/ghc-cpp at Glasgow Haskell Compiler / GHC

Commits:

0fb37893

by Matthew Pickering at 2025-06-23T13:55:10-04:00

Move ModuleGraph into UnitEnv

The ModuleGraph is a piece of information associated with the
ExternalPackageState and HomeUnitGraph. Therefore we should store it
inside the HomeUnitEnv.

3bf6720e

by soulomoon at 2025-06-23T13:55:52-04:00

Remove hptAllFamInstances usage during upsweep

Fixes #26118

This change eliminates the use of hptAllFamInstances during the upsweep phase,
as it could access non-below modules from the home package table.

The following updates were made:

* Updated checkFamInstConsistency to accept an explicit ModuleEnv FamInstEnv
  parameter and removed the call to hptAllFamInstances.
* Adjusted hugInstancesBelow so we can construct ModuleEnv FamInstEnv
  from its result,
* hptAllFamInstances and allFamInstances functions are removed.

83ee7b78

by Ben Gamari at 2025-06-24T05:02:07-04:00

configure: Don't force value of OTOOL, etc. if not present

Previously if `otool` and `install_name_tool` were not present they
would be overridden by `fp_settings.m4`. This logic was introduced in
4ff93292243888545da452ea4d4c1987f2343591 without explanation.

9329c9e1

by Ben Gamari at 2025-06-24T05:02:07-04:00

ghc-toolchain: Add support for otool, install_name_tool

Fixes part of ghc#23675.

25f5c998

by Ben Gamari at 2025-06-24T05:02:08-04:00

ghc-toolchain: Add support for llc, opt, llvm-as

Fixes #23675.

51d150dd

by Rodrigo Mesquita at 2025-06-24T05:02:08-04:00

hadrian: Use settings-use-distro-mingw directly

The type `ToolchainSetting` only made sense when we had more settings to
fetch from the system config file. Even then "settings-use-distro-mingw"
is arguably not a toolchain setting.

With the fix for #23675, all toolchain tools were moved to the
`ghc-toolchain` `Toolchain` format. Therefore, we can inline
`settings-use-distro-mingw` accesses and delete `ToolchainSetting`.

dcf68a83

by Rodrigo Mesquita at 2025-06-24T05:02:08-04:00

configure: Check LlvmTarget exists for LlvmAsFlags

If LlvmTarget was empty, LlvmAsFlags would be just "--target=".
If it is empty now, simply keep LlvmAsFlags empty.

ghc-toolchain already does this right. This fix makes the two
configurations match up.

580a3353

by Ben Gamari at 2025-06-24T05:02:51-04:00

rts/linker/LoadArchive: Use bool

Improve type precision by using `bool` instead of `int` and `StgBool`.

76d1041d

by Ben Gamari at 2025-06-24T05:02:51-04:00

rts/linker/LoadArchive: Don't rely on file extensions for identification

Previously archive members would be identified via their file extension,
as described in #13103. We now instead use a more principled approach,
relying on the magic number in the member's header.

As well, we refactor treatment of archive format detection to improve
code clarity and error handling.

Closes #13103.

4b748a99

by Teo Camarasu at 2025-06-24T15:31:07-04:00

template-haskell: improve changelog


stable -> more stable, just to clarify that this interface isn't fully stable.

errornously -> mistakenly: I typod this and also let's go for a simpler word

e358e477
by Sylvain Henry at 2025-06-24T15:31:58-04:00
```
Bump stack resolver to use GHC 9.6.7

Cf #26139
```

4bf5eb63

by fendor at 2025-06-25T17:05:43-04:00

Teach `:reload` about multiple home units

`:reload` needs to lookup the `ModuleName` and must not assume the given
`ModuleName` is in the current `HomeUnit`.

We add a new utility function which allows us to find a `HomeUnitModule`
instead of a `Module`.

Further, we introduce the `GhciCommandError` type which can be used to
abort the execution of a GHCi command.
This error is caught and printed in a human readable fashion.

b3d97bb3

by fendor at 2025-06-25T17:06:25-04:00

Implement `-fno-load-initial-targets` flag

We add the new flag `-fno-load-initial-targets` which doesn't load all `Target`s
immediately but only computes the module graph for all `Target`s.
The user can then decide to load modules from that module graph using
the syntax:

    ghci> :reload <Mod>

This will load everything in the module graph up to `Mod`.

The user can return to the initial state by using the builtin target
`none` to unload all modules.

    ghci> :reload none

Is in principle identical to starting a new session with the
`-fno-load-initial-targets` flag.

The `-fno-load-initial-targets` flag allows for faster startup time of GHCi when a
user has lots of `Target`s.

We additionally extend the `:reload` command to accept multiple
`ModuleName`s. For example:

    ghci> :reload <Mod1> <Mod2>

Loads all modules up to the modules `Mod1` and `Mod2`.

49f44e52

by Teo Camarasu at 2025-06-26T04:19:51-04:00

Expose ghc-internal unit id through the settings file

This in combination with the unit id of the compiler library allows
cabal to know of the two unit ids that should not be reinstalled (in
specific circumstances) as:
- when using plugins, we want to link against exactly the compiler unit
id
- when using TemplateHaskell we want to link against exactly the package
that contains the TemplateHaskell interfaces, which is `ghc-internal`

See: <https://github.com/haskell/cabal/issues/10087>

Resolves #25282

499c4efe

by Bryan Richter at 2025-06-26T04:20:33-04:00

CI: Fix and clean up capture of timings

* Fixes the typo that caused 'cat ci-timings' to report "no such file or
  directory"
* Gave ci_timings.txt a file extension so it may play better with other
  systems
* Fixed the use of time_it so all times are recorded
* Fixed time_it to print name along with timing

86c90c9e

by Bryan Richter at 2025-06-26T04:20:33-04:00

CI: Update collapsible section usage

The syntax apparently changed at some point.

04308ee4
by Bryan Richter at 2025-06-26T04:20:33-04:00
```
CI: Add more collapsible sections
```

6ac906a1

by Alan Zimmerman at 2025-06-26T18:37:53+01:00

GHC-CPP: first rough proof of concept

Processes

     #define FOO
     #ifdef FOO
     x = 1
     #endif

Into

    [ITcppIgnored [L loc ITcppDefine]
    ,ITcppIgnored [L loc ITcppIfdef]
    ,ITvarid "x"
    ,ITequal
    ,ITinteger (IL {il_text = SourceText "1", il_neg = False, il_value = 1})
    ,ITcppIgnored [L loc ITcppEndif]
    ,ITeof]

In time, ITcppIgnored will be pushed into a comment

fe321296

by Alan Zimmerman at 2025-06-26T18:37:53+01:00

Tidy up before re-visiting the continuation mechanic

e02bec42

by Alan Zimmerman at 2025-06-26T18:37:53+01:00

Switch preprocessor to continuation passing style

Proof of concept, needs tidying up

44112223
by Alan Zimmerman at 2025-06-26T18:37:53+01:00
```
Small cleanup
```
b94f4af1
by Alan Zimmerman at 2025-06-26T18:37:53+01:00
```
Get rid of some cruft
```

175d73a2

by Alan Zimmerman at 2025-06-26T18:37:53+01:00

Starting to integrate.

Need to get the pragma recognised and set

2bf717e7

by Alan Zimmerman at 2025-06-26T18:37:53+01:00

Make cppTokens extend to end of line, and process CPP comments

1b23c156
by Alan Zimmerman at 2025-06-26T18:37:53+01:00
```
Remove unused ITcppDefined
```

b6d668e4

by Alan Zimmerman at 2025-06-26T18:37:53+01:00

Allow spaces between # and keyword for preprocessor directive

2b4dbf6f

by Alan Zimmerman at 2025-06-26T18:37:53+01:00

Process CPP continuation lines

They are emited as separate ITcppContinue tokens.
Perhaps the processing should be more like a comment, and keep on
going to the end.
BUT, the last line needs to be slurped as a whole.

c03d85cb

by Alan Zimmerman at 2025-06-26T18:37:53+01:00

Accumulate CPP continuations, process when ready

Can be simplified further, we only need one CPP token

be8490b3

by Alan Zimmerman at 2025-06-26T18:37:53+01:00

Simplify Lexer interface. Only ITcpp

We transfer directive lines through it, then parse them from scratch
in the preprocessor.

4effc38a

by Alan Zimmerman at 2025-06-26T18:37:53+01:00

Deal with directive on last line, with no trailing \n

1ca0e44c
by Alan Zimmerman at 2025-06-26T18:37:53+01:00
```
Start parsing and processing the directives
```
f0bc133e
by Alan Zimmerman at 2025-06-26T18:37:54+01:00
```
Prepare for processing include files
```

36cd4957

by Alan Zimmerman at 2025-06-26T18:37:54+01:00

Move PpState into PreProcess

And initParserState, initPragState too

ebbceda2

by Alan Zimmerman at 2025-06-26T18:37:54+01:00

Process nested include files

Also move PpState out of Lexer.x, so it is easy to evolve it in a ghci
session, loading utils/check-cpp/Main.hs

4b141ac7
by Alan Zimmerman at 2025-06-26T18:37:54+01:00
```
Split into separate files
```

08df4a1c

by Alan Zimmerman at 2025-06-26T18:37:54+01:00

Starting on expression parser.

But it hangs. Time for Text.Parsec.Expr

70bd2d95

by Alan Zimmerman at 2025-06-26T18:37:54+01:00

Start integrating the ghc-cpp work

From https://github.com/alanz/ghc-cpp

fd93d551
by Alan Zimmerman at 2025-06-26T18:37:54+01:00
```
WIP
```
84b0c99c
by Alan Zimmerman at 2025-06-26T18:37:54+01:00
```
Fixup after rebase
```
6ce804b2
by Alan Zimmerman at 2025-06-26T18:38:57+01:00
```
WIP
```
c7102f6c
by Alan Zimmerman at 2025-06-26T18:39:01+01:00
```
Fixup after rebase, including all tests pass
```
d68b4ebc
by Alan Zimmerman at 2025-06-26T18:39:01+01:00
```
Change pragma usage to GHC_CPP from GhcCPP
```
d5a3082b
by Alan Zimmerman at 2025-06-26T18:39:01+01:00
```
Some comments
```
ce3fa6b2
by Alan Zimmerman at 2025-06-26T18:39:01+01:00
```
Reformat
```
4a3e884b
by Alan Zimmerman at 2025-06-26T18:39:01+01:00
```
Delete unused file
```
e72ee597
by Alan Zimmerman at 2025-06-26T18:39:01+01:00
```
Rename module Parse to ParsePP
```
d73ef7ce
by Alan Zimmerman at 2025-06-26T18:39:01+01:00
```
Clarify naming in the parser
```

32a10684

by Alan Zimmerman at 2025-06-26T18:39:01+01:00

WIP. Switching to alex/happy to be able to work in-tree

Since Parsec is not available

21d7ac3f

by Alan Zimmerman at 2025-06-26T18:39:01+01:00

Layering is now correct

- GHC lexer, emits CPP tokens
- accumulated in Preprocessor state
- Lexed by CPP lexer, CPP command extracted, tokens concated with
  spaces (to get rid of token pasting via comments)
- if directive lexed and parsed by CPP lexer/parser, and evaluated

4bc73df1

by Alan Zimmerman at 2025-06-26T18:39:01+01:00

First example working

Loading Example1.hs into ghci, getting the right results

```
{-# LANGUAGE GHC_CPP #-}
module Example1 where

y = 3

x =
  "hello"
  "bye now"

foo = putStrLn x
```

f7526d16

by Alan Zimmerman at 2025-06-26T18:39:01+01:00

Rebase, and all tests pass except whitespace for generated parser

e94212c9
by Alan Zimmerman at 2025-06-26T18:39:01+01:00
```
More plumbing. Ready for testing tomorrow.
```

08215abd

by Alan Zimmerman at 2025-06-26T18:39:01+01:00

Proress. Renamed module State from Types

And at first blush it seems to handle preprocessor scopes properly.

5afc16ba

by Alan Zimmerman at 2025-06-26T18:39:01+01:00

Insert basic GHC version macros into parser

__GLASGOW_HASKELL__
__GLASGOW_HASKELL_FULL_VERSION__
__GLASGOW_HASKELL_PATCHLEVEL1__
__GLASGOW_HASKELL_PATCHLEVEL2__

6363a3f1
by Alan Zimmerman at 2025-06-26T18:39:01+01:00
```
Re-sync check-cpp for easy ghci work
```
ffaf09bf
by Alan Zimmerman at 2025-06-26T18:39:01+01:00
```
Get rid of warnings
```

616fcf5a

by Alan Zimmerman at 2025-06-26T18:39:01+01:00

Rework macro processing, in check-cpp

Macros kept at the top level, looked up via name, multiple arity
versions per name can be stored

3828a951

by Alan Zimmerman at 2025-06-26T18:39:01+01:00

WIP. Can crack arguments for #define

Next step it to crack out args in an expansion

8f8dbdb1
by Alan Zimmerman at 2025-06-26T18:39:01+01:00
```
WIP on arg parsing.
```
b4f72620
by Alan Zimmerman at 2025-06-26T18:39:02+01:00
```
Progress. Still screwing up nested parens.
```
84a649d9
by Alan Zimmerman at 2025-06-26T18:39:02+01:00
```
Seems to work, but has redundant code
```
facdd19c
by Alan Zimmerman at 2025-06-26T18:39:02+01:00
```
Remove redundant code
```
36c7705d
by Alan Zimmerman at 2025-06-26T18:39:02+01:00
```
Reformat
```

1f8dc622

by Alan Zimmerman at 2025-06-26T18:39:02+01:00

Expand args, single pass

Still need to repeat until fixpoint

7474547d
by Alan Zimmerman at 2025-06-26T18:39:02+01:00
```
Fixed point expansion
```
5c2944a9
by Alan Zimmerman at 2025-06-26T18:39:02+01:00
```
Sync the playground to compiler
```

58670a3f

by Alan Zimmerman at 2025-06-26T18:39:02+01:00

Working on dumping the GHC_CPP result

But We need to keep the BufSpan in a comment

2c2e5d44

by Alan Zimmerman at 2025-06-26T18:39:02+01:00

Keep BufSpan in queued comments in GHC.Parser.Lexer

a3455dd7

by Alan Zimmerman at 2025-06-26T18:39:02+01:00

Getting close to being able to print the combined tokens

showing what is in and what is out

6ad166ec

by Alan Zimmerman at 2025-06-26T18:39:02+01:00

First implementation of dumpGhcCpp.

Example output

First dumps all macros in the state, then the source, showing which
lines are in and which are out

------------------------------

- |#define FOO(A,B) A + B
- |#define FOO(A,B,C) A + B + C
- |#if FOO(1,FOO(3,4)) == 8
- |-- a comment
  |x = 1
- |#else
- |x = 5
- |#endif

907aa1d6
by Alan Zimmerman at 2025-06-26T18:39:02+01:00
```
Clean up a bit
```
c41f6787
by Alan Zimmerman at 2025-06-26T18:39:02+01:00
```
Add -ddump-ghc-cpp option and a test based on it
```

c9c55897

by Alan Zimmerman at 2025-06-26T18:39:02+01:00

Restore Lexer.x rules, we need them for continuation lines

0e5ee662

by Alan Zimmerman at 2025-06-26T18:39:02+01:00

Lexer.x: trying to sort out the span for continuations

- We need to match on \n at the end of the line
- We cannot simply back up for it

69f140f3

by Alan Zimmerman at 2025-06-26T18:39:02+01:00

Inserts predefined macros. But does not dump properly

Because the cpp tokens have a trailing newline

a7375551

by Alan Zimmerman at 2025-06-26T18:39:02+01:00

Remove unnecessary LExer rules

We *need* the ones that explicitly match to the end of the line.

4c478d03

by Alan Zimmerman at 2025-06-26T18:39:02+01:00

Generate correct span for ITcpp

Dump now works, except we do not render trailing `\` for continuation
lines. This is good enough for use in test output.

5eb2bd28
by Alan Zimmerman at 2025-06-26T18:39:02+01:00
```
Reduce duplication in lexer
```
ff8cf54b
by Alan Zimmerman at 2025-06-26T18:39:02+01:00
```
Tweaks
```

e110292d

by Alan Zimmerman at 2025-06-26T18:39:02+01:00

Insert min_version predefined macros into state

The mechanism now works. Still need to flesh out the full set.

f8010f34

by Alan Zimmerman at 2025-06-26T18:39:02+01:00

Trying my alternative pragma syntax.

It works, but dumpGhcCpp is broken, I suspect from the ITcpp token
span update.

bd3b86a3

by Alan Zimmerman at 2025-06-26T18:39:02+01:00

Pragma extraction now works, with both CPP and GHC_CPP

For the following

    {-# LANGUAGE CPP #-}
    #if __GLASGOW_HASKELL__ >= 913
    {-# LANGUAGE GHC_CPP #-}
    #endif

We will enable GHC_CPP only

877b0918
by Alan Zimmerman at 2025-06-26T18:39:02+01:00
```
Remove some tracing
```
8b6e6e7e
by Alan Zimmerman at 2025-06-26T18:39:02+01:00
```
Fix test exes for changes
```

e15dc78d

by Alan Zimmerman at 2025-06-26T18:39:02+01:00

For GHC_CPP tests, normalise config-time-based macros

c1c6d5fc
by Alan Zimmerman at 2025-06-26T18:39:02+01:00
```
WIP
```
63bd3393
by Alan Zimmerman at 2025-06-26T18:39:02+01:00
```
WIP again. What is wrong?
```
62d77eb8
by Alan Zimmerman at 2025-06-26T18:39:02+01:00
```
Revert to dynflags for normal not pragma lexing
```
368205f2
by Alan Zimmerman at 2025-06-26T18:39:02+01:00
```
Working on getting check-exact to work properly
```
599857bf
by Alan Zimmerman at 2025-06-26T18:39:02+01:00
```
Passes CppCommentPlacement test
```

94ea2764

by Alan Zimmerman at 2025-06-26T18:39:02+01:00

Starting on exact printing with GHC_CPP

While overriding normal CPP

973edb55

by Alan Zimmerman at 2025-06-26T18:39:02+01:00

Correctly store CPP ignored tokens as comments

By populating the lexeme string in it, based on the bufpos

bad712ac
by Alan Zimmerman at 2025-06-26T18:39:02+01:00
```
WIP
```
b5b7560d
by Alan Zimmerman at 2025-06-26T18:39:02+01:00
```
Simplifying
```
0b68bfb0
by Alan Zimmerman at 2025-06-26T18:39:02+01:00
```
Update the active state logic
```
ae6de9c3
by Alan Zimmerman at 2025-06-26T18:39:02+01:00
```
Work the new logic into the mainline code
```
ea22a018
by Alan Zimmerman at 2025-06-26T18:39:02+01:00
```
Process `defined` operator
```

ea3bfb18

by Alan Zimmerman at 2025-06-26T18:39:02+01:00

Manage lexer state while skipping tokens

There is very intricate layout-related state used when lexing. If a
CPP directive blanks out some tokens, store this state when the
blanking starts, and restore it when they are no longer being blanked.

f1a49aed

by Alan Zimmerman at 2025-06-26T18:39:02+01:00

Track the last token buffer index, for ITCppIgnored

We need to attach the source being skipped in an ITCppIgnored token.
We cannot simply use its BufSpan as an index into the underlying
StringBuffer as it counts unicode chars, not bytes.
So we update the lexer state to store the starting StringBuffer
location for the last token, and use the already-stored length to
extract the correct portion of the StringBuffer being parsed.

8acd3528
by Alan Zimmerman at 2025-06-26T18:39:02+01:00
```
Process the ! operator in GHC_CPP expressions
```
5ad037f4
by Alan Zimmerman at 2025-06-26T18:39:02+01:00
```
Predefine a constant when GHC_CPP is being used.
```
c997f1f8
by Alan Zimmerman at 2025-06-26T18:39:02+01:00
```
WIP
```
4f13bcfa
by Alan Zimmerman at 2025-06-26T18:39:02+01:00
```
Skip lines directly in the lexer when required
```

af131e84