The things which can't be compacted are * Pinned objects * Functions * Mutable variables It is only a hypothesis at the moment that compacting a ModIface will help GC times in an IDE, but in order to try it we have to implement this roadmap.. It is certainly true that the EPS can get very large for realistic projects with hundreds of dependencies, and not traversing it during GC could be a huge win. Cheers, Matt On Tue, Mar 24, 2020 at 9:27 PM Simon Peyton Jones wrote:

Thanks for writing this down Matthew.

But I look at #17097 and I am baffled. Why is that the right list of tasks? Why do we need FastStrings backed by an unpinned ByteArray? (And similarly for each other bullet.) What will the API look like if this project is successful? Why do we want ModIfaces in a compact region? To reduce residency? Do we have data showing that this is a real issue in practice.

I feel as if a wiki page to explain the problem and articulate the proposed solution would make it easier for outsiders to contribute.

Thanks

Simon

| -----Original Message----- | From: ghc-devs On Behalf Of Matthew | Pickering | Sent: 24 March 2020 10:58 | To: GHC developers | Subject: Roadmap to compacting ModIface | | Hello all, | | I have written down the remaining steps which need to be taken in | order to compact a ModIface, which we hope will be useful for | applications such as IDEs to reduce GC time. | | https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.h | askell.org%2Fghc%2Fghc%2Fissues%2F17097%23roadmap-to-compacting-a- | modiface&data=02%7C01%7Csimonpj%40microsoft.com%7C5614bf7bb16847bf5bab | 08d7cfe23972%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372064428432732 | 44&sdata=YWpD1VEj%2FF4JVrRRi9cdNzlZ%2BQqgfFeRZ40NXC1kI2o%3D&reserv | ed=0 | | If there is anyone who wishes to help with this project then please | ping me on IRC. So far this is joint work between myself and Daniel G. | | The first step we need to take is to get 1675 merged which replaces | the type backing a FastString from a ByteString to a ShortByteString | (and hence from a pinned ByteArray to an unpinned ByteArray). | | https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.h | askell.org%2Fghc%2Fghc%2F- | %2Fmerge_requests%2F1675&data=02%7C01%7Csimonpj%40microsoft.com%7C5614 | bf7bb16847bf5bab08d7cfe23972%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C | 637206442843273244&sdata=N7fLEXXhlSESuye9BiCFQo76UmQ4%2B6GSbegaQcef0Lc | %3D&reserved=0 | | Cheers, | | Matt | _______________________________________________ | ghc-devs mailing list | ghc-devs@haskell.org | https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.hask | ell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc- | devs&data=02%7C01%7Csimonpj%40microsoft.com%7C5614bf7bb16847bf5bab08d7 | cfe23972%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637206442843283238&a | mp;sdata=a1gBw6q0tuSxuPaByJgR9Jq0Ksk5%2BsP0kzMhaxeVgzs%3D&reserved=0

Omer, In particular the EPS contains the PackageIfaceTable which contains a map of modules to ModIface for external packages. This can end up containing a lot of ModIfaces in a big project. In every `HomeModInfo` there is also a `ModIface` which can be compacted in order to save GC traversal. ModIface is also chosen because it's possible serialise/deserialise and hence more likely to be easily compactable, ModDetails will contain too much stuff which can't be compacted, just looking now the `TypeEnv` could certainly not be compated. Cheers, Matt On Wed, Mar 25, 2020 at 10:06 AM Ömer Sinan Ağacan wrote:

How is ModIface used by IDEs exactly? I'd expect IDEs to use ModDetails, not ModIface.

They're basically two different representations of the same thing (a module interface), but ModIface is more focused on serialization and deseriazliation (the type is designed to make that easy) and ModDetails is what GHC is using to e.g. type check an imported module.

For example, In batch mode we add ModDetails to the module graph, not ModIface, because that's what we use to compile downstream.

(In one-shot mode GHC makes ModDetails for imported modules after reading the interfaces using IfaceToCore.typecheckIface)

Ömer

Matthew Pickering , 25 Mar 2020 Çar, 00:31 tarihinde şunu yazdı:

...

The things which can't be compacted are

* Pinned objects * Functions * Mutable variables

It is only a hypothesis at the moment that compacting a ModIface will help GC times in an IDE, but in order to try it we have to implement this roadmap..

It is certainly true that the EPS can get very large for realistic projects with hundreds of dependencies, and not traversing it during GC could be a huge win.

Cheers,

Matt

On Tue, Mar 24, 2020 at 9:27 PM Simon Peyton Jones wrote:

...

Thanks for writing this down Matthew.

But I look at #17097 and I am baffled. Why is that the right list of tasks? Why do we need FastStrings backed by an unpinned ByteArray? (And similarly for each other bullet.) What will the API look like if this project is successful? Why do we want ModIfaces in a compact region? To reduce residency? Do we have data showing that this is a real issue in practice.

I feel as if a wiki page to explain the problem and articulate the proposed solution would make it easier for outsiders to contribute.

Thanks

Simon

| -----Original Message----- | From: ghc-devs On Behalf Of Matthew | Pickering | Sent: 24 March 2020 10:58 | To: GHC developers | Subject: Roadmap to compacting ModIface | | Hello all, | | I have written down the remaining steps which need to be taken in | order to compact a ModIface, which we hope will be useful for | applications such as IDEs to reduce GC time. | | https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.h | askell.org%2Fghc%2Fghc%2Fissues%2F17097%23roadmap-to-compacting-a- | modiface&data=02%7C01%7Csimonpj%40microsoft.com%7C5614bf7bb16847bf5bab | 08d7cfe23972%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372064428432732 | 44&sdata=YWpD1VEj%2FF4JVrRRi9cdNzlZ%2BQqgfFeRZ40NXC1kI2o%3D&reserv | ed=0 | | If there is anyone who wishes to help with this project then please | ping me on IRC. So far this is joint work between myself and Daniel G. | | The first step we need to take is to get 1675 merged which replaces | the type backing a FastString from a ByteString to a ShortByteString | (and hence from a pinned ByteArray to an unpinned ByteArray). | | https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.h | askell.org%2Fghc%2Fghc%2F- | %2Fmerge_requests%2F1675&data=02%7C01%7Csimonpj%40microsoft.com%7C5614 | bf7bb16847bf5bab08d7cfe23972%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C | 637206442843273244&sdata=N7fLEXXhlSESuye9BiCFQo76UmQ4%2B6GSbegaQcef0Lc | %3D&reserved=0 | | Cheers, | | Matt | _______________________________________________ | ghc-devs mailing list | ghc-devs@haskell.org | https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.hask | ell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc- | devs&data=02%7C01%7Csimonpj%40microsoft.com%7C5614bf7bb16847bf5bab08d7 | cfe23972%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637206442843283238&a | mp;sdata=a1gBw6q0tuSxuPaByJgR9Jq0Ksk5%2BsP0kzMhaxeVgzs%3D&reserved=0

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs