Hi Niklas - sounds like a nice usecase. Though we need to make sure no usages are missing to make that decision (there are some edge cases yet, for example around TH splice application, or type families). Also it's only valid on the set of indexed packages. I'll merge later, but https://github.com/robinp/haskell-indexer/commit/0592c3d934515c4661c69c5b998... has the changes necessary. Note: - It didn't index ~50 packages - For some it complained that C libs are missing (like SDL, though I was using the full stack-build Docker image) - I had to kill a few huge compilations that ate my RAM (Agda, language-python, and some yi packages) I would estimate 8-12 hours, but didn't follow closely (also it was idling quite some due to the huge compilations). By the way the per-package entries are available for download in a tarball, so if you just index your local packages and dump the entries along with those of your dependencies into the graphstore, you can get a ~full local index without reexecuting the whole indexing. In the future it would be nice to have tooling that magically fetches the pre-generated entries you need, and merges them locally. 2017-08-31 13:23 GMT+02:00 Niklas Hambüchen :

Hey Robin,

This is really awesome and useful.

I think it will help a lot to decide whether or not we can change/deprecate a certain function, as now we can really easily judge how many callers it has.

How long does that take to generate? And how did you run it to do it for all of Stackage?

Thanks!

On 31/08/17 06:41, Robin Palotai wrote:

...

To find out who else uses your favorite functions and how: http://stuff.codereview.me/lts/9.2 .

I used fpco/stack-build:lts-9.2. Might explain, thank you! Is the GHC directory layout the same in snoyberg/stackage? Just asking since the derived container hijacks the GHC executable for the indexer. 2017-08-31 14:18 GMT+02:00 Michael Snoyman :

Which Docker image did you use for building? The (inappropriately named but hysterical raisins and all that) snoyberg/stackage Docker images are what we use for doing the actual Stackage builds.

On Thu, Aug 31, 2017, 2:37 PM Robin Palotai wrote:

...

Hi Niklas - sounds like a nice usecase.

Though we need to make sure no usages are missing to make that decision (there are some edge cases yet, for example around TH splice application, or type families). Also it's only valid on the set of indexed packages.

I'll merge later, but https://github.com/robinp/haskell-indexer/commit/ 0592c3d934515c4661c69c5b998e26a543b93b52 has the changes necessary. Note: - It didn't index ~50 packages - For some it complained that C libs are missing (like SDL, though I was using the full stack-build Docker image) - I had to kill a few huge compilations that ate my RAM (Agda, language-python, and some yi packages)

I would estimate 8-12 hours, but didn't follow closely (also it was idling quite some due to the huge compilations).

By the way the per-package entries are available for download in a tarball, so if you just index your local packages and dump the entries along with those of your dependencies into the graphstore, you can get a ~full local index without reexecuting the whole indexing. In the future it would be nice to have tooling that magically fetches the pre-generated entries you need, and merges them locally.

2017-08-31 13:23 GMT+02:00 Niklas Hambüchen :

...

Hey Robin,

This is really awesome and useful.

I think it will help a lot to decide whether or not we can change/deprecate a certain function, as now we can really easily judge how many callers it has.

How long does that take to generate? And how did you run it to do it for all of Stackage?

Thanks!

On 31/08/17 06:41, Robin Palotai wrote:

...

To find out who else uses your favorite functions and how: http://stuff.codereview.me/lts/9.2 .

_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

I mean the build/indexing scripts takes the GHC in the container, and replaces it with the indexer wrapper (that will call both the original GHC, and the indexer). This is to capture the compiler arguments. If the paths are the same, it's easy to adopt to the other container. 2017-08-31 16:06 GMT+02:00 Michael Snoyman :

It looks like it should be using the same method of installing GHC (Herbert's PPA). I'm not sure what you mean by hijacking the GHC executable though.

On Thu, Aug 31, 2017 at 3:27 PM, Robin Palotai wrote:

...

I used fpco/stack-build:lts-9.2. Might explain, thank you! Is the GHC directory layout the same in snoyberg/stackage? Just asking since the derived container hijacks the GHC executable for the indexer.

2017-08-31 14:18 GMT+02:00 Michael Snoyman :

...

Which Docker image did you use for building? The (inappropriately named but hysterical raisins and all that) snoyberg/stackage Docker images are what we use for doing the actual Stackage builds.

On Thu, Aug 31, 2017, 2:37 PM Robin Palotai wrote:

...

Hi Niklas - sounds like a nice usecase.

Though we need to make sure no usages are missing to make that decision (there are some edge cases yet, for example around TH splice application, or type families). Also it's only valid on the set of indexed packages.

I'll merge later, but https://github.com/robinp/hask ell-indexer/commit/0592c3d934515c4661c69c5b998e26a543b93b52 has the changes necessary. Note: - It didn't index ~50 packages - For some it complained that C libs are missing (like SDL, though I was using the full stack-build Docker image) - I had to kill a few huge compilations that ate my RAM (Agda, language-python, and some yi packages)

I would estimate 8-12 hours, but didn't follow closely (also it was idling quite some due to the huge compilations).

By the way the per-package entries are available for download in a tarball, so if you just index your local packages and dump the entries along with those of your dependencies into the graphstore, you can get a ~full local index without reexecuting the whole indexing. In the future it would be nice to have tooling that magically fetches the pre-generated entries you need, and merges them locally.

2017-08-31 13:23 GMT+02:00 Niklas Hambüchen :

...

Hey Robin,

This is really awesome and useful.

I think it will help a lot to decide whether or not we can change/deprecate a certain function, as now we can really easily judge how many callers it has.

How long does that take to generate? And how did you run it to do it for all of Stackage?

Thanks!

On 31/08/17 06:41, Robin Palotai wrote:

...

To find out who else uses your favorite functions and how: http://stuff.codereview.me/lts/9.2 .

_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

I consider usages on Hackage an OK estimator of usages outside of Stackage, certainly enough to get a first expression. Also, with haskell-indexer becoming really easy to use, the maintainers of off-Hackage code bases can themselves more easily determine whether they use functions in question and chime into the discussion with real data. On 31/08/17 19:31, Joachim Durchholz wrote:

It does not (and cannot) tell you about caller code that is not on Hackage. If you have a low-level library then that information is still better than nothing, but if your code is a public API this information is almost worse than nothing.