Re: Stable name allocation

On Wed, 26 Sep 2018 at 10:54, David Feuer wrote:

Simon seems a bit busy right now. Can anyone else advise me on the basics of heap allocation in primops?

On Tue, Sep 25, 2018 at 1:42 PM, David Feuer wrote:

...

Let's forget about allocate(). I can definitely handle this part in C--. But I'm still lost in the macros and such. For example, I'm very unclear on the differences among the ALLOC, HP_CHK, and MAYBE_GC classes of macro. I can't find anything in the commentary, and the source code documentation is very sparse. I'm okay with either of the following approaches, but either way I need a bit more info.

The best way to understand these macros is to look at their implementations and see how they're used in other parts of the RTS. MAYBE_GC doesn't bump Hp, it is used to ensure that we don't indefinitely call allocate() in primops that use it. The ALLOC family are wrappers around the lower level HP_CHK functions, these bump Hp. Some of the variations are optimisations to generate less verbose code - again, see the other primops for examples. It's always safe to use HP_CHK_GEN_TICKY(), but there might be better alternatives depending on the type of your primitive. Cheers Simon

...

1. First see if we need to allocate a StableName#. If so, check whether GC would be required to allocate the StableName# (how?). If so, drop the lock, run GC (how?) and start over. This looks cleanest to me if it can be done easily.

2. First run the GC if we're low on memory (how?). Then if we need to allocate a StableName#, we'll be sure to have room.

...

You can do it unconditionally before taking the lock, or you can do it conditionally as long as you release the lock if the heap check fails. I think in the latter case there might not be a macro that allows this, but you could use the `allocate()` method for allocating memory (like newByteArray#) and then you could write a heap check like the MAYBE_GC() macro. Doing it unconditionally is easier and probably not a big

...

hit, but note that you'll have to retreat Hp if you don't use the memory.

Cheers Simon

On Sat, 22 Sep 2018 at 13:08, David Feuer wrote:

...

How do I check if GC will be required, and how do I trigger it? Should

I

...

perform the check unconditionally at the beginning of the operation so I don't have to drop the lock, GC, then retake? I don't know the right ways to deal with this stuff, and the macros are mostly undocumented.

On Sep 22, 2018 3:53 AM, "Simon Marlow" wrote:

Yes, the current implementation looks like it creates the object after adding the entry to the StableName table and releasing the lock, which is unsafe because another thread could read that same entry before the object has been created. The easiest solution to that is to take and release

...

...

lock in C-- in the right places instead of in the C lookupStableName() function (you might need to make a separate version of lookupStableName() that doesn't take the lock).

Cheers Simon

On Fri, 21 Sep 2018 at 12:53, David Feuer wrote:

...

It seems awkward to do it in C--, but maybe you can help me work out

how.

...

The allocation facilities definitely seem much nicer there, and allocating a small heap object in C feels like an abuse of the facilities we have

...

...

...

The essential challenge, as I see it, is that we need the key to

On Tue, Sep 25, 2018 at 6:25 AM, Simon Marlow wrote: performance the there. point to a

...

...

...

valid stable name object by the time we drop the hash table lock. The process, as I imagine it:

1. Follow indirections, untag, choose the right generation. (All this is in C) 2. Take the appropriate hash table lock. (C) 3. Look up the key in the hash table (C).

Now there's a branch. If we found the key, then we don't need to allocate an SNO. We just drop the lock and return. Otherwise

4. Allocate an SNO and set its info pointer (most easily done in C--). If this necessitates GC, we need to drop the lock first and might as well just go back to the very beginning afterwards. 5. Populate the SNO with its "hash value" (we can do this anywhere we like, I imagine). 6. Insert the key and SNO into the hash table and drop the hash table lock (C) 7. If necessary, insert the SNO into the remembered set (C)

How would you recommend structuring this back-and-forth?

On Fri, Sep 21, 2018, 3:19 AM Simon Marlow wrote:

...

I'm a bit sceptical that you need to allocate a heap object in C

instead

...

of C--, but still, here's an example:

https://phabricator.haskell.org/diffusion/GHC/browse/master/rts%2FThreads.c$...

...

It's slightly less efficient to do this in C than C--, because `allocate()` is slower than allocating by bumping `Hp`.

On Mon, 17 Sep 2018 at 21:25, David Feuer

wrote:

...

> > How can I allocate a heap object in C code in the rts? I've only seen > heap objects allocated in C--, and doing that here would be lousy for > performance and worse for clarity. > > David