Packing Haskell values into a C array

I'm working on a game in Haskell and I'm at a point where I have to do some optimization so I'm looking for advice or resources from people who may have already solved my problem. Specifically, I have a grid of 10,000 circles and I need to get them rendered. This involves translating the 10,000 circles into a single C array (of CFloats in my case) so that I can ship that array off to OpenGL for rendering. This is something I can't change, that's just the way OpenGL works. What I can change is the way I create and maintain the array. Currently I'm doing the most naive thing I can, which is to convert the list of circles into a list of floats, and then convert that list into a C array every frame. This is obviously very expensive. I've thought of several things to try including: * Only allocate an array when the number of circles changes. * Convert a circles into a C struct and "poke" them directly into an existing array instead of going through the intermediate form of a list. * Do some manual memory management and index each circle into a preallocated array, only updating the values of the array that correspond to circles which have changed. I'm wondering if there are already common solutions to this sort of problem (packing a lot of Haskell values that might change each frame into an array efficiently). Any resources that are tangential to this sort of thing would also be nice. For instance, how I can use the type system to ensure that the resulting array has the shape I want.

Hi Michael, On Sat, Apr 26, 2014 at 01:24:00PM -0500, Michael Baker wrote:
Specifically, I have a grid of 10,000 circles and I need to get them rendered. This involves translating the 10,000 circles into a single C array (of CFloats in my case) so that I can ship that array off to OpenGL for rendering. This is something I can't change, that's just the way OpenGL works.
Perhaps you don't need to change the Haskell representation that much, but only memorize which cricles have change and only update these ones. I would use a VertexBuffer-Object on the OpenGL side and memorize on the Haskell side which circles have changed per frame. Then you could map the VertexBuffer-Object and only update the corresponding coordinates. It might make sense to use a Vector instead of a List, then you could just memorize the indices of the changed Vector entries and the update should be faster when indexing into a Vector instead of a List. Greetings, Daniel

Using a Storable vector is what I mostly do. It goes nicely with rendering arrays of objects using OpenGL `BufferObject`s, using `fromVector` from http://hackage.haskell.org/package/GLUtil-0.6.4/docs/Graphics-GLUtil-BufferO.... On 26/04/14 19:24, Michael Baker wrote:
I've thought of several things to try including: * Only allocate an array when the number of circles changes. * Convert a circles into a C struct and "poke" them directly into an existing array instead of going through the intermediate form of a list. * Do some manual memory management and index each circle into a preallocated array, only updating the values of the array that correspond to circles which have changed.

On Apr 26, 2014, at 3:53 PM, Niklas Hambüchen
wrote: Using a Storable vector is what I mostly do.
It goes nicely with rendering arrays of objects using OpenGL `BufferObject`s, using `fromVector` from http://hackage.haskell.org/package/GLUtil-0.6.4/docs/Graphics-GLUtil-BufferO....
I second this recommendation. If you are doing frequent updates on a small subset of these values, I have most of an interface for doing so, but I haven't bundled it into GLUtil yet, so let me know (better: open an issue on github) if the existing interface just doesn't work for you. Anthony
On 26/04/14 19:24, Michael Baker wrote: I've thought of several things to try including: * Only allocate an array when the number of circles changes. * Convert a circles into a C struct and "poke" them directly into an existing array instead of going through the intermediate form of a list. * Do some manual memory management and index each circle into a preallocated array, only updating the values of the array that correspond to circles which have changed.
Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

On 26/04/14 19:24, Michael Baker wrote: I've thought of several things to try including: * Only allocate an array when the number of circles changes. * Convert a circles into a C struct and "poke" them directly into an existing array instead of going through the intermediate form of a list. * Do some manual memory management and index each circle into a preallocated array, only updating the values of the array that correspond to circles which have changed.
On Apr 26, 2014, at 3:53 PM, Niklas Hambüchen
wrote: Using a Storable vector is what I mostly do.
It goes nicely with rendering arrays of objects using OpenGL `BufferObject`s, using `fromVector` from
http://hackage.haskell.org/package/GLUtil-0.6.4/docs/Graphics-GLUtil-BufferO... .
On Sat, Apr 26, 2014 at 3:13 PM, Anthony Cowley
wrote: I second this recommendation. If you are doing frequent updates on a small subset of these values, I have most of an interface for doing so, but I haven't bundled it into GLUtil yet, so let me know (better: open an issue on github) if the existing interface just doesn't work for you.
So do you all just create a new vector every frame? Could you go into a little more detail?

On Apr 26, 2014, at 10:28 PM, Michael Baker
wrote: On 26/04/14 19:24, Michael Baker wrote: I've thought of several things to try including: * Only allocate an array when the number of circles changes. * Convert a circles into a C struct and "poke" them directly into an existing array instead of going through the intermediate form of a list. * Do some manual memory management and index each circle into a preallocated array, only updating the values of the array that correspond to circles which have changed.
On Apr 26, 2014, at 3:53 PM, Niklas Hambüchen
wrote: Using a Storable vector is what I mostly do.
It goes nicely with rendering arrays of objects using OpenGL `BufferObject`s, using `fromVector` from http://hackage.haskell.org/package/GLUtil-0.6.4/docs/Graphics-GLUtil-BufferO....
On Sat, Apr 26, 2014 at 3:13 PM, Anthony Cowley
wrote: I second this recommendation. If you are doing frequent updates on a small subset of these values, I have most of an interface for doing so, but I haven't bundled it into GLUtil yet, so let me know (better: open an issue on github) if the existing interface just doesn't work for you. So do you all just create a new vector every frame? Could you go into a little more detail?
You can create a new vector every frame, or you can use a mutable vector that you freeze before passing to 'fromVector' or 'replaceVector'. You can also investigate using, say, 10 vectors of 1k elements each if your updates are spatially correlated somehow. I do the latter when I am generating new data over time, for example. The nice thing about using Vector is that you can use it like an array that you peek from and poke to, or you can take advantage of the fusion machinery to generate or update it on the Haskell side before shipping the underlying data off to OpenGL. The specifics of your update patterns will govern what is the best strategy. Anthony

Yes, what Anthony says. I create the BufferObject from the Vector when what I want to render changes (which happens less frequently than every frame), and each frame then renders from the BufferObject. If you want to change single elements, like he says, the mutable vector should make you happy. On 27/04/14 04:54, Anthony Cowley wrote:
So do you all just create a new vector every frame? Could you go into a little more detail?
You can create a new vector every frame, or you can use a mutable vector that you freeze before passing to 'fromVector' or 'replaceVector'. You can also investigate using, say, 10 vectors of 1k elements each if your updates are spatially correlated somehow. I do the latter when I am generating new data over time, for example. The nice thing about using Vector is that you can use it like an array that you peek from and poke to, or you can take advantage of the fusion machinery to generate or update it on the Haskell side before shipping the underlying data off to OpenGL.
The specifics of your update patterns will govern what is the best strategy.
Anthony

Thanks!
On Sun, Apr 27, 2014 at 8:15 AM, Niklas Hambüchen
Yes, what Anthony says. I create the BufferObject from the Vector when what I want to render changes (which happens less frequently than every frame), and each frame then renders from the BufferObject.
If you want to change single elements, like he says, the mutable vector should make you happy.
On 27/04/14 04:54, Anthony Cowley wrote:
So do you all just create a new vector every frame? Could you go into a little more detail?
You can create a new vector every frame, or you can use a mutable vector that you freeze before passing to 'fromVector' or 'replaceVector'. You can also investigate using, say, 10 vectors of 1k elements each if your updates are spatially correlated somehow. I do the latter when I am generating new data over time, for example. The nice thing about using Vector is that you can use it like an array that you peek from and poke to, or you can take advantage of the fusion machinery to generate or update it on the Haskell side before shipping the underlying data off to OpenGL.
The specifics of your update patterns will govern what is the best strategy.
Anthony
participants (4)
-
Anthony Cowley
-
Daniel Trstenjak
-
Michael Baker
-
Niklas Hambüchen