lundi 5 octobre 2020

Memory design pattern for a large number of ephemeral small structs

I have a system that requires a large number of ephemeral signals to be allocated, processed, and disposed of after they have aged out. Each signal is a struct that is 12 bytes long - int clock + float size. I have chosen struct rather than class to avoid the additional overhead of pointers to default constructor and destructor. The number of items I am looking to handle is of the order of 100,000,000 - that's 10^8. This will consume 1.2 * 10^9 (1.2 GB) so very containable on the heap but impossible on the stack. These signals are generated for a future point in time that will cause the signal objects to age rather rapidly - within 10,000 - 100,000 clock cycles. Other objects in the system will have a queue of these signals that they process in ascending clock order and then dispose of them. With this many ephemeral objects I need a low overhead memory management model.

First idea was to use an STL vector (priority queue in clock-order) to store the signals, but STL makes copies of the objects pushed into the container so that would double the storage requirements plus mess up reference pointers to the original signal objects. Besides, it's untenable to perform several hundred million allocations and deallocations on the heap.

Next idea was to use pointers to the signals to avoid object copying, but I'm still stuck with the overhead of allocation/deallocation for the millions of signals.

I like the idea of the priority-queue of pointers to the signals, as it carries low overhead and a copy of a pointer made by STL still points to object in question. What remains to be solved is how to allocate millions of signals, in rapid succession, enqueue the pointers to the signals, and dispose of them efficiently.

So far, the best design pattern I have come up with is to allocate a large array of slots - say 1,000,000 for holding unused signals, perform my own management of initializing the signals with the desired values and passing the pointer to the array slot on to the queuing and processing mechanism.

For this design pattern I would need to create a mechanism that "overloads" the C++ new function by allocating the next available signal slot, and "overloads" the C++ delete function to free up the slot in the array. When the array fills up I would need to allocate another array of unused signals to keep going. Once a signal delete indicates an array is now empty, I could then dispose of the moribund array.

The unique challenge for this memory design pattern is the constant creation, consumption, then disposing of millions of small signal objects.

Any pointers to fast, low-overhead memory design patters greatly appreciated.

Aucun commentaire:

Enregistrer un commentaire