The .Net Framework features a precise, generational and compacting Garbage Collector (GC): precise because it specifically traces only managed object pointers, generational because it distinguishes long-lived objects objects from temporary ones, and compacting because it moves data in memory to avoid leaving holes behind. The GC is a great tool in the .Net arsenal, not only because it increases productivity but also because it provides extremely fast memory allocations (compared to standard C/C++ malloc/new).
One of the challenges of working with garbage collection is garbage collection pauses. If these last longer than 5-7ms, it can impact interactive updates. While collector pauses are highly dependent on the the implementation, typically they are managed-pointer-count proportional pauses which occur because of the tracing or marking of the tenured heap. Note that even in the CLR 4.0 "concurrent" collector, there is still a pause for tenured-generation mark.
There are two main strategies for dealing with garbage collection pauses. First, minimize the duration of the pause, second, minimize the frequency of the pause.
To minimize the duration of the pause, minimize the total number of long-lived reachable managed pointers. It's important to note this is not the same as keeping the heap small. Large value-type arrays are not scanned by the GC if they contain no pointers. Likewise, raw byte data buffers are also not scanned by the GC. Placing large data-chunks, such as vertex-buffers, index-buffers, textures, and other raw-data into value-type arrays which don't contain pointers can substantially minimize the number of GC traced pointers.
To minimize the frequency of the pause, minimize tenured churn. Tenured churn is when objects are frequently allocated, survive long enough to make it into the tenured generation, and then are released. Reclaiming the space from those objects requires the GC to trace the entire tenured generation, causing the pause. This can be avoided by avoiding heap allocated objects which live too long and then die, and in general by avoiding allocation when possible. Ideally all objects either die very fast, or live a very long time. Aside from minimizing allocation through stack-allocated types, churn can be minimized by using reusable object pools for medium-lifetime objects.
The managed heap is not the only memory usage in the process. Buffers handed to OpenGL may be copied into the unmanaged resource pool. This creates memory usage outside of the managed heap.
Another important memory consideration is the OpenGL robustness configuration. Historically, OpenGL preserves all state and buffers in system-ram, even when those assets are sent to the video card, so it can restore those assets in the event that the application loses and regains control of the 3d hardware graphics context. For some applications, this can mean assets like textures and vertex buffers are stored three times, once in the application ram, once in OpenGL ram, and once in video-card ram.
As of OpenGL 3.2, the GL_ARB_robustness extension can be used to control OpenGL robustness, allowing OpenGL to forgo storing system-ram copies of resources. If resources are lost, the application will need to restore them. In OpenGL ES, only non-robust operation is allowed, mirroring the behavior of Direct3d.
[Describe the unmanaged resource pool, pinning and performance considerations]