JTalton's picture

High-Level OpenGL Classes (TextureManager)

Are there any plans for high level OpenGL classes such as a texture manager? I would like to have several 3D models use the same texture and when the last one has gone away to free up the texture. While garbage collection does happen on objects, it would be advantageous to have the OpenGL texture memory freed without having to wait on garbage collection.

My thought is to have a handle class to the texture with the texture being reference counted. When the last handle is disposed (reference == 0), it would call dispose on the texture.

There are other OpenGL constructs that this could also apply to.


Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
the Fiddler.'s picture

I don't plan to implement high-level OpenGL classes currently, at least not before the low-level functionality is complete, but I wouldn't object to contributed code. This topic has come up before in the Tao forums and elsewhere, so there definitely is interest for such constructs.

The first idea was to wrap OpenGL resources in IDisposable classes so that you can either Dispose() them deterministically or leave them to the GC if you wanted. Problem is, this doesn't really work as the finalizer runs in a different thread (and is non-deterministic in its behavior, i.e. it could destroy the OpenGL context before the resources). The idea of manually implementing reference counting sounds interesting in the face of this problem - how do you imagine that system would work? Would the user have to call IncRef/DecRef functions manually, or is there another way of handling this?

One possible implementation of a Texture class:

using OpenTK.OpenGL;
using OpenTK.OpenGL.Enums;
// IGLTexture to provide a common interface migration to OpenGL 3 easier.
public sealed class Texture : IGLTexture, IDisposable
{
    private int id;
    private int width, height;
    private TextureFormat format;
    private bool disposed;
 
    public Texture() { }
    public Texture(int width, int height, TextureFormat format)
    {
         GL.GenTextures(1, out id);
         GL.BindTexture(TextureTarget.Texture2d, id);
         GL.TexImage2D(TextureTarget.Texture2d, 0,
                                  format.ConvertToGL2PixelInternalFormat(),
                                  width, height, 0,
                                  PixelFormat.Bgra, PixelType.UnsignedByte, IntPtr.Zero);
     }
     // Other contructors that load from files, memory streams etc...
 
     // Other useful public methods from IGLTexture
 
     # region IDisposable Members
 
     private Dispose(bool manual)
     {
         if (!disposed) 
         {
             if (manual)    // Otherwise we don't know if a context exists anymore.
                 GL.DeleteTextures(1, ref id);
             disposed = true;
         }
     }
 
     public Dispose() 
     {
         this.Dispose(true);
         GC.SuppressFinalize(this);
     }
 
     // This is highly problematic
     ~Texture() { this.Dispose(false); }
 
     #endregion
}
JTalton's picture

In C++ I wrote some nice SmartPointers that acted just like a pointer but behind the scenes it called AddRef and RemoveRef on the object it was pointing to. It overloaded many operators such as the operator*.

In C# I don't think you can overload some of the operators needed, and it doesn't have the concept of pointers.

public interface IReferenceCounted
{
    public int AddRef();
    public int RemoveRef();
}
public class ReferenceHandle<ObjectT> where ObjectT : IReferenceCounted
{
    ObjectT object;
    public ObjectT Object
    {
        get{ return object; }
        set
        {
            if( object != null )
            {
                if( object.RemoveRef() == 0 )
                {
                    object.Dispose();
                }
            }
 
            object = value;
 
            if( object != null )
            {
                object.AddRef();
            }
        }
    }
}

Thats just a rough idea. It needs some work and clarification but would work.

the Fiddler.'s picture

I spent the whole afternoon trying to think up a solution to this problem. Did some interesting reading on WeakReferences, SafeHandles, reference counting and garbage collection too (disclaimer: I find this topic fascinating, so excuse me if I get overzealous at times!)

As you said, C# doesn't provide the facilities to make smart pointers tolerable (never liked them :) ). On the other hand and unlike C++, it has an efficient GC at its disposal, which by and large works much better than any ref-counting scheme. My gut feeling is that it would be best to rely on the GC whenever we can and avoid ref-counting in OpenTK itself, *unless* there is a way to make it completely transparent to the user.

As far as I can see, there are two problems we need to solve:
a) Deterministic cleanup of OpenGL resources. This can be solved efficiently by wrapping OpenGL resources in IDisposable classes, while the user can add the necessary cleanup logic in his application-specific code (ref-counting/caching mechanisms, resource loading/unloading etc).

b) Cleanup of orphaned OpenGL resources, i.e. resources that are no longer in use but were never cleaned up. This is a much more difficult problem to solve, mainly because of the non-deterministic nature of the GC.

One solution to (b) is implement reference counting. We know this will work, but maybe there is a better solution? (better as in more transparent and efficient)

I only have a rough idea, but it might be workable: When the GC identifies an orphaned & finalizable object, it puts it into the "freachable" queue, and finalizes it during its next run. Since we can't safely destroy OpenGL resources in the finalizer thread (no OpenGL context), we have to do something different or leak resources. So what happens if we resurrect the object from the "freachable" queue so that we can destroy it in our leisure?

I think this might just work. Resurrection is possible: just create a reference to the object during the finalizer call. There are some problems however:
a) how do we identify which object belongs to which context? Do we maintain object queues for each OpenGL context? What about shared objects?
b) how do we keep the relevant context alive until all objects belonging to it are dead?
c) what happens during a violent AppDomain teardown? AFAIK, non-critical finalizers are skipped then.
d) We have to avoid any and all exceptions in the finalizers (or see (c) above).
e) The GL class must stay alive till the bitter end. I think that the fact that it is static plays a role here, but I'm not sure what that role is.

This has the advantage that it can be transparent to the user, and quite efficient, but it could as well be that this isn't possible or robust enough for production use. We can either fall back to manual reference counting then, or even avoid the issue entirely by declaring the problem as outside the scope of OpenTK ;)

JTalton's picture

The use of the garbage collector is an interesting idea. I would like to see someone try that! I would worry though about the .NET versus the mono implementations.

Ease of use it a must.
I would personally like to see a Texture class that is really a handle to a ReferenceCountedTexture. Make it all transparent to the user.

Texture texture1 = new Texture("myTexture.jpg"); // RefCount of 1
Texture texture2 = new Texture(texture1); // RefCount of 2
texture1 = null; // RefCount of 1
texture2 = null; // RefCount of 0 = Dispose();

The only thing that would make the above nicer is a TextureManager that you could add and lookup textures.

While I understand that we want to keep the OpenTK interface light, it is designed for C# and should make using OpenGL with C# easier. C# garbage collects memory when needed, but does not free resources (graphics card texture memory) when needed.

Inertia's picture

Interesting topic, I like the idea to use the GC for approaching this. The problem I see with that is the GC's unpredictability when it will clean up memory, I doubt you want to manually enforce garbage collection at a regular interval to your app just to possibly clean up a possibly orphaned Texture.
Another thing that should be mentioned is that unless you manually set the Texture's priority, OpenGL will move least recently used textures off video memory. This works to your advantage in your case. Even if you use a simpler approach like my DDSManager does, and only delete the whole texture pool at once , the textures not used by any material will not impact performance since they get moved to system memory when it gets crowded (as in not occupy video memory).

The way I would integrate it into the DDSManager class would be to keep a reference count in the TextureDetails struct and add a method to remove a single Texture by Filename or Texture object Handle. The reference count would be increased by LoadTextureManaged(), while decreasing - and in case of 0 references left deleting the texture - would be handled by some RemoveTexture() method.

Don't quote me on this, but a while ago I read that the OpenGL driver's behaviour on a context loss is invalidating all allocated buffers (Frame-, Texture- and Vertex-Buffers aswell), so If a Process using OpenGL is terminated randomly, the video memory occupied by that terminated process is invalidated and effectively freed.

..avoid the issue entirely by declaring the problem as outside the scope of OpenTK ;)
Actually I'd rather not have this functionality in OpenTK :P While it's an interesting problem to solve, It's not really a commonly required functionality to delete textures in the "mainloop", more common is to move all texture loading/deleting into the initialization/load phase and changelevel/exit. At least my preference is to let OpenGL handle texture residency and only clean up textures when it's known that the memory is needed for other things.

the Fiddler.'s picture

I can imagine two scenarios were such functionality would prove useful:
a) A very common mistake by newcomers to OpenTK/Tao is cleaning up OpenGL resources in finalizers. Instant crash and it's not obvious what the problem is (the finalizer running on a different thread) unless you are experienced with both OpenGL and .Net.

b) Possible resource leaks stemming from the assumption that the Garbage Collector will clean up all allocated resources (again, the difference between the managed and unmanaged resource pools is not obvious unless you have read up on .Net interop). The object is cleaned, but allocated textures are not, consuming memory until the application is terminated (the driver will move them out of video memory, but still they are leaked resources).

While both can be overcome, it would be nice to avoid such issues in the first place.

In reality, an implementation based on the GC would be orthogonal to resouce management logic. It wouldn't alleviate the need to manage e.g. textures manually (like your DDSManager, probably using reference counts or some other method and preferrably out of the main loop) , but rather it would safeguard against leaking resources by fitting better in the GC-ed .Net paradigm.

All this assuming it's possible to hack the GC like this in a cross-platform way, which is something I'm going to try soon.

Inertia's picture

As long as it remains optional to use the feature I won't complain ;) It sure will become handy once there's a reliable implementation, for example one could also cleanup VBO after this principle. Another advantage of your approach is, that when changing levels in a game the previously loaded textures could be re-used, the unused ones will be GC at some later point. In my approach you have to clear all and reload all textures from disk.
No idea how to solve the a)-e) Problems mentioned above atm though.

the Fiddler.'s picture

A user of the Tao Framework implemented this idea with promising results. He wrote wrappers for OpenGL resources and implemented the disposable pattern like this:

private void Dispose(bool manual)
{
    if (!disposed)
    {
        if (manual)
        {
             Gl.glDeleteTextures(1, ref _tid);
             GC.SuppressFinalize(this);
             disposed = true;
        }
        else
        {
            GC.KeepAlive(SimpleOpenGlControl.DisposingQueue);
            SimpleOpenGlControl.DisposingQueue.Enqueue(this);
        }
    }
}

SimpleOpenGlControl.DisposingQueue is a queue that holds references to OpenGL resources. It is visited regularly during program execution, disposing data contained therein. Note that the 'else' clause will never be executed, unless you actually forget to release resources. This is the best of both worlds: you can release resources manually (with no performance hit), but the garbage collector will still clean up after you if you forget something. Even better, the implementation is dead simple!

Now, we only need to find out how to handle multiple OpenGL contexts.

Anonymous's picture

Bene from TAO Forum here:

There is mistake in the snipped:
disposed = true;

should go into the if (manual) {...} Block

Else the whole stuff would never be executed anymore (first if)

the Fiddler.'s picture

Thanks, corrected!