OpenTK and WPF interoperation

Posted Wednesday, 9 September, 2009 - 15:28 by the Fiddler

Project:	The Open Toolkit library
Version:	0.9.9-2b
Component:	Code
Category:	support request
Priority:	normal
Assigned:	Unassigned
Status:	in progress (review)

Attachment	Size
OpenTKWPF.zip	1.81 MB

Comments

Yes, modern systems SwapBuffers is pretty fast on modern hardware (microseconds) and the ReadPixels call overshadows it completely.

Auxiliary buffers are a relic of the past, there's absolutely no reason to use them nowadays.

Ok, so I will remove their use...

But what will be the best solution... I have also try to use FBO but I can't !

When I call :

Gl.glGenRenderbuffersEXT(1, out DepthRenderbuffer);

It crash !!

Also, how do I specify to OpenGL to draw on the FBO

Here is my code to create the FBO :

private void CreateFrameBuffer()
{
            // Create Color Texture
            Gl.glEnable(Gl.GL_TEXTURE_2D);
            Gl.glGenTextures(1, out ColorTexture);
            Gl.glBindTexture(Gl.GL_TEXTURE_2D, ColorTexture);
            Gl.glTexParameteri(Gl.GL_TEXTURE_2D, Gl.GL_TEXTURE_MIN_FILTER, Gl.GL_NEAREST);
            Gl.glTexParameteri(Gl.GL_TEXTURE_2D, Gl.GL_TEXTURE_MAG_FILTER, Gl.GL_NEAREST);
            Gl.glTexParameteri(Gl.GL_TEXTURE_2D, Gl.GL_TEXTURE_WRAP_S, Gl.GL_CLAMP);
            Gl.glTexParameteri(Gl.GL_TEXTURE_2D, Gl.GL_TEXTURE_WRAP_T, Gl.GL_CLAMP);
            Gl.glTexImage2D(Gl.GL_TEXTURE_2D, 0, Gl.GL_RGBA8, FboWidth, FboHeight, 0, Gl.GL_RGBA, Gl.GL_UNSIGNED_BYTE, IntPtr.Zero);
 
            // test for GL Error here (might be unsupported format)
 
            Gl.glBindTexture(Gl.GL_TEXTURE_2D, 0); // prevent feedback, reading and writing to the same image is a bad idea
 
            // Create Depth Renderbuffer
            Gl.glGenRenderbuffersEXT(1, out DepthRenderbuffer);
            Gl.glBindRenderbufferEXT(Gl.GL_RENDERBUFFER_EXT, DepthRenderbuffer);
            Gl.glRenderbufferStorageEXT(Gl.GL_RENDERBUFFER_EXT, Gl.GL_DEPTH_COMPONENT32, FboWidth, FboHeight);
 
            // test for GL Error here (might be unsupported format)
 
            // Create a FBO and attach the textures
            Gl.glGenFramebuffersEXT(14, out FboHandle);
            Gl.glBindFramebufferEXT(Gl.GL_FRAMEBUFFER_EXT, FboHandle);
            Gl.glFramebufferTexture2DEXT(Gl.GL_FRAMEBUFFER_EXT, Gl.GL_COLOR_ATTACHMENT0_EXT, Gl.GL_TEXTURE_2D, ColorTexture, 0);
            Gl.glFramebufferRenderbufferEXT(Gl.GL_FRAMEBUFFER_EXT, Gl.GL_DEPTH_ATTACHMENT_EXT, Gl.GL_RENDERBUFFER_EXT, DepthRenderbuffer);
 
            // now GL.Ext.CheckFramebufferStatus( FramebufferTarget.FramebufferExt ) can be called, check the end of this page for a snippet.
 
            // since there's only 1 Color buffer attached this is not explicitly required
            Gl.glDrawBuffer(Gl.GL_COLOR_ATTACHMENT0_EXT);
 
            Gl.glPushAttrib(Gl.GL_VIEWPORT_BIT); // stores GL.Viewport() parameters
            Gl.glViewport(0, 0, FboWidth, FboHeight);
 
            // render whatever your heart desires, when done ...
}

Can you help me to debug it ?

Thanks

Do you get a crash or a NullReferenceException? The former indicates a driver bug, while the latter means that your video card does not support FBOs.

Have you checked for FBO support? Search the extension string for "EXT_framebuffer_object": Gl.glGetString(Gl.GL_EXTENSIONS).Contains("EXT_framebuffer_object"). Many Intel cards don't support this (try updating your drivers).

Your're right, I have update my drivers and now all is fine...

Except that when I try to create the BitmapSource... the application is blocked !!!

Here is the code of the class, can you take a closer look... it sounds that the FBO is locked or something like this !

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Data;
using System.Windows.Documents;
using System.Windows.Input;
using System.Windows.Media;
using System.Windows.Media.Imaging;
using System.Windows.Navigation;
using System.Windows.Shapes;
using System.Windows.Interop;
using System.Runtime.InteropServices;
using System.Diagnostics;
 
using Tao.OpenGl;
using Tao.Platform.Windows;
 
namespace RS.UI
{
 
    public partial class OGLPanel : UserControl
    {
 
        #region Variables
 
        public event EventHandler OnOGLRender;
 
        private bool _isInitialized = false;
 
        //---- Global rendering lock
        static object RENDER_LOCK = new object();
 
        //----
        private IntPtr deviceContext = IntPtr.Zero;                         // GDI device context
        private IntPtr renderingContext = IntPtr.Zero;                      // Rendering context
        private IntPtr windowHandle = IntPtr.Zero;                          // Holds our window handle
        private bool autoCheckErrors = false;                               // Should we provide glGetError()?
        private bool autoFinish = false;                                    // Should we provide a glFinish()?
        private bool autoMakeCurrent = true;                                // Should we automatically make the rendering context current?
        private bool autoSwapBuffers = true;                                // Should we automatically swap buffers?
        private byte accumBits = 0;                                         // Accumulation buffer bits
        private byte colorBits = 32;                                        // Color buffer bits
        private byte depthBits = 16;                                        // Depth buffer bits
        private byte stencilBits = 0;                                       // Stencil buffer bits
        private int errorCode = Gl.GL_NO_ERROR;                             // The GL error code
 
        private int logScaleX = 96;                                         // DPI Resolution in X dir
        private int logScaleY = 96;                                         // DPI Resolution in Y dir
 
        private InvisibleForm _form;
 
        private System.Drawing.Bitmap _bitmap;
        private System.Drawing.Imaging.BitmapData _bitmapData;
 
        #endregion
 
        #region Constructor
 
        public OGLPanel()
        {
            InitializeComponent();
            this.SizeChanged += new SizeChangedEventHandler(OGLPanel_SizeChanged);
        }
 
        #endregion
 
        #region OGLPanel_SizeChanged
 
        void OGLPanel_SizeChanged(object sender, SizeChangedEventArgs e)
        {
            if (!_isInitialized)
                return;
 
            _form.ClientSize = new System.Drawing.Size((int)ActualWidth, (int)ActualHeight);
            _bitmap = new System.Drawing.Bitmap(_form.ClientSize.Width, _form.ClientSize.Height, System.Drawing.Imaging.PixelFormat.Format32bppArgb);
 
            Refresh();
        }
 
        #endregion
 
        #region Initialize
 
        public void Initialize()
        {
            if (_isInitialized)
                return;
 
            InitializeContexts();
 
            _form.ClientSize = new System.Drawing.Size((int)ActualWidth, (int)ActualHeight);
            _bitmap = new System.Drawing.Bitmap(_form.ClientSize.Width, _form.ClientSize.Height, System.Drawing.Imaging.PixelFormat.Format32bppArgb);
 
            _isInitialized = true;
        }
 
        #endregion
 
        #region InitializeContexts
 
        /// <summary>  
        /// Creates the OpenGL contexts.
        /// </summary>
        public void InitializeContexts()
        {
            int pixelFormat;                                                // Holds the selected pixel format
 
            //--- Create the window handle
            //HwndSource hwndSource = PresentationSource.FromVisual(this) as HwndSource;
            //HwndTarget hwndTarget = hwndSource.CompositionTarget;
 
            //// this is the new WPF API to force render mode. 
            //hwndTarget.RenderMode = RenderMode.SoftwareOnly;
 
            //windowHandle = hwndSource.Handle;                                     // Get window handle
 
            _form = new InvisibleForm();
            _form.Show();
 
            windowHandle = _form.Handle;
 
            if (windowHandle == IntPtr.Zero)
                Environment.Exit(-1);
 
            //---- Create the device context
            deviceContext = User.GetDC(windowHandle);                       // Attempt to get the device context
            if (deviceContext == IntPtr.Zero)
                Environment.Exit(-1);
 
            //---- Define the Gdi pixel format
            Gdi.PIXELFORMATDESCRIPTOR pfd = new Gdi.PIXELFORMATDESCRIPTOR();// The pixel format descriptor
            pfd.nSize = (short)Marshal.SizeOf(pfd);                        // Size of the pixel format descriptor
            pfd.nVersion = 1;                                               // Version number (always 1)
            pfd.dwFlags = Gdi.PFD_DRAW_TO_WINDOW |                          // Format must support windowed mode
                        Gdi.PFD_SUPPORT_OPENGL |                            // Format must support OpenGL
                        Gdi.PFD_DOUBLEBUFFER;                               // Must support double buffering
            pfd.iPixelType = (byte)Gdi.PFD_TYPE_RGBA;                      // Request an RGBA format
            pfd.cColorBits = (byte)colorBits;                              // Select our color depth
            pfd.cRedBits = 0;                                               // Individual color bits ignored
            pfd.cRedShift = 0;
            pfd.cGreenBits = 0;
            pfd.cGreenShift = 0;
            pfd.cBlueBits = 0;
            pfd.cBlueShift = 0;
            pfd.cAlphaBits = 0;                                             // No alpha buffer
            pfd.cAlphaShift = 0;                                            // Alpha shift bit ignored
            pfd.cAccumBits = accumBits;                                     // Accumulation buffer
            pfd.cAccumRedBits = 0;                                          // Individual accumulation bits ignored
            pfd.cAccumGreenBits = 0;
            pfd.cAccumBlueBits = 0;
            pfd.cAccumAlphaBits = 0;
            pfd.cDepthBits = depthBits;                                     // Z-buffer (depth buffer)
            pfd.cStencilBits = stencilBits;                                 // No stencil buffer
            pfd.cAuxBuffers = 0;                                            // No auxiliary buffer
            pfd.iLayerType = (byte)Gdi.PFD_MAIN_PLANE;                     // Main drawing layer
            pfd.bReserved = 0;                                              // Reserved
            pfd.dwLayerMask = 0;                                            // Layer masks ignored
            pfd.dwVisibleMask = 0;
            pfd.dwDamageMask = 0;
            pixelFormat = Gdi.ChoosePixelFormat(deviceContext, ref pfd);    // Attempt to find an appropriate pixel format
            if (pixelFormat == 0)
                Environment.Exit(-1);
 
            // Are we not able to set the pixel format?
            if (!Gdi.SetPixelFormat(deviceContext, pixelFormat, ref pfd))
                Environment.Exit(-1);
 
            //----
            logScaleX = Gdi.GetDeviceCaps(deviceContext, (int)Gdi.DevCaps.LOGPIXELSX); // Attempt to retrieve DPI-Setting
            logScaleY = Gdi.GetDeviceCaps(deviceContext, (int)Gdi.DevCaps.LOGPIXELSY); // Attempt to retrieve DPI-Setting
 
            //---- Create a rendering context
            renderingContext = Wgl.wglCreateContext(deviceContext);         // Attempt to get the rendering context
            if (renderingContext == IntPtr.Zero)
                Environment.Exit(-1);
 
            _isInitialized = true;
 
            MakeCurrent();                                                  // Attempt to activate the rendering context
 
            // Force A Reset On The Working Set Size
            Kernel.SetProcessWorkingSetSize(Process.GetCurrentProcess().Handle, -1, -1);
        }
 
        #endregion
 
        #region CreateFrameBuffer
 
        int FboWidth = 512;
        int FboHeight = 512;
 
        uint FboHandle;
        uint ColorTexture;
        uint DepthRenderbuffer;
 
        private void CreateFrameBuffer()
        {
            // Does it support FBO ?
            if (!Gl.glGetString(Gl.GL_EXTENSIONS).Contains("EXT_framebuffer_object"))
                return;
 
            FboWidth = _form.ClientSize.Width;
            FboHeight = _form.ClientSize.Height;
 
            // Create Color Texture
            Gl.glEnable(Gl.GL_TEXTURE_2D);
            Gl.glGenTextures(1, out ColorTexture);
            Gl.glBindTexture(Gl.GL_TEXTURE_2D, ColorTexture);
            Gl.glTexParameteri(Gl.GL_TEXTURE_2D, Gl.GL_TEXTURE_MIN_FILTER, Gl.GL_NEAREST);
            Gl.glTexParameteri(Gl.GL_TEXTURE_2D, Gl.GL_TEXTURE_MAG_FILTER, Gl.GL_NEAREST);
            Gl.glTexParameteri(Gl.GL_TEXTURE_2D, Gl.GL_TEXTURE_WRAP_S, Gl.GL_CLAMP);
            Gl.glTexParameteri(Gl.GL_TEXTURE_2D, Gl.GL_TEXTURE_WRAP_T, Gl.GL_CLAMP);
            Gl.glTexImage2D(Gl.GL_TEXTURE_2D, 0, Gl.GL_RGBA8, FboWidth, FboHeight, 0, Gl.GL_RGBA, Gl.GL_UNSIGNED_BYTE, IntPtr.Zero);
 
            // test for GL Error here (might be unsupported format)
 
            Gl.glBindTexture(Gl.GL_TEXTURE_2D, 0); // prevent feedback, reading and writing to the same image is a bad idea
 
            // Create Depth Renderbuffer
            Gl.glGenRenderbuffersEXT(1, out DepthRenderbuffer);
            Gl.glBindRenderbufferEXT(Gl.GL_RENDERBUFFER_EXT, DepthRenderbuffer);
            Gl.glRenderbufferStorageEXT(Gl.GL_RENDERBUFFER_EXT, Gl.GL_DEPTH_COMPONENT32, FboWidth, FboHeight);
 
            // test for GL Error here (might be unsupported format)
 
            // Create a FBO and attach the textures
            Gl.glGenFramebuffersEXT(14, out FboHandle);
            Gl.glBindFramebufferEXT(Gl.GL_FRAMEBUFFER_EXT, FboHandle);
            Gl.glFramebufferTexture2DEXT(Gl.GL_FRAMEBUFFER_EXT, Gl.GL_COLOR_ATTACHMENT0_EXT, Gl.GL_TEXTURE_2D, ColorTexture, 0);
            Gl.glFramebufferRenderbufferEXT(Gl.GL_FRAMEBUFFER_EXT, Gl.GL_DEPTH_ATTACHMENT_EXT, Gl.GL_RENDERBUFFER_EXT, DepthRenderbuffer);
 
            // now GL.Ext.CheckFramebufferStatus( FramebufferTarget.FramebufferExt ) can be called, check the end of this page for a snippet.
 
            // since there's only 1 Color buffer attached this is not explicitly required
            Gl.glDrawBuffer(Gl.GL_COLOR_ATTACHMENT0_EXT);
 
            Gl.glPushAttrib(Gl.GL_VIEWPORT_BIT); // stores GL.Viewport() parameters
            Gl.glViewport(0, 0, FboWidth, FboHeight);
 
            // render whatever your heart desires, when done ...
        }
 
        private void ReleaseFB()
        {
            // Does it support FBO ?
            if (!Gl.glGetString(Gl.GL_EXTENSIONS).Contains("EXT_framebuffer_object"))
                return;
 
            Gl.glPopAttrib(); // restores GL.Viewport() parameters
            Gl.glBindFramebufferEXT(Gl.GL_FRAMEBUFFER_EXT, 0); // return to visible framebuffer
            Gl.glDrawBuffer(Gl.GL_BACK);
        }
 
        #endregion
 
        #region MakeCurrent
 
        public void MakeCurrent()
        {
            if (!_isInitialized)
                return;
 
            // Are we not able to activate the rending context?
            if (!Wgl.wglMakeCurrent(deviceContext, renderingContext))
            {
                //MessageBox.Show("Can not activate the GL rendering context.", "Fatal Error", MessageBoxButtons.OK, MessageBoxIcon.Error);
                Environment.Exit(-1);
            }
        }
 
        #endregion
 
        #region SwapBuffers
 
        public void SwapBuffers()
        {
            Gdi.SwapBuffersFast(deviceContext);
            //lock(LOCK_BUFFER)
            //{
            //    unsafe
            //    {
            //        GL.ReadPixels(0,
            //            0,
            //            bufferWidth,
            //            bufferHeight,
            //            OpenTK.Graphics.OpenGL.PixelFormat.Rgb,
            //            PixelType.Bitmap,
            //            buffer);
 
            //        System.Windows.Media.PixelFormat format = PixelFormats.Rgb24;
 
            //        InteropBitmap source = System.Windows.Interop.Imaging.CreateBitmapSourceFromMemorySection(
            //            buffer,
            //            bufferWidth,
            //            bufferHeight,
            //            format,
            //            (int)(bufferWidth * format.BitsPerPixel / 8), 0) as InteropBitmap;
 
            //        source.Invalidate();
            //        this.Source = (BitmapSource)source.GetAsFrozen();
            //    }
            //}
        }
 
        #endregion
 
        #region OnRender
 
        protected override void OnRender(DrawingContext drawingContext)
        {
            base.OnRender(drawingContext);
 
            if (!_isInitialized)
                return;
 
            lock (RENDER_LOCK)
            {
                MakeCurrent();
 
                CreateFrameBuffer();
 
                if (OnOGLRender != null)
                    OnOGLRender(null, null);
 
                //SwapBuffers();
 
                Gl.glFlush();
                Gl.glFinish();
 
                //---- Read from OGL and create a WPF image
                _bitmapData = _bitmap.LockBits(new System.Drawing.Rectangle(0, 0, _form.ClientSize.Width, _form.ClientSize.Height),
                                                                System.Drawing.Imaging.ImageLockMode.ReadWrite,
                                                                System.Drawing.Imaging.PixelFormat.Format32bppArgb);
 
                // BGRA is the fastest pixel format - everything else might involve conversions on the CPU.
                Gl.glReadBuffer(Gl.GL_COLOR_ATTACHMENT0_EXT);
                Gl.glReadPixels(0, 0, _form.ClientSize.Width, _form.ClientSize.Height, Gl.GL_BGRA, Gl.GL_UNSIGNED_BYTE, _bitmapData.Scan0);
 
                BitmapSource source = System.Windows.Interop.Imaging.CreateBitmapSourceFromHBitmap(_bitmap.GetHbitmap(),
                                                                                                IntPtr.Zero,
                                                                                                Int32Rect.Empty,
                                                                                                System.Windows.Media.Imaging.BitmapSizeOptions.FromEmptyOptions());
 
                image.Source = source;
 
                ReleaseFB();
                _bitmap.UnlockBits(_bitmapData);
            }
        }
 
        #endregion
 
        #region Refresh
 
        private static Action EmptyDelegate = delegate() { };
        public void Refresh()
        {
            this.InvalidateVisual();
            this.Dispatcher.Invoke(System.Windows.Threading.DispatcherPriority.Render, EmptyDelegate);
        }
 
        #endregion
 
        #region Interop
 
        public const int SRCCOPY = 13369376;
 
        [DllImport("kernel32", EntryPoint = "CreateFileMapping", SetLastError = true, CharSet = CharSet.Auto)]
        private static extern IntPtr CreateFileMapping(IntPtr hFile, IntPtr lpAttributes, int flProtect, int dwMaximumSizeLow, int dwMaximumSizeHigh, string lpName);
 
        [DllImport("kernel32", EntryPoint = "CloseHandle", SetLastError = true)]
        private static extern bool CloseHandle(IntPtr handle);
 
        [DllImport("gdi32.dll", EntryPoint = "DeleteDC")]
        public static extern IntPtr DeleteDC(IntPtr hDc);
 
        [DllImport("gdi32.dll", EntryPoint = "DeleteObject")]
        public static extern IntPtr DeleteObject(IntPtr hDc);
 
        [DllImport("gdi32.dll", EntryPoint = "BitBlt")]
        public static extern bool BitBlt(IntPtr hdcDest, int xDest, int yDest, int
        wDest, int hDest, IntPtr hdcSource, int xSrc, int ySrc, int RasterOp);
 
        [DllImport("gdi32.dll", EntryPoint = "CreateCompatibleBitmap")]
        public static extern IntPtr CreateCompatibleBitmap(IntPtr hdc, int nWidth, int nHeight);
 
        [DllImport("gdi32.dll", EntryPoint = "CreateCompatibleDC")]
        public static extern IntPtr CreateCompatibleDC(IntPtr hdc);
 
        [DllImport("gdi32.dll", EntryPoint = "SelectObject")]
        public static extern IntPtr SelectObject(IntPtr hdc, IntPtr bmp);
 
        #endregion
 
 
    }
 
    #region InvisibleForm
 
    public class InvisibleForm : System.Windows.Forms.Form
    {
        protected override void SetVisibleCore(bool value)
        {
            base.SetVisibleCore(false);
        }
    }
 
    #endregion
 
}

Try moving the line

_bitmap.UnlockBits(_bitmapData);

just after the ReadPixels call (before you create the BitmapSource).

Thanks,

I have already try... but it is the same situation.

Even, without FBO it works like this... I have just added a call to "CreateFrameBuffer" !

I am not familiar with WPF so I can't tell what's wrong here. Wild guess: BitmapSource expects to be created on the main thread.

A quick google search uncovered an interesting hack to modify an existing BitmapSource directly (instead of creating a System.Drawing.Bitmap). This should be much faster and could avoid the threading issues (you create the BitmapSource once in the main thread and modify its data as needed).

I already use it in another version... it change nothing because we already the buffer pointer here.

I have also try this (base on another sample about FBO) , but :

1) I can only create 2 PBOs
2) It display nothing ! and I don't know why !

To generate the PBO :

// generate the PBO
            uint g_pbo;
            Gl.glGenBuffersARB(1, out g_pbo);
 
            // bind it
            Gl.glBindBufferARB(Gl.GL_PIXEL_PACK_BUFFER_ARB, g_pbo);
 
            // Copy pixel data to the buffer object
            int size = _form.Width * _form.Height * 4;
            IntPtr ptrSize = Marshal.AllocHGlobal(sizeof(int));
            Marshal.StructureToPtr(size, ptrSize, true);
            Gl.glBufferDataARB(Gl.GL_PIXEL_PACK_BUFFER_ARB, ptrSize, null, Gl.GL_DYNAMIC_READ);
 
            //---- Mapping PBO
            _pboBufferPtr = Gl.glMapBufferARB(Gl.GL_PIXEL_PACK_BUFFER_ARB, Gl.GL_READ_WRITE_ARB);

To generate the image :

                if (_pboBufferPtr != IntPtr.Zero)
                    _bitmapSource = BitmapSource.Create(_form.ClientSize.Width,
                                            _form.ClientSize.Height,
                                            96,
                                            96,
                                            PixelFormats.Pbgra32,
                                            null,
                                            _pboBufferPtr,
                                            _form.ClientSize.Width * _form.ClientSize.Height * 4,
                                            _form.ClientSize.Width * 4);
 
 
                image.Source = _bitmapSource;
 
image.InvalidateVisual();

Another problem is that when I use glReadPixel the image is flipped vertically !

The OpenGl image is draw from the bottom to the top... and the XP image is from the top to the bottom.
Is there a way to change this ?

Home of the Open Toolkit library

OpenTK and WPF interoperation

Jump to:

Comments

Comment viewing options

#11

#12

#13

#14

#15

#16

#17

#18

#19

#20

Recent