the Fiddler's picture

OpenTK and WPF interoperation

Project:The Open Toolkit library
Version:all versions
Component:Code
Category:support request
Priority:normal
Assigned:Unassigned
Status:closed
Description

Right now, it is possible to create an OpenTK.GLControl and host it in a WPF using a WindowsFormsHost.

The question is: can we create an OpenGL context directly on a WPF window and do away with the host control?

The attached solution creates a WPF solution, obtains its handle and creates a GraphicsContext. This works without errors, however OpenGL rendering results in artifacts. The msdn entry for the OnRender method, notes that:

msdn wrote:

The rendering instructions for this element are not used directly when this method is invoked, and are instead preserved for later asynchronous use by layout and drawing.

This explains why the rendering artifacts appear: OpenGL tries to render to the window directly and does not participate into WPF composition.

One *could* use offscreen OpenGL rendering, perform a readback and present the results into the WPF window, but this is inefficient. It would be nice to find a better solution or confirm that it cannot be done.

Any ideas?

AttachmentSize
OpenTKWPF.zip1.81 MB

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
the Fiddler's picture

#11

Yes, modern systems SwapBuffers is pretty fast on modern hardware (microseconds) and the ReadPixels call overshadows it completely.

Auxiliary buffers are a relic of the past, there's absolutely no reason to use them nowadays.

viewon01's picture

#12

Ok, so I will remove their use...

But what will be the best solution... I have also try to use FBO but I can't !

When I call :

Gl.glGenRenderbuffersEXT(1, out DepthRenderbuffer);

It crash !!

Also, how do I specify to OpenGL to draw on the FBO

Here is my code to create the FBO :

private void CreateFrameBuffer()
{
            // Create Color Texture
            Gl.glEnable(Gl.GL_TEXTURE_2D);
            Gl.glGenTextures(1, out ColorTexture);
            Gl.glBindTexture(Gl.GL_TEXTURE_2D, ColorTexture);
            Gl.glTexParameteri(Gl.GL_TEXTURE_2D, Gl.GL_TEXTURE_MIN_FILTER, Gl.GL_NEAREST);
            Gl.glTexParameteri(Gl.GL_TEXTURE_2D, Gl.GL_TEXTURE_MAG_FILTER, Gl.GL_NEAREST);
            Gl.glTexParameteri(Gl.GL_TEXTURE_2D, Gl.GL_TEXTURE_WRAP_S, Gl.GL_CLAMP);
            Gl.glTexParameteri(Gl.GL_TEXTURE_2D, Gl.GL_TEXTURE_WRAP_T, Gl.GL_CLAMP);
            Gl.glTexImage2D(Gl.GL_TEXTURE_2D, 0, Gl.GL_RGBA8, FboWidth, FboHeight, 0, Gl.GL_RGBA, Gl.GL_UNSIGNED_BYTE, IntPtr.Zero);
 
            // test for GL Error here (might be unsupported format)
 
            Gl.glBindTexture(Gl.GL_TEXTURE_2D, 0); // prevent feedback, reading and writing to the same image is a bad idea
 
            // Create Depth Renderbuffer
            Gl.glGenRenderbuffersEXT(1, out DepthRenderbuffer);
            Gl.glBindRenderbufferEXT(Gl.GL_RENDERBUFFER_EXT, DepthRenderbuffer);
            Gl.glRenderbufferStorageEXT(Gl.GL_RENDERBUFFER_EXT, Gl.GL_DEPTH_COMPONENT32, FboWidth, FboHeight);
 
            // test for GL Error here (might be unsupported format)
 
            // Create a FBO and attach the textures
            Gl.glGenFramebuffersEXT(14, out FboHandle);
            Gl.glBindFramebufferEXT(Gl.GL_FRAMEBUFFER_EXT, FboHandle);
            Gl.glFramebufferTexture2DEXT(Gl.GL_FRAMEBUFFER_EXT, Gl.GL_COLOR_ATTACHMENT0_EXT, Gl.GL_TEXTURE_2D, ColorTexture, 0);
            Gl.glFramebufferRenderbufferEXT(Gl.GL_FRAMEBUFFER_EXT, Gl.GL_DEPTH_ATTACHMENT_EXT, Gl.GL_RENDERBUFFER_EXT, DepthRenderbuffer);
 
            // now GL.Ext.CheckFramebufferStatus( FramebufferTarget.FramebufferExt ) can be called, check the end of this page for a snippet.
 
            // since there's only 1 Color buffer attached this is not explicitly required
            Gl.glDrawBuffer(Gl.GL_COLOR_ATTACHMENT0_EXT);
 
            Gl.glPushAttrib(Gl.GL_VIEWPORT_BIT); // stores GL.Viewport() parameters
            Gl.glViewport(0, 0, FboWidth, FboHeight);
 
            // render whatever your heart desires, when done ...
}

Can you help me to debug it ?

Thanks

the Fiddler's picture

#13

Do you get a crash or a NullReferenceException? The former indicates a driver bug, while the latter means that your video card does not support FBOs.

Have you checked for FBO support? Search the extension string for "EXT_framebuffer_object": Gl.glGetString(Gl.GL_EXTENSIONS).Contains("EXT_framebuffer_object"). Many Intel cards don't support this (try updating your drivers).

viewon01's picture

#14

Your're right, I have update my drivers and now all is fine...

Except that when I try to create the BitmapSource... the application is blocked !!!

Here is the code of the class, can you take a closer look... it sounds that the FBO is locked or something like this !

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Data;
using System.Windows.Documents;
using System.Windows.Input;
using System.Windows.Media;
using System.Windows.Media.Imaging;
using System.Windows.Navigation;
using System.Windows.Shapes;
using System.Windows.Interop;
using System.Runtime.InteropServices;
using System.Diagnostics;
 
using Tao.OpenGl;
using Tao.Platform.Windows;
 
namespace RS.UI
{
 
    public partial class OGLPanel : UserControl
    {
 
        #region Variables
 
        public event EventHandler OnOGLRender;
 
        private bool _isInitialized = false;
 
        //---- Global rendering lock
        static object RENDER_LOCK = new object();
 
        //----
        private IntPtr deviceContext = IntPtr.Zero;                         // GDI device context
        private IntPtr renderingContext = IntPtr.Zero;                      // Rendering context
        private IntPtr windowHandle = IntPtr.Zero;                          // Holds our window handle
        private bool autoCheckErrors = false;                               // Should we provide glGetError()?
        private bool autoFinish = false;                                    // Should we provide a glFinish()?
        private bool autoMakeCurrent = true;                                // Should we automatically make the rendering context current?
        private bool autoSwapBuffers = true;                                // Should we automatically swap buffers?
        private byte accumBits = 0;                                         // Accumulation buffer bits
        private byte colorBits = 32;                                        // Color buffer bits
        private byte depthBits = 16;                                        // Depth buffer bits
        private byte stencilBits = 0;                                       // Stencil buffer bits
        private int errorCode = Gl.GL_NO_ERROR;                             // The GL error code
 
        private int logScaleX = 96;                                         // DPI Resolution in X dir
        private int logScaleY = 96;                                         // DPI Resolution in Y dir
 
        private InvisibleForm _form;
 
        private System.Drawing.Bitmap _bitmap;
        private System.Drawing.Imaging.BitmapData _bitmapData;
 
        #endregion
 
        #region Constructor
 
        public OGLPanel()
        {
            InitializeComponent();
            this.SizeChanged += new SizeChangedEventHandler(OGLPanel_SizeChanged);
        }
 
        #endregion
 
        #region OGLPanel_SizeChanged
 
        void OGLPanel_SizeChanged(object sender, SizeChangedEventArgs e)
        {
            if (!_isInitialized)
                return;
 
            _form.ClientSize = new System.Drawing.Size((int)ActualWidth, (int)ActualHeight);
            _bitmap = new System.Drawing.Bitmap(_form.ClientSize.Width, _form.ClientSize.Height, System.Drawing.Imaging.PixelFormat.Format32bppArgb);
 
            Refresh();
        }
 
        #endregion
 
        #region Initialize
 
        public void Initialize()
        {
            if (_isInitialized)
                return;
 
            InitializeContexts();
 
            _form.ClientSize = new System.Drawing.Size((int)ActualWidth, (int)ActualHeight);
            _bitmap = new System.Drawing.Bitmap(_form.ClientSize.Width, _form.ClientSize.Height, System.Drawing.Imaging.PixelFormat.Format32bppArgb);
 
            _isInitialized = true;
        }
 
        #endregion
 
        #region InitializeContexts
 
        /// <summary>  
        /// Creates the OpenGL contexts.
        /// </summary>
        public void InitializeContexts()
        {
            int pixelFormat;                                                // Holds the selected pixel format
 
            //--- Create the window handle
            //HwndSource hwndSource = PresentationSource.FromVisual(this) as HwndSource;
            //HwndTarget hwndTarget = hwndSource.CompositionTarget;
 
            //// this is the new WPF API to force render mode. 
            //hwndTarget.RenderMode = RenderMode.SoftwareOnly;
 
            //windowHandle = hwndSource.Handle;                                     // Get window handle
 
            _form = new InvisibleForm();
            _form.Show();
 
            windowHandle = _form.Handle;
 
            if (windowHandle == IntPtr.Zero)
                Environment.Exit(-1);
 
            //---- Create the device context
            deviceContext = User.GetDC(windowHandle);                       // Attempt to get the device context
            if (deviceContext == IntPtr.Zero)
                Environment.Exit(-1);
 
            //---- Define the Gdi pixel format
            Gdi.PIXELFORMATDESCRIPTOR pfd = new Gdi.PIXELFORMATDESCRIPTOR();// The pixel format descriptor
            pfd.nSize = (short)Marshal.SizeOf(pfd);                        // Size of the pixel format descriptor
            pfd.nVersion = 1;                                               // Version number (always 1)
            pfd.dwFlags = Gdi.PFD_DRAW_TO_WINDOW |                          // Format must support windowed mode
                        Gdi.PFD_SUPPORT_OPENGL |                            // Format must support OpenGL
                        Gdi.PFD_DOUBLEBUFFER;                               // Must support double buffering
            pfd.iPixelType = (byte)Gdi.PFD_TYPE_RGBA;                      // Request an RGBA format
            pfd.cColorBits = (byte)colorBits;                              // Select our color depth
            pfd.cRedBits = 0;                                               // Individual color bits ignored
            pfd.cRedShift = 0;
            pfd.cGreenBits = 0;
            pfd.cGreenShift = 0;
            pfd.cBlueBits = 0;
            pfd.cBlueShift = 0;
            pfd.cAlphaBits = 0;                                             // No alpha buffer
            pfd.cAlphaShift = 0;                                            // Alpha shift bit ignored
            pfd.cAccumBits = accumBits;                                     // Accumulation buffer
            pfd.cAccumRedBits = 0;                                          // Individual accumulation bits ignored
            pfd.cAccumGreenBits = 0;
            pfd.cAccumBlueBits = 0;
            pfd.cAccumAlphaBits = 0;
            pfd.cDepthBits = depthBits;                                     // Z-buffer (depth buffer)
            pfd.cStencilBits = stencilBits;                                 // No stencil buffer
            pfd.cAuxBuffers = 0;                                            // No auxiliary buffer
            pfd.iLayerType = (byte)Gdi.PFD_MAIN_PLANE;                     // Main drawing layer
            pfd.bReserved = 0;                                              // Reserved
            pfd.dwLayerMask = 0;                                            // Layer masks ignored
            pfd.dwVisibleMask = 0;
            pfd.dwDamageMask = 0;
            pixelFormat = Gdi.ChoosePixelFormat(deviceContext, ref pfd);    // Attempt to find an appropriate pixel format
            if (pixelFormat == 0)
                Environment.Exit(-1);
 
            // Are we not able to set the pixel format?
            if (!Gdi.SetPixelFormat(deviceContext, pixelFormat, ref pfd))
                Environment.Exit(-1);
 
            //----
            logScaleX = Gdi.GetDeviceCaps(deviceContext, (int)Gdi.DevCaps.LOGPIXELSX); // Attempt to retrieve DPI-Setting
            logScaleY = Gdi.GetDeviceCaps(deviceContext, (int)Gdi.DevCaps.LOGPIXELSY); // Attempt to retrieve DPI-Setting
 
            //---- Create a rendering context
            renderingContext = Wgl.wglCreateContext(deviceContext);         // Attempt to get the rendering context
            if (renderingContext == IntPtr.Zero)
                Environment.Exit(-1);
 
            _isInitialized = true;
 
            MakeCurrent();                                                  // Attempt to activate the rendering context
 
            // Force A Reset On The Working Set Size
            Kernel.SetProcessWorkingSetSize(Process.GetCurrentProcess().Handle, -1, -1);
        }
 
        #endregion
 
        #region CreateFrameBuffer
 
        int FboWidth = 512;
        int FboHeight = 512;
 
        uint FboHandle;
        uint ColorTexture;
        uint DepthRenderbuffer;
 
        private void CreateFrameBuffer()
        {
            // Does it support FBO ?
            if (!Gl.glGetString(Gl.GL_EXTENSIONS).Contains("EXT_framebuffer_object"))
                return;
 
            FboWidth = _form.ClientSize.Width;
            FboHeight = _form.ClientSize.Height;
 
            // Create Color Texture
            Gl.glEnable(Gl.GL_TEXTURE_2D);
            Gl.glGenTextures(1, out ColorTexture);
            Gl.glBindTexture(Gl.GL_TEXTURE_2D, ColorTexture);
            Gl.glTexParameteri(Gl.GL_TEXTURE_2D, Gl.GL_TEXTURE_MIN_FILTER, Gl.GL_NEAREST);
            Gl.glTexParameteri(Gl.GL_TEXTURE_2D, Gl.GL_TEXTURE_MAG_FILTER, Gl.GL_NEAREST);
            Gl.glTexParameteri(Gl.GL_TEXTURE_2D, Gl.GL_TEXTURE_WRAP_S, Gl.GL_CLAMP);
            Gl.glTexParameteri(Gl.GL_TEXTURE_2D, Gl.GL_TEXTURE_WRAP_T, Gl.GL_CLAMP);
            Gl.glTexImage2D(Gl.GL_TEXTURE_2D, 0, Gl.GL_RGBA8, FboWidth, FboHeight, 0, Gl.GL_RGBA, Gl.GL_UNSIGNED_BYTE, IntPtr.Zero);
 
            // test for GL Error here (might be unsupported format)
 
            Gl.glBindTexture(Gl.GL_TEXTURE_2D, 0); // prevent feedback, reading and writing to the same image is a bad idea
 
            // Create Depth Renderbuffer
            Gl.glGenRenderbuffersEXT(1, out DepthRenderbuffer);
            Gl.glBindRenderbufferEXT(Gl.GL_RENDERBUFFER_EXT, DepthRenderbuffer);
            Gl.glRenderbufferStorageEXT(Gl.GL_RENDERBUFFER_EXT, Gl.GL_DEPTH_COMPONENT32, FboWidth, FboHeight);
 
            // test for GL Error here (might be unsupported format)
 
            // Create a FBO and attach the textures
            Gl.glGenFramebuffersEXT(14, out FboHandle);
            Gl.glBindFramebufferEXT(Gl.GL_FRAMEBUFFER_EXT, FboHandle);
            Gl.glFramebufferTexture2DEXT(Gl.GL_FRAMEBUFFER_EXT, Gl.GL_COLOR_ATTACHMENT0_EXT, Gl.GL_TEXTURE_2D, ColorTexture, 0);
            Gl.glFramebufferRenderbufferEXT(Gl.GL_FRAMEBUFFER_EXT, Gl.GL_DEPTH_ATTACHMENT_EXT, Gl.GL_RENDERBUFFER_EXT, DepthRenderbuffer);
 
            // now GL.Ext.CheckFramebufferStatus( FramebufferTarget.FramebufferExt ) can be called, check the end of this page for a snippet.
 
            // since there's only 1 Color buffer attached this is not explicitly required
            Gl.glDrawBuffer(Gl.GL_COLOR_ATTACHMENT0_EXT);
 
            Gl.glPushAttrib(Gl.GL_VIEWPORT_BIT); // stores GL.Viewport() parameters
            Gl.glViewport(0, 0, FboWidth, FboHeight);
 
            // render whatever your heart desires, when done ...
        }
 
        private void ReleaseFB()
        {
            // Does it support FBO ?
            if (!Gl.glGetString(Gl.GL_EXTENSIONS).Contains("EXT_framebuffer_object"))
                return;
 
            Gl.glPopAttrib(); // restores GL.Viewport() parameters
            Gl.glBindFramebufferEXT(Gl.GL_FRAMEBUFFER_EXT, 0); // return to visible framebuffer
            Gl.glDrawBuffer(Gl.GL_BACK);
        }
 
        #endregion
 
        #region MakeCurrent
 
        public void MakeCurrent()
        {
            if (!_isInitialized)
                return;
 
            // Are we not able to activate the rending context?
            if (!Wgl.wglMakeCurrent(deviceContext, renderingContext))
            {
                //MessageBox.Show("Can not activate the GL rendering context.", "Fatal Error", MessageBoxButtons.OK, MessageBoxIcon.Error);
                Environment.Exit(-1);
            }
        }
 
        #endregion
 
        #region SwapBuffers
 
        public void SwapBuffers()
        {
            Gdi.SwapBuffersFast(deviceContext);
            //lock(LOCK_BUFFER)
            //{
            //    unsafe
            //    {
            //        GL.ReadPixels(0,
            //            0,
            //            bufferWidth,
            //            bufferHeight,
            //            OpenTK.Graphics.OpenGL.PixelFormat.Rgb,
            //            PixelType.Bitmap,
            //            buffer);
 
            //        System.Windows.Media.PixelFormat format = PixelFormats.Rgb24;
 
            //        InteropBitmap source = System.Windows.Interop.Imaging.CreateBitmapSourceFromMemorySection(
            //            buffer,
            //            bufferWidth,
            //            bufferHeight,
            //            format,
            //            (int)(bufferWidth * format.BitsPerPixel / 8), 0) as InteropBitmap;
 
            //        source.Invalidate();
            //        this.Source = (BitmapSource)source.GetAsFrozen();
            //    }
            //}
        }
 
        #endregion
 
        #region OnRender
 
        protected override void OnRender(DrawingContext drawingContext)
        {
            base.OnRender(drawingContext);
 
            if (!_isInitialized)
                return;
 
            lock (RENDER_LOCK)
            {
                MakeCurrent();
 
                CreateFrameBuffer();
 
                if (OnOGLRender != null)
                    OnOGLRender(null, null);
 
                //SwapBuffers();
 
                Gl.glFlush();
                Gl.glFinish();
 
                //---- Read from OGL and create a WPF image
                _bitmapData = _bitmap.LockBits(new System.Drawing.Rectangle(0, 0, _form.ClientSize.Width, _form.ClientSize.Height),
                                                                System.Drawing.Imaging.ImageLockMode.ReadWrite,
                                                                System.Drawing.Imaging.PixelFormat.Format32bppArgb);
 
                // BGRA is the fastest pixel format - everything else might involve conversions on the CPU.
                Gl.glReadBuffer(Gl.GL_COLOR_ATTACHMENT0_EXT);
                Gl.glReadPixels(0, 0, _form.ClientSize.Width, _form.ClientSize.Height, Gl.GL_BGRA, Gl.GL_UNSIGNED_BYTE, _bitmapData.Scan0);
 
                BitmapSource source = System.Windows.Interop.Imaging.CreateBitmapSourceFromHBitmap(_bitmap.GetHbitmap(),
                                                                                                IntPtr.Zero,
                                                                                                Int32Rect.Empty,
                                                                                                System.Windows.Media.Imaging.BitmapSizeOptions.FromEmptyOptions());
 
                image.Source = source;
 
                ReleaseFB();
                _bitmap.UnlockBits(_bitmapData);
            }
        }
 
        #endregion
 
        #region Refresh
 
        private static Action EmptyDelegate = delegate() { };
        public void Refresh()
        {
            this.InvalidateVisual();
            this.Dispatcher.Invoke(System.Windows.Threading.DispatcherPriority.Render, EmptyDelegate);
        }
 
        #endregion
 
        #region Interop
 
        public const int SRCCOPY = 13369376;
 
        [DllImport("kernel32", EntryPoint = "CreateFileMapping", SetLastError = true, CharSet = CharSet.Auto)]
        private static extern IntPtr CreateFileMapping(IntPtr hFile, IntPtr lpAttributes, int flProtect, int dwMaximumSizeLow, int dwMaximumSizeHigh, string lpName);
 
        [DllImport("kernel32", EntryPoint = "CloseHandle", SetLastError = true)]
        private static extern bool CloseHandle(IntPtr handle);
 
        [DllImport("gdi32.dll", EntryPoint = "DeleteDC")]
        public static extern IntPtr DeleteDC(IntPtr hDc);
 
        [DllImport("gdi32.dll", EntryPoint = "DeleteObject")]
        public static extern IntPtr DeleteObject(IntPtr hDc);
 
        [DllImport("gdi32.dll", EntryPoint = "BitBlt")]
        public static extern bool BitBlt(IntPtr hdcDest, int xDest, int yDest, int
        wDest, int hDest, IntPtr hdcSource, int xSrc, int ySrc, int RasterOp);
 
        [DllImport("gdi32.dll", EntryPoint = "CreateCompatibleBitmap")]
        public static extern IntPtr CreateCompatibleBitmap(IntPtr hdc, int nWidth, int nHeight);
 
        [DllImport("gdi32.dll", EntryPoint = "CreateCompatibleDC")]
        public static extern IntPtr CreateCompatibleDC(IntPtr hdc);
 
        [DllImport("gdi32.dll", EntryPoint = "SelectObject")]
        public static extern IntPtr SelectObject(IntPtr hdc, IntPtr bmp);
 
        #endregion
 
 
    }
 
    #region InvisibleForm
 
    public class InvisibleForm : System.Windows.Forms.Form
    {
        protected override void SetVisibleCore(bool value)
        {
            base.SetVisibleCore(false);
        }
    }
 
    #endregion
 
}
the Fiddler's picture

#15

Try moving the line

_bitmap.UnlockBits(_bitmapData);

just after the ReadPixels call (before you create the BitmapSource).

viewon01's picture

#16

Thanks,

I have already try... but it is the same situation.

Even, without FBO it works like this... I have just added a call to "CreateFrameBuffer" !

the Fiddler's picture

#17

I am not familiar with WPF so I can't tell what's wrong here. Wild guess: BitmapSource expects to be created on the main thread.

A quick google search uncovered an interesting hack to modify an existing BitmapSource directly (instead of creating a System.Drawing.Bitmap). This should be much faster and could avoid the threading issues (you create the BitmapSource once in the main thread and modify its data as needed).

viewon01's picture

#18

I already use it in another version... it change nothing because we already the buffer pointer here.

viewon01's picture

#19

I have also try this (base on another sample about FBO) , but :

1) I can only create 2 PBOs
2) It display nothing ! and I don't know why !

To generate the PBO :

// generate the PBO
            uint g_pbo;
            Gl.glGenBuffersARB(1, out g_pbo);
 
            // bind it
            Gl.glBindBufferARB(Gl.GL_PIXEL_PACK_BUFFER_ARB, g_pbo);
 
            // Copy pixel data to the buffer object
            int size = _form.Width * _form.Height * 4;
            IntPtr ptrSize = Marshal.AllocHGlobal(sizeof(int));
            Marshal.StructureToPtr(size, ptrSize, true);
            Gl.glBufferDataARB(Gl.GL_PIXEL_PACK_BUFFER_ARB, ptrSize, null, Gl.GL_DYNAMIC_READ);
 
            //---- Mapping PBO
            _pboBufferPtr = Gl.glMapBufferARB(Gl.GL_PIXEL_PACK_BUFFER_ARB, Gl.GL_READ_WRITE_ARB);

To generate the image :

                if (_pboBufferPtr != IntPtr.Zero)
                    _bitmapSource = BitmapSource.Create(_form.ClientSize.Width,
                                            _form.ClientSize.Height,
                                            96,
                                            96,
                                            PixelFormats.Pbgra32,
                                            null,
                                            _pboBufferPtr,
                                            _form.ClientSize.Width * _form.ClientSize.Height * 4,
                                            _form.ClientSize.Width * 4);
 
 
                image.Source = _bitmapSource;
 
image.InvalidateVisual();
viewon01's picture

#20

Another problem is that when I use glReadPixel the image is flipped vertically !

The OpenGl image is draw from the bottom to the top... and the XP image is from the top to the bottom.
Is there a way to change this ?