djk's picture

Selection/Picking

Background:
Over the last 20 years I have been using the Hoops3d graphics system from TechSoft to provide rendering and graphical selection of objects in a large engineering application. My last foray into OpenGl was 15 years ago on an SGI Onyx using the GLUT framework to render a simulation there was no picking required.

I have recently started with a new company that is chartered to create a similar engineering system as open source using the .Net framework. The requirements have lead us to OpenGL and the OpenTK GlControl has risen to the top of the list from the 3-4 open source C# OpenGl controls evaluated.

In the Hoops3d system one would apply a filter to the segment tree (scene graph) to specify what you were going to select, ie whole object, face of a shell, line/edge, or vertex. Every graphical item had a key which could be remapped to a reference to the object that generated it. Thus when you selected an item you accessed the key to get the underlying object to operate on.

Question:
I have tried to port several picking implementations from other C/C++ projects to the OpenTK GlControl without success, I am sure the issue is with my inability to get my head wrapped around the OpenGL picking model. Does anyone have an example app built on OpenTk's GlControl that implements picking that they would be willing to share?

thanks
djk

AttachmentSize
SelectionExample.zip87.83 KB

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
JTalton's picture

If that does not do it for you, let us know.

teichgraf's picture

There is also another, very easy and efficient way which should work not only with OpenGL. This method uses Unique Color IDs and is also described in the RedBook.

So how does it work?
From Lighhouse:
The color coding scheme does not require any perspective changes and therefore it is simpler in theory. Just define a rendering function where the relevant objects (pickable and occluders) are assigned each a different color. When the user clicks the mouse over the scene, render the scene on the back buffer, read back the selected pixel from the back buffer and check its color. The process is completely transparent to the user because the buffers are not swapped, so the color coding rendering is never seen.

You can find some nice tutorials here:
http://gpwiki.org/index.php/OpenGL_Selection_Using_Unique_Color_IDs
and here:
http://www.lighthouse3d.com/opengl/picking/index.php3?color1

The colour-coded method also has some drawbacks:

  • Pickable objects are limited by the colour depth.
  • You can only get the first object in the front, you won't get a list of overlapped objects. But in most cases you are only interessed in the front object.

But is has a main advantage, it should be significant faster than the deprecated OpenGL Selection mode. The GL_SELECT method is done in software (!) and the colour-coded picking is just an extra render pass (in hardware) without light & stuff.

I implemented this method some time ago in a CAD/CAM system which could have a lot of pickable objects. For that I used the aux buffer.
Using the colour-coded picking was almost 4 times faster than the OGL Selection mode. And I also heared that GL_SELECT will be dropped in OGL 3.0.

There are also some discussions about that on the web. For example:
http://www.opengl.org/discussion_boards/ubbthreads.php?ubb=showflat&Numb...

And these are not the only methods for picking (Ray-Intersection test, ...).
If someone is interessed, I can do a little example for colour-coded picking?
Where can I find the OpenTK example for GL_SELECT, so I can use it as a starter?

Technolithic's picture

You folks are awesome!

JTalton, thanks.

teichgraf, CAD/CAM sounds right up my alley. I built a CNC mill by hand. It took forever to get the parts close enough. The machine works, but the controller board doesn't anymore... needs a new one. It would be awesome to contribute to open source CAD/CAM development... then when I get a new controller board the machine could come to life better than before!

Anyhow, I think I'm stuck to using the picking technique because the UI needs to allow the user to click and drag a box around a section of the view-port, which needs to return objects at a depth. I'm still looking into getting it implemented. Apparently, the picking method has an improvement over the selection method in that it only "draws" the objects that are within the tolerance of the point of reference.

triton's picture

teichgraf: If someone is interessed, I can do a little example for colour-coded picking?

I would be interested. :)

djk's picture

I would be very interested, and I was the one who started this thread.

Our application is essential a CAD Product Model system where I need to "pick/select" whole objects, faces of a solid, edges of a face and finally the vertex of the edges.

I never got past the selection of the whole object with the code contained in the example I provided at the top of the thread.

I am back working on the graphics aspects of the application for the next two weeks, so this is a very timely offer.

djk

Technolithic's picture

Ok, I have a bunch of noob questions, if someone is willing to help me harvest the spoils of OpenGL.

Why is it necessary to save the current transformation state to later recall it?
Why couldn't the existing matrix be applied to the selection/picking method?

With the transformations already applied to the existing matrix, it makes little sense to me to have to create a new one for drawing the selection to.

The SelectBuffer, the array that the GL.SelectBuffer method requires for writing hits to, how does this SelectBuffer work? The examples are confusing when they start out with the 1st value without a name. How does the name get there and why doesn't it have a name? Why add a hit to the buffer if it has no reference to it?

An one last one to help me figure out where I'm at: Is it appropriate to ask these questions? I mean, maybe I should have picked up this stuff through all the reading, gack!

Can't call this method until this... can't do this before this... need this before this...
Maybe I'm thinking about it too hard and the answers are really only obvious.

JTalton's picture

You save off the current projection matrix.
Then you clear the projection matrix with LoadIdentity().
Then you initialize the matrix with your Glu.PickMatrix(x, y, 0.001f, 0.001f, viewport);
- The 0.001f here is actuall used to create a small area. You can put in bigger values to pick a region about the x, y point.
You then multiple that matrix times the original projection matrix
Which gives you a pick matrix for the area you want using the correct projection matrix.

Not sure about the examples added the 1st hit value without a name.
You have to call GL.LoadName(id) before drawing each object.
You cannot call GL.LoadName inside a GL.Begin() and GL.End(), so you have to break the drawing up.

4 values are put in the selction buffer for each object in the area specified by your projection matrix. (using the PickMatrix)
The distance is at an offset of 1. uint distance = (uint)selectBuffer[i * 4 + 1];
The object id is at the offset of 3. selectedId = (int)selectBuffer[i * 4 + 3];

teichgraf's picture

I have written a little OpenTK example based on a OpenTK WinForms example. You can download the source file "W05_Picking" here or as a SVN patch from here.
Usage:
Click on a sphere to select it. Press [C] to see the rendered color-index image, which is used for this technique.

The whole "magic" happens in the methods DoPicking and DrawColorIds:

private void glControl_MouseDown(object sender, MouseEventArgs e)
{
   DoPicking(e.X, e.Y);
}
 
private void DoPicking(int x, int y)
{
   // Draw to back buffer
   DrawColorIds();
 
   // Read pixel from back buffer at mouse pos.
   byte[] pixel = new byte[3];
   int[] viewport = new int[4];
   // Flip Y-axis (Windows <-> OpenGL)
   GL.GetInteger(GetPName.Viewport, viewport);
   GL.ReadPixels(x, viewport[3] - y, 1, 1, PixelFormat.Rgb, PixelType.UnsignedByte, pixel);
   // Since the color is the list index, we can use it directly and don't need to search the list for it
   int index = (int)pixel[0] + (((int)pixel[1]) << 8) + ((((int)pixel[2]) << 16));
   if (index > -1 && index < shapes.Count)
   {
	  selectedShape = shapes[index];
   }
}
 
private void DrawColorIds()
{
   // Disable some caps (we want the flat / raw objects)
   GL.PushAttrib(AttribMask.EnableBit | AttribMask.ColorBufferBit);
   GL.Disable(EnableCap.Fog);
   GL.Disable(EnableCap.Texture2D);
   GL.Disable(EnableCap.Dither);
   GL.Disable(EnableCap.Lighting);
   GL.Disable(EnableCap.LineStipple);
   GL.Disable(EnableCap.PolygonStipple);
   GL.Disable(EnableCap.CullFace);
   GL.Disable(EnableCap.Blend);
   GL.Disable(EnableCap.AlphaTest);
 
   // Clear the buffer
   GL.ClearColor(1.0f, 1.0f, 1.0f, 1.0f);
   GL.Clear(ClearBufferMask.ColorBufferBit | ClearBufferMask.DepthBufferBit);
 
   // Set unique color based on list index and draw
   // If the index isn't fix like here, you should use a real unique id as colour
   for (int i = 0; i < shapes.Count; i++)
   {
	  byte r = (byte) (i & 0x000000FF);
	  byte g = (byte)((i & 0x0000FF00) >> 08);
	  byte b = (byte)((i & 0x00FF0000) >> 16);
	  GL.Color3(r, g, b);
	  shapes[i].Draw();
   }
 
   // Restore the caps
   GL.PopAttrib();
}

Not that much. ;-)
There are a lot of improvements which could be done:

  • The aux buffer could be used instead of the back buffer for the color-index rendering.
  • It would also be better to read the pixels in a certain range around the mouse position and estimate the one with highest occurrence. This way it would be better to select small objects.

...

Technolithic's picture

I'm lost in the sauce with the matrix push/pop and transformations. I just don't understand what's going on there. Here's the chunk of code that I'm working on that pertains to rendering and picking. I'm trying to output the hits to the listbox object in this form; although, ultimately, I'll be doing something with the data that goes into the listbox... it would be nice to see the listbox provide some kind of info if I'm headed in the right direction.

        // PROPERTIES:
        #region Properties      
 
        // OpenGL window variables
        private OpenTK.GLControl glControl1;        
        private bool loaded = false;
 
        // data info
        private List<Vertex> ModelRenderData;
        private byte yBranchFactor = 3;
        private int iDepth = 3;
 
        // View-port Control Variables
        // mouse position variables
        private Point MousePoint1;
        private Point MousePoint2;
 
        // model rotation orientation
        private float fRotateX;
        private float fRotateY;
        private float fRotateZ = 0f;
 
        // camera position
        private float fTransX;
        private float fTransY;
 
        // zoom ratio
        private float fZoom = 45.0f;
 
 
        // Viewport object user-selection variables
        static private OpenTK.OpenGL.Enums.RenderingMode RenderMode = RenderingMode.Render;        
        static private int BUFFER_SIZE = 256;
        private ListBox lbxSelection;
 
        // Last mouse event
        private MouseEventArgs LastMouseEvent;
 
        #endregion Properties
 
 
        public MeshViewport()
        {
            ModelRenderData = new List<Vertex>();
            SetModelShape(yModelShape);
            InitializeComponent();
        }
 
 
        private void MeshViewport_Load(object sender, EventArgs e)
        {
            loaded = true;
            GL.ClearColor(Color.Black);
            SetupViewport();
        }
 
 
        private void SetupViewport()
        {
            int w = glControl1.Width;
            int h = glControl1.Height;
            int n; // indexer
 
            GL.ClearColor(0f, 0f, 0f, 1f);
 
            GL.Viewport(0, 0, w, h); // Use all of the glControl painting area
            GL.Enable(EnableCap.DepthTest);
        }
 
 
        private void glControl1_Resize(object sender, EventArgs e)
        {
            int myWidth = Width;
 
            if (!loaded)
                return;
 
            if (myWidth == 0)
                myWidth = 1;
 
            RenderMeshModel();
        }
 
 
        private void lblViewPort_Click(object sender, EventArgs e)
        {
 
        }
 
 
        // render the graphics in the viewport
        private void glControl1_Paint(object sender, PaintEventArgs e)
        {
            if (!loaded) // don't execute if not already loaded!
                return;
 
            RenderMeshModel();
        }
 
 
        // RenderMeshModel:
        //
        // This function takes all of the modeling data and renders it to the screen
        public void RenderMeshModel()
        {
            int n, m;              // indexers
            int iMPCL = 0;      // max population of current level
            int iCurrentLevelPopulation = 0; // pop of current vertex within the current level
            int iCurrentLevel = 0;           // current depth level of tree
            int[] viewport; // place to retrieve the viewport numbers
            double[] doubleArray;
 
            // Set Link Width
            GL.LineWidth(1.0f);
            //GL.RenderMode(RenderMode);
 
            GL.Clear(ClearBufferMask.ColorBufferBit | ClearBufferMask.DepthBufferBit);
 
            GL.MatrixMode(MatrixMode.Modelview);
            GL.LoadIdentity();
 
            // Set the camera zoom
            Glu.Perspective(fZoom, 1.0, 1.0, 2048);
 
            // Set panning position
            GL.Translate(fTransX, fTransY, -500f);
 
            // Rotate the mesh model about it's own axis
            GL.Rotate(fRotateX, 1f, 0f, 0f);
            GL.Rotate(fRotateY, 0f, 1f, 0f);
            GL.Rotate(fRotateZ, 0f, 0f, 1f);
 
 
            for (n = 0; n < ModelRenderData.Count; n++)
            {
                //------------------------------------------------------------------------------------------------------------------
                // Handle which level the rendering is taking place:
 
                iCurrentLevelPopulation++;
 
                // if found to be next level
 
                if (iCurrentLevelPopulation > iMPCL)
                {
                    iCurrentLevel++; // increment to next level
                    iMPCL = (int)Math.Pow((double)yBranchFactor, (double)iCurrentLevel); // recalculate maximum population of next level
                    iCurrentLevelPopulation = 1; // reset current level population to one (1)
                }
                //------------------------------------------------------------------------------------------------------------------
 
                // DRAW PARENT-CHILD SEGMENTS:
                DrawSegment(ModelRenderData[n].GetThisVertex(), ModelRenderData[n].GetVertexParent(), 1);
 
                // DRAW LINK AND REFERENCE SEGMENTS, iterate for each link
                for (m = 0; m < ModelRenderData[n].GetVertexLinks().Count; m++)
                {
                    DrawSegment(ModelRenderData[n].GetThisVertex(), ModelRenderData[n].GetVertexLink(m).GetThisVertex(), 1);
                    DrawSegment(ModelRenderData[n].GetThisVertex(), ModelRenderData[n].GetVertexLink(m).GetThisVertex(), 2);                    
                }
 
                // DRAW NODE:
                //GL.RenderMode(RenderingMode.Render);
                GL.LoadName(n);
                DrawVertex(ModelRenderData[n].GetThisVertex(), fNodeDiameter + iCurrentLevel * fNodeDiameterStepping + 5);
            }
        }
 
 
        // This fuction will later be for drawing cones for link, reference, and parent-child relationships.
        // type: 1 for outbound connection, 2 for inbound connection
        // tpye is used to determine which direction the data stream is flowing
        private void DrawSegment(Vertex Src, Vertex Dst, byte type)
        {
            Vertex StreamSource = null;
 
            // Set color depending on the origination of the data stream
            switch (type)
            {
                case 1: StreamSource = Src; break;
                case 2: StreamSource = Dst; break;
            }
 
            // Set the color of the segment
            switch (StreamSource.GetDataStream())
            {
                case 1: GL.Color3(1.0f, 0.0f, 0.0f); break;
                case 2: GL.Color3(0.5f, 1.0f, 0.3f); break;
                case 3: GL.Color3(0.0f, 0.0f, 1.0f); break;
                case 4: GL.Color3(1.0f, 0.3f, 0.5f); break;
                case 5: GL.Color3(0.0f, 0.8f, 0.3f); break;
                case 6: GL.Color3(0.0f, 1.0f, 1.0f); break;
                case 7: GL.Color3(0.0f, 0.5f, 0.9f); break;
            }
 
            GL.Begin(BeginMode.Lines);
            GL.Vertex3(Src.GetX(), Src.GetY(), Src.GetZ());
            GL.Vertex3(Dst.GetX(), Dst.GetY(), Dst.GetZ());
            GL.End();
        }
 
 
        // This function will later be for drawing spherical verteces
        private void DrawVertex(Vertex vertex, float fSize)
        {
            switch (vertex.IsSelected())
            {
                case true: GL.Color3(1.0f, 1.0f, 1.0f); break;
                case false: GL.Color3(1.0f, 1.0f, 0.0f); break;
            }
 
            GL.PointSize(fSize);
            GL.Begin(BeginMode.Points);
            GL.Vertex3(vertex.GetX(), vertex.GetY(), vertex.GetZ());
            GL.End();
        }
 
 
        // Zoom:
        //
        // This function modifies the scaling of the mesh model object to make it appear to "zoom-in"
        //
        public void Zoom(float fZoomRatio)
        {
            if (fZoomRatio > 0)
                fZoom += 1f;
            else
                fZoom -= 1f;
 
            RenderMeshModel();
        }
 
 
        // Rotate:
        //
        // This function modifies the rotational position of the mesh model object
        // with respect to zoom ratio, implemented later.
        public void Rotate(float fX, float fY, float fZ)
        {
            // add scaler assignments
            fRotateX -= fX / (float)(Math.PI * 2);
            fRotateY += fY / (float)(Math.PI * 2);            
            fRotateZ += fZ;
 
            RenderMeshModel();
        }
 
 
        // Translate:
        //
        // THis function modifies the position of the mesh model object in 2D space, parallel to the viewport plane.
        public void Translate(int iX, int iY, int iZ)
        {
            fTransX -= (float)iX;
            fTransY += (float)iY;
 
            RenderMeshModel();
        }
 
 
        // This function repositions the model in the center of the viewport
        private void CenterModel()
        {
            fTransX = 0f;
            fTransY = 0f;
            Translate(0, 0, 0);
        }
 
 
        // This function handles the user-selection of objects on the viewport
        private void SelectAt(int iCursorX, int iCursorY)
        {
 
            int[] View = new int[4];
            int[] Buff = new int[BUFFER_SIZE];
            int n, hits;
 
            // sets the buffer to where the values will be stored for the selection data
            GL.SelectBuffer(BUFFER_SIZE, Buff);
 
            // Retrieve info about viewport
            GL.GetInteger(GetPName.Viewport, View);
 
            // Switch to selection mode
            GL.RenderMode(RenderingMode.Select);
 
            // Clear the names stack
            GL.InitNames();
 
            // Fill the names stack with one arbitrary element
            GL.PushName(0);
 
            // Modify the viewing volume, restricting selection area around the cursor
            GL.MatrixMode(MatrixMode.Projection);
            GL.PushMatrix();
            GL.LoadIdentity();
 
            // Restrict the draw to an area around the cursor
            Glu.PickMatrix(iCursorX, iCursorY, 1.0, 1.0, View);
            Glu.Perspective(fZoom, 1.0, 1.0, 2048);
 
            // Draw the objects onto the screen
            GL.MatrixMode(MatrixMode.Modelview);
 
            // Draw only the names in the stack, and fill the array
            glControl1.SwapBuffers();
            RenderMeshModel();
 
            // Restore the old matrix
            GL.MatrixMode(MatrixMode.Projection);
            GL.PopMatrix();
 
            // Get the number of object rendering within the selected area
            hits = GL.RenderMode(RenderingMode.Render);
 
            // Process hits
            for (n = 0; n < hits; n++)
            {
 
                lbxSelection.Items.Add(Buff[n * 4 + 3].ToString());
            }
        }
 
 
        private void MouseDowner(object sender, MouseEventArgs e)
        {
            int iMidX, iMidY;
            float fX, fY;
            MousePoint1.X = e.X;
            MousePoint1.Y = e.Y;
            LastMouseEvent = e;
 
            // calculate the real coordinates, making the center (0,0)
            iMidX = glControl1.Height / 2;
            iMidY = glControl1.Width / 2;
            fX = MousePoint1.X - iMidX;
            fY = MousePoint1.Y - iMidY;
 
            // Right mouse button was clicked
            if (e.Button == MouseButtons.Right)
            {
 
            }
 
            // Left mouse button clicked
            if (e.Button == MouseButtons.Left)
            {
                SelectAt(e.X, e.Y);                
            }
        }
    }
}
djk's picture

Technolithic in your render code it does not look like you are loading names with Gl.PushName(id) for each entity when you are drawing for select.

The only name in your code is the id zero and everything is drawn associated with that name.

djk