Polidobj's picture

Can't get the stencil buffer to work

I can't seem to get anything working with the Stencil Buffer. I even tried porting the simple example here:
http://www.opengl.org/resources/code/samples/glut_examples/examples/sten...
from this page:
http://www.opengl.org/resources/code/samples/glut_examples/examples/exam...
But all I get is black.

I've made sure I've defined to use 8 bpp for the stencil buffer in the graphics mode.

I'm using a NVIDIA Quadro FX 570.

Any ideas?


Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
the Fiddler's picture

Try running the "Stencil CSG" sample in Examples->OpenGL. You should see a sphere "carving out" part of the capsule shape. If that works, then this is likely an application bug.

Can't really give any more advice without seeing some code but you can use the source code for the Stencil CSG sample as a guide.

Polidobj's picture

The sample works ok.

The relevant pieces of code:

Creating the control:

            GraphicsMode gm = GraphicsMode.Default;
            var graphicsmode = new GraphicsMode(
                gm.ColorFormat,
                gm.Depth,
                8, //gm.Stencil,
                gm.Samples, // 4 // anti-alias
                gm.AccumulatorFormat,
                gm.Buffers,
                gm.Stereo);
            GlControl = new GLControl(graphicsmode);

In the load event for the control:

GL.ClearColor(0.0f, 0.0f, 0.0f, 0.0f);
            GL.ClearStencil(0);
            GL.StencilMask(1);
            GL.Enable(EnableCap.StencilTest);

When Drawing:

            GL.MatrixMode(MatrixMode.Projection);
            GL.LoadIdentity();
            int[] viewport = new int[4];
            GL.GetInteger(GetPName.Viewport, viewport);
 
            GL.Ortho(0, viewport[2], viewport[3], 0, 0, 1);
 
            GL.MatrixMode(MatrixMode.Modelview);
            GL.LoadIdentity();
 
           GL.Clear(ClearBufferMask.ColorBufferBit | ClearBufferMask.StencilBufferBit);
 
            GL.StencilFunc(StencilFunction.Always, 1, 1);
            GL.StencilOp(StencilOp.Keep, StencilOp.Keep, StencilOp.Replace);
 
            /* red triangle */
            GL.Color3(200, 0, 0);
            GL.Begin(BeginMode.Polygon);
            GL.Vertex3(-40, -40, 0);
            GL.Vertex3(40, -40, 0);
            GL.Vertex3(0, 40, 0);
            GL.End();
 
 
            GL.StencilFunc(StencilFunction.Equal, 1, 1);
            GL.StencilOp(StencilOp.Incr, StencilOp.Keep, StencilOp.Decr);
 
            /* green square */
            GL.Color3(0, 200, 0);
            GL.Begin(BeginMode.Polygon);
            GL.Vertex3(30, 30, 0);
            GL.Vertex3(-30, 30, 0);
            GL.Vertex3(-30, -30, 0);
            GL.Vertex3(30, -30, 0);
            GL.End();
 
            GL.StencilFunc(StencilFunction.Equal, 1, 1);
            GL.StencilOp(StencilOp.Keep, StencilOp.Keep, StencilOp.Keep);
 
            /* blue square */
            GL.Color3(0, 0, 200);
            GL.Begin(BeginMode.Polygon);
            GL.Vertex3(30, 30, 0);
            GL.Vertex3(-30, 30, 0);
            GL.Vertex3(-30, -30, 0);
            GL.Vertex3(30, -30, 0);
            GL.End();

On resize:

            GlControl.MakeCurrent();
            int w = GlControl.Width;
            int h = GlControl.Height;
            GL.MatrixMode(MatrixMode.Projection);
            GL.LoadIdentity();
            GL.Viewport(0, 0, w, h);
Polidobj's picture

Friday afternoons suck for getting work done.

With some fresh eyes I found my problems:

I'm using my call to Ortho instead of the what's in the example:
GL.Ortho(-5.0, 5.0, -5.0, 5.0, -5.0, 5.0);

And I missed a problem with my porting of the code.
 glColor3ub(0, 0, 200);
I needed to change the 200. So the call should become:
GL.Color3(0, 0, 0.784);

Now to tinker some more with how this thing works.