Soundbomber's picture

Cut outs

I am trying to acheive the appearance of cut outs or holes in solid objects. Can anyone tell me a good way to approach this? I have considered using alpha blending but without much success.
I would like my "cut-out object" to be transparent when in front of my "solid object".
Where the "cut-out object" intersects my "solid object", I would like the intersection to be visible.
Is this acheivable?


Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
Inertia's picture

Indeed, doing CSG - even with limitations like convex and closed objects only - is very hard. I've given this some thought whether there are more options, and I believe you can fake the appearance of CSG purely by drawing operations and GL state.

I've used the wikipedia pictures as example (cube and sphere) and doing the union between the objects is dead-easy. Enable Depth test and simply draw both. Noone will ever know if the surface is opaque. Doing a substraction is harder, it involves using the stencil and depth buffer, draw the cube first. Then reverse the depth test direction, set stencil comparison to zpass and cull front faces, then draw the sphere. The backfaces of the sphere should 'carve' from the previously drawn cube to give a result like the 2nd picture. Intersection should be doable with correct setup of depth and stencil aswell.

However this fake has limitations:
1. Extremely fillrate hungry, but it will be better for interactive manipulation by a user (as opposed to merging 2 VBO into 1 VBO)
2. the number of possible boolean operations is limited by the available stencil planes.
3. You cannot save the final object, you got to save all operand's geometry and some form of tree in which the boolean ops are represented.
4. Bad to integrate with either shadow mapping or shadow volumes. Shadow volumes probably impossible because you have no knowledge about edges.
5. I just came up with this today, no source code available and it's a rather immature theory :P (do not quote or sue if it does not work)

Edit: search the web ftw:

Soundbomber's picture

Excuse my naiivity but how do I set the stencil comparison to zpass?

the Fiddler's picture

Use GL.StencilOp(...) edit and GL.StencilFunc(...).

Old reference (html):

New reference (pdf):

Inertia's picture

What I meant was only update the stencil (when drawing the sphere) if both the stencil and depth test pass and the fragment is updated. If either stencil test or depth test fails don't modify anything.

Soundbomber's picture

Ok, thanks again for your time, but I have another novice question. What initial values will be in the stencil buffer when I perform the first stencil test? Is it just 1 for a 'rendered' pixel and 0 for a clear pixel? Presumeably I need to know this to perform the test against the reference value in StencilFunc?

Soundbomber's picture

As a test I have just applied GL_NEVER as the function before drawing my sphere so that the stencil test always fails. The sphere is still rendered. Is this what should be happenening? I thought that when the stencil test failed, the pixel wasn't rendered!

the Fiddler's picture

(Misread your comment.)

Soundbomber's picture

I am just saying that because I am not getting any expected behaviour, maybe because I am a novice, and no way to test my results, that as a benchmark I decided to implement a situation where I knew what the results should be (or so I thought). I am starting to think that this is just too painful, I am getting nowhere.

the Fiddler's picture

From my experience, it's always frustrating when you first approach a new concept. However, there's a tipping point, after which you start getting results rapidly and it's quite satisfying.

I had a similar experience with relief mapping: I spent a week getting nothing spectacular failures and I was close to quitting (it's not worth it) before I got the first acceptable result (turns out I had simply swapped the Y and Z vectors).

Here is a tutorial about the stencil buffer. As far as I can see, here are the necessary steps:

  1. Request a GraphicsMode with a stencil buffer.
  2. Enable stencil tests (with GL.Enable).
  3. Clear the stencil buffer (GL.Clear).
  4. Draw into the stencil buffer (GL.StencilFunc to Always and GL.StencilOp to Replace).
  5. Draw normally (again, use GL.StencilFunc to [Less|Greater|Equal|etc] and GL.StencilOp to Keep).

Edit: You can also post your code for someone to take a look. It's always easier to find problems with two sets of eyes. I'll add a stencil buffer sample to Examples.exe to make things easier in the future.

Soundbomber's picture

Ok, I will persevere for a while longer, but the more I think about it, the more unsure I am about weather stenciling can be used effectively for a 3D shape. Thanks again.