Anonymous's picture

bytes are 1 bit offset [Resolved]

Project:The Open Toolkit library
Category:support request

Disclaimer: I'm new to OpenGL/TK, so this could very well be my own fault.

I made a VBO ( a 64x64 square grid ), and in order to do picking, I'm using the vertex colors to determine which grid square I've clicked on. Note: I'm using LINQ for clarity of the concept -- I'm aware of the performance implications.

        public void GeneratePickVBO() {
            var vertices = faces
                // get a list of [face, faceIndex, vertex]
                // each faceindex shows up 4 times, once per vertex in that face
                .SelectMany( (face, i) => 
                        face.vertices.Select(v => new { vertex = v, index = i }))
                // convert to list of [vertex]          
                .Select(v => {
                    return new PickVertex() {
                        RGBA = (uint)v.index,
                        Position = v.vertex.Position
            GL.BindBuffer(BufferTarget.ArrayBuffer, vboPickID[0]);
                (IntPtr)(vertices.Length * Marshal.SizeOf(typeof(PickVertex))),
            pickReady = true;

where pickVertex is shown below (Marshal.SizeOf(typeof(PickVertex)) returns 16, as expected)

    struct PickVertex {
        public UInt32  RGBA;
        public Vector3 Position;

and the rendering code is

                GL.BindBuffer(BufferTarget.ArrayBuffer, mesh.vboPickID[0]);
                GL.InterleavedArrays(InterleavedArrayFormat.C4ubV3f, 0, IntPtr.Zero);
                GL.DrawArrays(BeginMode.Quads, 0, mesh.faces.Count * 4);
                byte[] pixels = new byte[10];
                GL.ReadPixels<byte>(MapZone.X, MapZone.Y, 1, 1, PixelFormat.Rgb, PixelType.Byte, pixels);
                Message.Post("foo", "RenderPick: [{0:X}] [{1:X}] [{2:X}] [{3:x}]", 
                    pixels[0], pixels[1], pixels[2], pixels[3]);

So the following image is produced:

Anyways, the problem is that every 2 adjacent squares has the same red-byte (so the progression is 00,00,01,01,02,02) etc. ditto for when the green byte comes into play. the first two are 00, the second couple are 01, etc.

I checked and I can partially fix the problem with the change, which skips the lowest-order bit of the red component. I'd have to shift a hole in the lowest order-bit of the green and blue as well in order to accomplish a more complete solution:
RGBA = (uint)v.index*2U,



Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
Typhon's picture


Category:bug report» support request

Sorry, wasn't logged in. Anyways, I just opened the screenshot and took samples of the pixels, and they are correctly drawn. So the error comes when I'm ReadPixeling them back out.

Ooops I figured it out. I'm reading them out as PixelFormat.Byte instead of PixelFormat.UnsignedByte. dumb mistake.

Typhon's picture


Title: bytes are 1 bit offset» bytes are 1 bit offset [Resolved]
Priority:normal» minor
Assigned to:Anonymous» Typhon
the Fiddler's picture


Status:open» closed