# Convert Screen Coordinate to World Coordinate

Hi everybody!
I has been trying to do small project in my Computer Graphics course in my university. I used glUnProject to convert screen coordinate to World Coordinate and my code here:
private Vector3d GetOGLPos(int x, int y)
{
int[] viewport = new int[4];
double[] matrixprojection = new double[16];
double[] matrixmodeview = new double[16];
double WinX, WinY;
float[] WinZ = new float[1];
double PosX, PosY, PosZ;
IntPtr PixelPtr = System.Runtime.InteropServices.Marshal.AllocHGlobal(sizeof(float));

GL.glGetDoublev(GL.GL_PROJECTION_MATRIX, matrixprojection);
GL.glGetDoublev(GL.GL_MODELVIEW_MATRIX, matrixmodeview);
GL.glGetIntegerv(GL.GL_VIEWPORT, viewport);

GL.glEnable(GL.GL_DEPTH_TEST);
GL.glDepthRange(0, 1);
System.Runtime.InteropServices.Marshal.Copy(WinZ, 0, PixelPtr, 1);

WinX = (double)x;
WinY = (double)viewport[3] - (double)y;
GL.glReadPixels(x, (int)WinY, 1, 1, GL.GL_DEPTH_COMPONENT, GL.GL_FLOAT, PixelPtr);

System.Runtime.InteropServices.Marshal.Copy(PixelPtr, WinZ, 0, 1);
System.Runtime.InteropServices.Marshal.FreeHGlobal(PixelPtr);

GL.gluUnProject(WinX, WinY,(double)WinZ[0], matrixmodeview, matrixprojection, viewport,out PosX,out PosY,out PosZ);
//PosZ = (double)WinZ[0];
Vector3d result = new Vector3d(PosX, PosY, PosZ);

return result;
}
but z value is always wrong, it doesn't change . I use arcball for my project.
Thank you

## Comment viewing options

### Re: Convert Screen Coordinate to World Coordinate

Unless you're using the Extension GL_ARB_depth_buffer_float, the depth value is an Integer. Do you get any GL errors?

### Re: Convert Screen Coordinate to World Coordinate

You can run my program - the Universe.exe file and check the wrong x,y,z values.
You can
- Zoom in and out by moving the middle mouse button.
- Rotate by holding down the middle mouse button and moving the mouse
- Translate by holding down Shift key and holding down the middle mouse button and moving the mouse
When I put a box in the screen and move the cursor over the box, the three values -x,y,z go so wrong and weird.
You can add a new box by hitting the Space key and choosing Add>Mesh>cube. A cube will be added to the screen.
I don't know exactly how the gluLookat function effects to these values. My PlotGL function is here:
public void PlotGL(){
try
{
lock (matrixLock)
{
ThisTransformation.get_Renamed(matrix);
}

GL.glClear(GL.GL_COLOR_BUFFER_BIT | GL.GL_DEPTH_BUFFER_BIT); // Clear screen and DepthBuffer
//////////////////////////////////////////////////////////////////////////
GL.gluLookAt(10, 0.0, 20.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0);
//////////////////////////////////////////////////////////////////////////

GL.glPushMatrix(); // NEW: Prepare Dynamic Transform
GL.glMultMatrixf(matrix); // NEW: Apply Dynamic Transform

GL.glPolygonMode(GL.GL_FRONT_AND_BACK, GL.GL_FILL);

#region plot something

uint count =0;
foreach (IDisplayList list in _displayLists)
{
GL.glPushName(count);
list.Call();
GL.glPopName();
if(count >= 4)
GL.glTranslated(1.0, 1.0, 1.0);

count++;
}

#endregion plot something

GL.glPopMatrix(); // NEW: Unapply Dynamic Transform
GL.glFlush(); // Flush the GL Rendering Pipeline
this.Invalidate();

}
catch
{
return;
}

}
I used arcball for three transformations. Will it effect in some way to these values?. I'm waiting for your help.
Do you have any demo (.exe file for this)? Can you give me some source code for this?

### Re: Convert Screen Coordinate to World Coordinate

The following works for me with OpenTK:

```        int[] viewport = new int[4];
double[] modelViewMatrix = new double[16];
double[] projectionMatrix = new double[16];
public override void MouseInput(int mouseX, mouseY)
{
BeginView(); // Setup Model and Projection Matrices

GL.GetInteger(GetPName.Viewport, viewport);
GL.GetDouble(GetPName.ModelviewMatrix, modelViewMatrix);
GL.GetDouble(GetPName.ProjectionMatrix, projectionMatrix);

Vector3 win = new Vector3(mouseX, viewport[3] - mouseY, 0);
object t = win.Z; // Must box the float
GL.ReadPixels(mouseX, viewport[3] - mouseY, 1, 1, PixelFormat.DepthComponent, PixelType.Float, t);
win.Z = (float)t; // Unbox the float

Vector3 worldPosition;
Glu.UnProject(win, modelViewMatrix, projectionMatrix, viewport, out worldPosition);

EndView();
}```

### Re: Convert Screen Coordinate to World Coordinate

You need to call GL.glEnable(GL.GL_DEPTH_TEST); in or before the PlotGL() function and you don't need it in your GetOGLPos() function.
That way the PlotGL will put values in the depth buffer for the GetOGLPos to read.

### Re: Convert Screen Coordinate to World Coordinate

Thanks for the update, it makes more sense than my brute force attempt. :)
I've just used OpenTK for my project. I feel good to use it.
JTalton, can you show me the function BeginView() and EndView()? The x,y,z values are still wrong.

### Re: Convert Screen Coordinate to World Coordinate

```void BeginView()
{

GL.MatrixMode(MatrixMode.Projection);
GL.PushMatrix();
GL.Ortho(-100, 100, -100, 100, -50.0, 50.0);

GL.MatrixMode(MatrixMode.Modelview);
GL.PushMatrix();
GL.Rotate(Environment.TickCount / 100.0f, 1, 1, 1);  // Rotate the view to show the unproject works with any view
}

void EndView()
{
GL.MatrixMode(MatrixMode.Projection);
GL.PopMatrix();

GL.MatrixMode(MatrixMode.Modelview);
GL.PopMatrix();

GL.PopClientAttrib();
GL.PopAttrib();
}

List<Vector3> pointList = new List<Vector3>();

public override void Render()
{
BeginView();

GL.Enable(EnableCap.DepthTest);

GL.Color3(Color.Gray);
{
GL.Vertex3(50, -50, 0);
GL.Vertex3(50, 50, 0);
GL.Vertex3(-50, 50, 0);
GL.Vertex3(-50, -50, 0);
}
GL.End();

// Draw the points we have hit
GL.Color3(Color.Blue);
GL.PointSize(5);
GL.Begin(BeginMode.Points);
{
foreach (Vector3 point in pointList)
{
GL.Vertex3(point);
}
}
GL.End();

EndView();
}```

### Re: Convert Screen Coordinate to World Coordinate

Thanks JTalton, I'm going to try your three functions.

### Re: Convert Screen Coordinate to World Coordinate

I put BeginView() and EndView() functions in the GetOGLPos(int x,int y) and Render() as you did, then
I tried loading the scene with your Render() function. It worked fine and all three x,y,z values were right . But in my case, I put them in my PlotGL() function. I'm confused about putting the right place for the GL.MultMatrix() in my PlotGL() function. So, everything goes wrong. Can you show me the way to do it?

### Re: Convert Screen Coordinate to World Coordinate

Finally, I got what I want. JTalton, thank you so so much. You're my great savior.

### Re: Convert Screen Coordinate to World Coordinate

Glad it worked! The point of BeginView and EndView was to abstract the setup of the projection and modelview matrices so that they could be used by both the drawing and hit testing functions.