master2k's picture


i know a well known discussed thematic in any forums, but i'd need specific help,

i got meshboundungs MaxX, MaxY and MaxZ, i want to send a vector through this boundings on click,
from the camera position in direction depth. ( what is something else as world -Z coordinates i believe ;-)
if i click i modify the Z value of the vector , but i get results in world coordinates, i only would need the Vector Translating in the right direction and from the right startpos, i tried following:

where the variable startray traces from (0f-->-500f or 0f-->500f)

Vector3 ray;
if (camerapos.Z > posObjektZ) {
//some transform/conversion missing?
ray = camerapos+new Vector3(-mouseX+ClientRectangle.Width/2,-mouseY+ClientRectangle.Height/2,-startray);
} else {
//some transform/conversion missing?
ray = camerapos+new Vector3(-mouseX+ClientRectangle.Width/2,-mouseY+ClientRectangle.Height/2,+startray);
	if (ray.X < mesh.posX+mesh.MaxX/2*mesh.scalefactor &&
                         ray.X > mesh.posX-mesh.MaxX/2*mesh.scalefactor &&
                         ray.Z < mesh.posZ+mesh.MaxZ/2*mesh.scalefactor &&
                         ray.Z > mesh.posZ-mesh.MaxZ/2*mesh.scalefactor &&
                         ray.Y < mesh.posY+mesh.MaxY/2*mesh.scalefactor &&
                         ray.Y > mesh.posY-mesh.MaxY/2*mesh.scalefactor) {
this.title =;

some transform of the modelviewmatrix/vector is missing, i've seen there are easy ways of doing this in openTK, but i got no result with invert of the modelview, transforming the vector etc.
maybe the 2d mouse coordinates are scaled wrong, or something? i believe 1 or 2 lines of code are missing to get it to work, any help would be great

finally in short words the problem is that i dont get the usual rotation of my camera that means when i rotate my camera the vector still shoots in the same Z direction (and not in the "camera rotated Z direction" . (im using a usual lookat modelview).



Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
ERP's picture

Since I just wrote this code.
To get the ray direction in model space you need to take two points at different depths in screen space and invert the tranform to get back into model space.
This is my current code to generate the ray, x, y is the mouse position, w,h is the viewport width/height.
Perspective is the projection matrix and modelView is the modelView matrix.
The ray you want starts at sr and ends at er, direction would be (er-sr)

// generate an object space ray
// convert the viewport coords to openGL normalized device coords
float xpos = 2 * (x / w) - 1;
float ypos = 2 * (1 - y / h) - 1;

Vector4 startRay = new Vector4(xpos, ypos, -1, 1);
Vector4 endRay = new Vector4(xpos, ypos, 1, 1);

// Reverse Project
Matrix4 trans = modelView * perspective;
startRay = Vector4.Transform(startRay, trans);
endRay = Vector4.Transform(endRay, trans);
Vector3 sr = startRay.Xyz / startRay.W;
Vector3 er = endRay.Xyz / endRay.W;

master2k's picture

hello erp, first thx for your reply it will help in few cases,

but may i tried a different method, i set the startray position to my modelview camera lookat coordinates,
and tried to retrieve the direction vector from the lookat modelview with succuess.. (row0.x = -z, row0.z = -X), but row0.Y != y direction, so the x and z parameters works pixelperfect at any distance i want (endray is defined by me for example +-500f) , so letting out the mouse movement my ray now always shoot in the right x,z direction but in a few cases not in the right y direction, i looked several modelview descriptions but it always told me row0.Y = y, but this doenst work if i rotate my camera along the z axis, but works if i rotate the camera in the x axis.


ERP's picture

I assume from your response that you are using an othographic projection?
Since any perpective will require the solution I provided above because rays would not be in the direction of the camera except at the center of the screen.

The modelView matrix transforms from object space into camera space.
The columns of the matrix represent to output X/Y and Z vectors in the transformed space.

Iy your using the standard OpenGL convention, -Z is into the screen.

Slightly more generally you need to back out all of the transforms to get a ray from the mouse position
First you need to convert the X/Y into normalized device coordinates

float xpos = 2 * (x / w) - 1;
float ypos = 2 * (1 - y / h) - 1;

Then remove the projection - if in your case it's orthogonal it's a simple scale and translate

Then remove the ModelView Transform

At which point you are in model space.

If you have an Orthogonal projection, then the ray direction will be the 3rd column of the model view matrix, but for the position you'll still need to back out any scale applied in the projection and account for y being up.

master2k's picture

isnt the plain modelviewmatrix (without perspective / projection multiplication) not a usual orthogonal matrix?) , since my code is short before working (with mouse) ?


ERP's picture

Sorry I meant orthographic when I typed orthogonal.

Yes, but your mouse position is in viewport coordinates, and your box is in model coordinates.

If the projection matrix is orthographic then compensating for it is a simple scale and translation, which you may be accomplishing in your code with the following.

camerapos+new Vector3(-mouseX+ClientRectangle.Width/2,-mouseY+ClientRectangle.Height/2,-startray);

This piece of code implies an orthographic projection where 1 unit is equivalen to 1 pixel on screen.
If you're following standard OpenGL conventions then X is inverted in the calculation above, but if you can draw a point at the coordinates you get here and it tracks under the mouse then it's fine.

If this is the case, then all you need is the 3rd column of the modelView Matrix as your ray direction.

master2k's picture

hello erp, just my 2d stuff (menus, cursor etc.) are orthographic, my usual modelview/projection is usual perspective projection + lookat modelview.

i tried your code sr is translating to my camera position but is not moving if i move my mouse?, do i have a think error?, since x,y,z should be mousex, mouseY afterwards, but my renderer testobject (which is visible ) with this coordinates is not moving if i move my mouse?

Best regards and Thx


ERP's picture

If you have a perspective matrix applied, pick lines from the mouse position are not parallel to the camera direction, so you need to back out all of the transforms.
Which is what the original code I posted does.

The following link is an explanation of what's involved, it's unfortunately using XNA, I couldn't find a readable OpenGL sample, it uses the unproject function. The Code I provided above basically unprojects two points under the mouse just like the example in the link. I know the code works, because my application uses the resulting ray it to pick triangles on a reasonably complex model.

You can also use the gluUnProject function to do the same thing.

You probably won't be able to visualize the sr by rendering it, because the Z I selected is behind the eye. If you set the z of startRay to some small positive number you should be able to see it.

master2k's picture

hello erp i used this piece of code to get my mouse position mapped to world coords:

float xpos = 2 * (mouseX / ClientRectangle.Width) - 1;
float ypos = 2 * (1 - mouseY / ClientRectangle.Height) - 1;
Vector4 sr =new Vector4(xpos,ypos,20f,1);
Matrix4 trans = modelviewMatrix * projectionMatrix;
Vector3 srTrans = Vector4.Transform(sr,trans).Xyz/sr.W;

my objects is rendered but the mouse is not moving , are the mouseX/w and mouseY/h
the x and h components others coordinates then my width and height of the window?
the transform seems to work correctly because if i look in the components while debuggig it fits nearly my camera coordinates.

best regards,


ERP's picture

20's way to big a number here

Vector4 sr =new Vector4(xpos,ypos,20f,1);
It's in normalized device coordinates, they range from -1 to 1, when I said small positive number I was thinking more like 0.001f

This line is also incorrect
Vector3 srTrans = Vector4.Transform(sr,trans).Xyz/sr.W;

You need to divide by the transformed W

But you have the right understanding of x, y, h and w, you'll have to cast them to floats before you do the divide, otherwise you'll end up with it always returning the center of the screen. Without the cast, the divide between ints will result in an int which will be rounded down to 0.
float xpos = 2 * ((float)mouseX / (float)ClientRectangle.Width) - 1;
float ypos = 2 * (1 - (float)mouseY / (float)ClientRectangle.Height) - 1;

for test rendering I'd just do

Vertex3(sr.x, sr.y, sr.z)

but whatever works.

master2k's picture

hey, ERP i'll thank you for your time it is finally working now,

it was exactly that what you wrote in your last sentence , it rounded it down to always zero or always 1

so i tried the quick dirty code:

float xpos = 2f * (Convert.ToSingle(mouseX) / Convert.ToSingle(ClientRectangle.Width)) - 1f;

and its working now, and that exactly! now i should be able to send my rays in the right direction!

That was the best Help i ever got, i'll keep your name in Mind!

best wishes