AndyKorth's picture

Matrix orientation question when transforming vector by MVP

So I guess I spent a day or so confusing myself with matrix math until I figured out that being row major had some consequences I didn't understand. So I found and did some reading.

However, there are a few things that confuse me:
If OpenGL is column major and OpenTK is row major, why isn't data transposed as it's sent through GL.LoadMatrix?
Or... row vs. column is just an internal representation, then why are Matrix4.Transpose, Matrix4.CreatePerspectiveFieldOfView, etc. transposed from what I'm used to seeing?
It must be more than just how it's internally represented if I need to flip the order of my multiplies.

But on to some code:

I'm attempting to transform a vector by the inverted MVP matrix to get the world space coordinates of a mouse click. I've done this sort of thing a bazillion times but now my brain is fried and I don't quite know what's going on.

projection = Matrix4.CreatePerspectiveFieldOfView ((float)Math.PI / 4, Width / (float)Height, 3.0f, 600.0f);
modelview = Matrix4.LookAt (cameraPos, cameraPos + new Vector3 (0f, -1f, 0f), -Vector3.UnitZ);
	public void HandleButtonDown (object sender, MouseButtonEventArgs e)
		float normalizedX = ((float) e.X / ClientRectangle.Width) * 2.0f - 1.0f; // (-1...1)
		float normalizedY = ((float) e.Y / ClientRectangle.Height) * 2.0f - 1.0f;
		// Make debugging simpler by assuming the click was in the middle of the screen.
		// normalizedX = 0f;
		// normalizedY = 0f;
		Matrix4 a = Matrix4.Mult(projection, modelview);
		Matrix4 ai = Matrix4.Invert(a);
		Vector4 p =  Vector4.Transform(new Vector4(normalizedX, normalizedY, 0.5f, 1f), ai);
		// perspective divide.
		Vector4 pos = p / p.W;
		Debug.Log ("mouse down at: " + pos + " from mouse coordinates: " + normalizedX + ", " + normalizedY);

I'm expecting world coordinates in pos, but I get an assortment of strange numbers. I *think* I've got the projection * modelview in the correct order, but now I've switched it so many times I'm not sure anymore. I'm not sure if Vector4.Transform is doing what I expect anymore.

I've entered each matrix into my interactive ruby shell to test them, and I get different results, but I guess that's just due to different major-ness?

Any thoughts would be appreciated.