.x loading
Posted Thursday, 21 February, 2008 - 18:32 by Inertia in
I'm currently working on a DirectX .x file loader for the OpenTK.Utilities.dll because of the widespread support for that format in modelling apps.
My question is now, does anyone has an elegant solution how to evaluate template structs inside the .x file like this one:
you can safely ignore the UUID bit (and don't worry about the null-terminated string or matrix order either), but they are creating a new struct with these templates and use this further down in the file like this:
Furthermore you are allowed to do this:
and then use MySkinWeight as a reference somewhere else in the model.
The template SkinWeights is just an example, templates can have an arbitrary number of members (which can be arrays aswell)
Won't cite any of my approach as it will be ugly to maintain, and hopefully that also increases chances to get some fresh ideas.
template SkinWeights {
<6f0d123b-bad2-4167-a0d0-80224f25fabb>
STRING transformNodeName;
DWORD nWeights;
array DWORD vertexIndices[nWeights];
array FLOAT weights[nWeights];
Matrix4x4 matrixOffset;
}
<6f0d123b-bad2-4167-a0d0-80224f25fabb>
STRING transformNodeName;
DWORD nWeights;
array DWORD vertexIndices[nWeights];
array FLOAT weights[nWeights];
Matrix4x4 matrixOffset;
}
SkinWeights {
"Bip01_Spine";
962;
0,
.....alot more values.....
7139;
0.979254,
.....alot more values.....
0.458862;
-0.093409,-0.000001,0.995628,0.000000,-0.000000,1.000000,0.000001,0.000000,-0.995628,...;;
}
"Bip01_Spine";
962;
0,
.....alot more values.....
7139;
0.979254,
.....alot more values.....
0.458862;
-0.093409,-0.000001,0.995628,0.000000,-0.000000,1.000000,0.000001,0.000000,-0.995628,...;;
}
SkinWeights MySkinWeight {
"Bip01_Spine";
962;
0,
.....alot more values.....
7139;
0.979254,
.....alot more values.....
0.458862;
-0.093409,-0.000001,0.995628,0.000000,-0.000000,1.000000,0.000001,0.000000,-0.995628,...;;
}
"Bip01_Spine";
962;
0,
.....alot more values.....
7139;
0.979254,
.....alot more values.....
0.458862;
-0.093409,-0.000001,0.995628,0.000000,-0.000000,1.000000,0.000001,0.000000,-0.995628,...;;
}
Edit: trimmed a few values, they were breaking the layout.




Comments
Feb 22
16:36:28Re: .x loading
posted by kanatoYou could use the stuff in System.Reflection.Emit to create an assembly at runtime that parses the values (or maybe a CodeDomCompiler). I guess the XmlSerializer actually does this sort of thing for performance reasons, so I imagine it would performan well here. I don't really know how easy it would be to do this sort of thing though.
Feb 24
13:34:06Re: .x loading
posted by the FiddlerIf I understand correctly, you need to be able to create a new struct at runtime, according to the SkinWeights template? In this case, how flexible can templates be? Is there a range of possible templates, or can the modeling program define templates in any way it would like?
If it is the first, it might be possible to write the necessary structs (or more probably, classes) ahead of time. Otherwise, I can think of two possible solutions:
What is your approach?
Feb 24
15:25:55Re: .x loading
posted by Inertia"can the modeling program define templates in any way it would like?"
Yes, exporters can declare arbitrary templates. The only restriction I've seen so far is that the template must be abstract (declaration only, no initialization of values). There seems to be no restriction of member count, but probably there's a minimum of 1 required. You cannot declare new types with it though, only a selected number of integer/floats.
There's a number of default templates used to store mesh, animation, etc. which I had just hardcoded, but when throwing some test-models at it the problem with custom templates arose. I've started implementing what you mention as 2), but like I said in the initial post this will be a nightmare to maintain/debug for people who are not familiar with the specs and bit&byte pushing, and is very error-prone in general. I'm mostly looking for options that make sense here, the .Net framework is just too big to know every single class and hopefully someone has dealt with this problem before.
I've started reading about Reflection.emit, but current events keep me too occupied to give an educated answer if this actually meets the requirements. Will look into this more closely and write a prototype soon.
Feb 24
15:40:49Re: .x loading
posted by the FiddlerSystem.Reflection.Emit.TypeBuilder looks promising.
I don't like the fact that it requires creating a new Assembly - I don't know if there is an easier way to build types.
Feb 24
19:12:18Re: .x loading
posted by InertiaYeah, that's what looked bad at the first glimpse at it and I haven't read enough about it to see if that can be avoided. There must be a cheaper way to achieve the same.
Another approach I had considered was using boxing to abstract which types are actually used in the template, and use a class instead of a struct. It would then internaly have 10 (or so) Object fields which are used to reference the boxed data. Feels to me more like a hack than a proper solution though.
Feb 24
20:12:00Re: .x loading
posted by the FiddlerInstead of that, you could write a generic class with ~10 types. Still a hack, but you'd avoid the object fields.
Note, it is very simple to create a generic class at runtime:
(if I remember the syntax correctly)
Aren't there any open source loaders? We could lift some ideas from C++ code.
Feb 24
22:00:07Re: .x loading
posted by Inertiawell there's one for XNA which could probably be ported to OpenTK http://www.codeplex.com/animationcomponents
where's the fun/challenge in that though? :P
Feb 24
22:24:00Re: .x loading
posted by the FiddlerMIT/X11 compatible license, too. Interesting find!
Feb 25
13:57:00Re: .x loading
posted by InertiaXNA has a couple of other questions answered where I'm trying to figure out a more "lightweight" solution for. E.g. the Vertex[] discussed in the other model loader topic, is solved in XNA with these classes iirc:
http://msdn2.microsoft.com/en-us/library/microsoft.xna.framework.graphic...
http://msdn2.microsoft.com/en-us/library/microsoft.xna.framework.graphic...
Pro/Contra regarding usage of XNA approaches:
Pro:
-the microsoft engineers probably put quite a bit of thought into their APIs before locking them down.
-tested, proven to work.
-the animation library does not only load .x, it also loads .bvh .asf .amc, which I believe are used in commercial motion capture environments (never had my hands on that equipment so cannot say for sure)
-major game development companies start to use .Net languages to code their tools, editors & Co. It's much quicker writing importers|optimizers in the robust .Net environment, and it doesn't exclude writing the "engine" in C++
-does not only provide loaders, but also means to playback the animations.
Con:
-Microsoft has designed XNA upon DirectX, with at least 1 eye on the XBox compatibility.
-Their "content pipeline" approach has either to be rebuilt, or replaced. Both is a considerable amount of work, which may actually exceed the effort required to cook your own.
-Other people might have done the first 90% of the work, but you may end up doing the 2nd (difficult) 90% of the library and still they do have the credit for doing the easy bits. The animation library's last update was 1 year ago, and it has 46 unresolved issues last time I checked.
-I've taken a look at their .x loader, and it will reject custom templates like:
<UUID junk>
array VECTOR Tangent[nVertexCount];
array VECTOR Bitangent[nVertexCount];
}
Although the list of cons is longer, the pros do weight quite a bit. It'd probably be quite easy porting the .ms3d loader to feed the XNA vertex declarations, as the file formats are basically achieving the same. This applies for (possible, not planned) .fbx .md3 .md5 .dae (collada) loaders aswell.
Also a unified playback solution would be kinda nice, at least as a starting point.
Feb 25
14:55:06Re: .x loading
posted by the FiddlerThanks for the links, but there's somethign I don't understand: how is VertexElement supposed to be used along e.g. VertexPositionNormalTexture?
I haven't studied the links too much (and unfortunately I don't have the time till tomorrow evening), so I'll have to chew on the rest a bit more before replying. :)
Feb 25
15:39:00Re: .x loading
posted by InertiaSomething like this:
VertexDeclaration bla = new VertexDeclaration( GraphicsDevice, VertexPositionNormalTexture.VertexElements );you may derive from some Vertex base class to create custom layouts, have never bothered with it as the default declarations give quite good choices.
P.S: should we move this into pms? People might get confused and switch to XNA :p
Feb 27
13:17:57Re: .x loading
posted by InertiaAfter giving this some more thought, I believe it's a bad idea porting any XNA related stuff. When relying on XNA it would most likely be the best solution to simply write a Windows XNA app that acts as a model exporter. Binary dump all data from a model loaded with XNA into your custom file format (there really is no way around that, if you want a fast-loading and supporting-all-your-requirements model format) and write an importer for OpenGL. Writing an exporter ontop of XNA also brings additional benefits, such as support for any file formats XNA can load - the animation library does not support them all.
This will be problematic with MacOS|Linux, but the amount of work required to port the XNA stuff is huge, and even I am not eccentric and masochistic enough to do this. Writing custom loaders for .dds made sense, since this format is very important to 3D rendering (and relatively easy to load, compared to models), but porting this animation package is way beyond what utilities.dll needs to provide imho.
Another approach could be using XNA as a foundation and override the DirectX related methods to use OpenTK functions instead.
Feb 27
14:32:18Re: .x loading
posted by objarnimy 2 cc: Building a "custom format exporter XNA app" seems sane ... then you can extract whatever information you think is important from any file formats supported by XNA
Feb 27
15:41:45Re: .x loading
posted by Inertia"Learn another API" might sound bad at first, but XNA is very easy to get into, and documentation through MSDN is good. You can get a (static) model loaded in less than 1 hour (without prior XNA knowledge) if you take a look at the examples that come with it.
The restriction that XNA will never evolve beyond DirectX9 is not a problem for such an exporter, and you can also chose how you want the mesh saved (serialize? binary? ascii? split skin&skeleton from animation and save in separate files? etc.). Those decisions are hard to make for OpenTK.Utilities.dll, because it's guaranteed that they will not satisfy everyone's needs.
I'm not saying OpenTK should not have any model loaders, but it should be carefully considered what makes sense here.
Feb 27
21:07:05Re: .x loading
posted by objarniInertia: I agree about model loaders -- they will be a "moving target" all the time as the technology evolves very fast in this area. On top of that, the data stored in real-time models are, I hazard, quite custom game-per-game. But some loaders for arcane, simplistic formats would be nice for indie game development.
I'm sure there are lots of such simple formats with tons of models available on the net. Quake model format? Q2 models? Unreal models ? etc. etc. I'm not really knowledgeable in this area, I must admit... Convex polygons with single, packed textures + texture coordinates and normals would go far..
Feb 27
22:33:36Re: .x loading
posted by InertiaIndeed, the problem starts at "how many bones may influence a vertex at max.?", goes to "store vertex weights in a byte? [0..100] or [0..255]? use float [0f..1f]?" and can also be pushed to include a GI coefficient per vertex. Some people might want to store only the Normal vector for a vertex, some might want Normal & Tangent and derive the Bitangents via crossproducts, some might want to store all 3 vectors - just to give some concrete examples.
This is why I had already said in the other model-loader topic that the ms3d loader is just middleware to get the model from a modelling application into C# and not something one would want to use in a shipping game. Maybe add the ms3d loader to TKU.dll (my abbreviation of OpenTK.Utilities.dll from now on :P) as a sample how to write a loader with some custom optimizations, but atm I'm not certain anymore that I should finish the .x loader. There's 2 different .x loaders for XNA one could use (1 from the animation component library, 1 from XNA itself) which are debugged and available already.