Inertia's picture

.x loading

I'm currently working on a DirectX .x file loader for the OpenTK.Utilities.dll because of the widespread support for that format in modelling apps. My question is now, does anyone has an elegant solution how to evaluate template structs inside the .x file like this one:
template SkinWeights {
 <6f0d123b-bad2-4167-a0d0-80224f25fabb>
 STRING transformNodeName;
 DWORD nWeights;
 array DWORD vertexIndices[nWeights];
 array FLOAT weights[nWeights];
 Matrix4x4 matrixOffset;
}
you can safely ignore the UUID bit (and don't worry about the null-terminated string or matrix order either), but they are creating a new struct with these templates and use this further down in the file like this:
 SkinWeights {
    "Bip01_Spine";
    962;
    0,
.....alot more values.....
    7139;
    0.979254,
.....alot more values.....
    0.458862;
    -0.093409,-0.000001,0.995628,0.000000,-0.000000,1.000000,0.000001,0.000000,-0.995628,...;;
   }
Furthermore you are allowed to do this:
SkinWeights MySkinWeight {
"Bip01_Spine";
    962;
    0,
.....alot more values.....
    7139;
    0.979254,
.....alot more values.....
    0.458862;
    -0.093409,-0.000001,0.995628,0.000000,-0.000000,1.000000,0.000001,0.000000,-0.995628,...;;
}
and then use MySkinWeight as a reference somewhere else in the model. The template SkinWeights is just an example, templates can have an arbitrary number of members (which can be arrays aswell) Won't cite any of my approach as it will be ugly to maintain, and hopefully that also increases chances to get some fresh ideas.

Edit: trimmed a few values, they were breaking the layout.


Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
kanato's picture

You could use the stuff in System.Reflection.Emit to create an assembly at runtime that parses the values (or maybe a CodeDomCompiler). I guess the XmlSerializer actually does this sort of thing for performance reasons, so I imagine it would performan well here. I don't really know how easy it would be to do this sort of thing though.

the Fiddler's picture

If I understand correctly, you need to be able to create a new struct at runtime, according to the SkinWeights template? In this case, how flexible can templates be? Is there a range of possible templates, or can the modeling program define templates in any way it would like?

If it is the first, it might be possible to write the necessary structs (or more probably, classes) ahead of time. Otherwise, I can think of two possible solutions:

  1. As mentioned by kanato, user Reflection.Emit to generate code at runtime. At a high level, you'd have a factory method that takes a list of types (in this case: String, Int32, Int32[], Single[], Matrix4) and it returns a new type which would be a struct containing these.
  2. The other would be to allocate an unmanaged (or even managed) blob of memory, and load the necessary information into that. Since you know the struct layout, you'll be able to use/upload the data into OpenGL, but I don't know how well this would work in practice.

What is your approach?

Inertia's picture

"can the modeling program define templates in any way it would like?"

Yes, exporters can declare arbitrary templates. The only restriction I've seen so far is that the template must be abstract (declaration only, no initialization of values). There seems to be no restriction of member count, but probably there's a minimum of 1 required. You cannot declare new types with it though, only a selected number of integer/floats.

There's a number of default templates used to store mesh, animation, etc. which I had just hardcoded, but when throwing some test-models at it the problem with custom templates arose. I've started implementing what you mention as 2), but like I said in the initial post this will be a nightmare to maintain/debug for people who are not familiar with the specs and bit&byte pushing, and is very error-prone in general. I'm mostly looking for options that make sense here, the .Net framework is just too big to know every single class and hopefully someone has dealt with this problem before.

I've started reading about Reflection.emit, but current events keep me too occupied to give an educated answer if this actually meets the requirements. Will look into this more closely and write a prototype soon.

the Fiddler's picture

System.Reflection.Emit.TypeBuilder looks promising.

I don't like the fact that it requires creating a new Assembly - I don't know if there is an easier way to build types.

Inertia's picture

Yeah, that's what looked bad at the first glimpse at it and I haven't read enough about it to see if that can be avoided. There must be a cheaper way to achieve the same.

Another approach I had considered was using boxing to abstract which types are actually used in the template, and use a class instead of a struct. It would then internaly have 10 (or so) Object fields which are used to reference the boxed data. Feels to me more like a hack than a proper solution though.

the Fiddler's picture

Instead of that, you could write a generic class with ~10 types. Still a hack, but you'd avoid the object fields.

Note, it is very simple to create a generic class at runtime:

typeof(List<>).MakeGenericType(typeof(int));

(if I remember the syntax correctly)

Aren't there any open source loaders? We could lift some ideas from C++ code.

Inertia's picture

well there's one for XNA which could probably be ported to OpenTK http://www.codeplex.com/animationcomponents

where's the fun/challenge in that though? :P

the Fiddler's picture

MIT/X11 compatible license, too. Interesting find!

Inertia's picture

XNA has a couple of other questions answered where I'm trying to figure out a more "lightweight" solution for. E.g. the Vertex[] discussed in the other model loader topic, is solved in XNA with these classes iirc:

http://msdn2.microsoft.com/en-us/library/microsoft.xna.framework.graphic...
http://msdn2.microsoft.com/en-us/library/microsoft.xna.framework.graphic...

Pro/Contra regarding usage of XNA approaches:

Pro:
-the microsoft engineers probably put quite a bit of thought into their APIs before locking them down.
-tested, proven to work.
-the animation library does not only load .x, it also loads .bvh .asf .amc, which I believe are used in commercial motion capture environments (never had my hands on that equipment so cannot say for sure)
-major game development companies start to use .Net languages to code their tools, editors & Co. It's much quicker writing importers|optimizers in the robust .Net environment, and it doesn't exclude writing the "engine" in C++
-does not only provide loaders, but also means to playback the animations.

Con:
-Microsoft has designed XNA upon DirectX, with at least 1 eye on the XBox compatibility.
-Their "content pipeline" approach has either to be rebuilt, or replaced. Both is a considerable amount of work, which may actually exceed the effort required to cook your own.
-Other people might have done the first 90% of the work, but you may end up doing the 2nd (difficult) 90% of the library and still they do have the credit for doing the easy bits. The animation library's last update was 1 year ago, and it has 46 unresolved issues last time I checked.
-I've taken a look at their .x loader, and it will reject custom templates like:

template TangentAndBitangent { // or binormal or whatever you enjoy calling it
  <UUID junk>
  array VECTOR Tangent[nVertexCount];
  array VECTOR Bitangent[nVertexCount];
}

Although the list of cons is longer, the pros do weight quite a bit. It'd probably be quite easy porting the .ms3d loader to feed the XNA vertex declarations, as the file formats are basically achieving the same. This applies for (possible, not planned) .fbx .md3 .md5 .dae (collada) loaders aswell.

Also a unified playback solution would be kinda nice, at least as a starting point.

the Fiddler's picture

Thanks for the links, but there's somethign I don't understand: how is VertexElement supposed to be used along e.g. VertexPositionNormalTexture?

I haven't studied the links too much (and unfortunately I don't have the time till tomorrow evening), so I'll have to chew on the rest a bit more before replying. :)