Inertia's picture

Interface for 3D Models?

Hello again,

i've been wondering if there are any plans for an Interface for 3D Models in OpenTK? I've written a .MS3D Parser (www.milkshape3d.com) that can read meshes/materials/joints from the file and i'd like to make it available to other Tao/OpenTK users, but i'm currently a bit uncertain what to ditch of the parsed data, because the possible uses for a mesh loader can range from an editor application (that wants as much data and comments as possible) to an application that needs nothing besides the vertex positions.


Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
the Fiddler.'s picture

It is more like an example (and a test) of what can be done. My plan was to evolve this according to the needs of the example applications, rather than try to provided a fully-fledged implementation. There is appeal in such an interface however, so if a more complete and flexible implementation arises I would like to add it in and adopt the OpenTK examples to it. It probably wouldn't go into the core dll (that would be too limiting), but rather into the (tentatively named) OpenTK.Utilities.dll as per this discussion.

There are two interesting design issues here:
a) Flexibility, as in different vertex formats, 16 vs 32bit indices (I'm looking at this from a performance standpoint).
b) Interoperability with both GL2 and GL3. This shouldn't be too hard to achieve as long as the interface doesn't make any assumptions for the underlying platform (and doesn't store platform-specific rendering-related hints).

I don't know what is the scope of the interface you have in mind, but there is appeal in something like this. If a suitable implementation arises I would like to add it into the (as-yet-inexistent) OpenTK.Utilities.dll and adapt the examples to it.

Inertia's picture

The scope is solely providing an abstract definition of a Mesh, the above mentioned functions can operate on. Note that this class only specifies a Mesh, not a Model.

Guess GL3 leaves no choice but VBO, so it makes sense to include this into the definition, but it's just an assumption that VBO will largely stay the same. Not all members of the class are necessary, some may be missing. The idea is to describe only the mesh itself, so it can be copied into a VBO and remember the necessary states for drawing.

The suggestion is based on accessibility of the vertex list, while the array sent to Gl.BufferData() is already created. This can be fed to GL.Interleaved or have Pointers set manually with strides.

struct Vertex
        { // GL_T2F_N3F_V3F style layout
            public Vector2 UV;
            public Vector3 Normal;
            public Vector3 Position;
        }
Vertex[] Verices;

The typical use of this mesh class would be either generating procedural geometry or importing from disk, which can then be edited/noise applied/scaled/optimized or directly sent into the VBO. If the user wants to write his own separate management and solely take VBO/IBO handles from an object that should be fine aswell.

Edit: I'm kinda stuck with this interface though and don't think I can complete this on my own. Some design questions aren't easy to solve (namely the Vertex-struct, whether the Material should be included aswell and if it'd be desireable to serialize a Mesh instead of a custom file format for storing it).

Edit2: Another approach, compacted but split into 3 interfaces (DrawArrays, DrawElements 16Bit Indices and 32Bit). Doesn't solve the biggest Problem (struct Vertex) tho. It would have been really handy to have this kind of interface available when writing the Torusknot generator, it could have been easily fed into these layouts.

fully abstract class iTopologyNoIndex
    {
        struct Vertex;
        Vertex[] Vertices;
        BeginMode PrimitiveType; // Points? Tristrip? Quads?
        uint VBO;
    }
 
fully abstract class iTopologyUint16
    {
        struct Vertex;
        Vertex[] Vertices;
        ushort[] Indices;
        byte VertexCacheOpt; // 0 if none, otherwise the Vertex Cache size the mesh is optimized for.
        BeginMode PrimitiveType; // Points? Tristrip? Quads?
        uint VBO, IBO;
    }
 
fully abstract class iTopologyUint32
    {
        struct Vertex;
        Vertex[] Vertices;
        uint[] Indices;
        byte VertexCacheOpt; // 0 if none, otherwise the Vertex Cache size the mesh is optimized for.
        BeginMode PrimitiveType; // Points? Tristrip? Quads?
        uint VBO, IBO;
    }

These 3 could inherit from a base class, namely the BeginMode is inevitable and the Vertex-related definitions.

@Fiddler:
b) I consider this a bare minimum of must be known about the mesh. While it's clear what my requirements are, I'm not certain how to properly abstract this to a lowest denominator for all User's needs.

the Fiddler.'s picture

I've been thinking about this. A bottom-up approach probably suits this topic better, so let's try that.

Vertex struct:
As long as its members have the correct datatypes, and it is declared with the Sequential Layout attribute, anything will work. There's little reason to come up with a fixed layout to suit all users, when custom formats are easy.

[Layout(LayoutKind.Sequential)]
class Vertex { }
 
class VertexT2N3V3 : Vertex
{   // GL_T2F_N3F_V3F style layout
    public Vector2 UV;
    public Vector3 Normal;
    public Vector3 Position;
}
 
class VertexCustom : Vertex
{   // Custom format declared by the user.
    public Vector3 Position;
    public Vector3 Normal;
    public Vector3 Tan;
    public Vector3 BiTan;
}

The DrawArrays/DrawElements functions can be overloaded to accept a Vertex directly (which also allows us to hide the sizeof() logic from the user).

Mesh (Topology):
Provide a generic mesh template and allow the user to inherit from this to create specialized mesh types. There should be two default implementations, one indexed and one non-indexed.

// Non-indexed
class Mesh<VertexType> where VertexType : Vertex
{
    public int VBO { get; protected set; }
    public virtual IList<VertexType> VertexCollection { get; protected set; }
}
 
// Indexed
class Mesh<VertexType, IndexType> : Mesh<VertexType> where VertexType : Vertex where IndexType : IConvertible
{
    public virtual IList<IndexType> IndexCollection { get; protected set; }
    public int IBO { get; protected set; }
 
    // Indexed access to the vertices.
    public VertexType this[int index]
    {
        get { return VertexCollection[IndexCollection[index]]; }
        set { VertexCollection[IndexCollection[[index]] = value; }
    }
}
 
// These can be then used like this:
public void UseMesh()
{
    Mesh<VertexPosition3> mesh = new Mesh<VertexPosition3>();
    Mesh<VertexPosition3, int> mesh_32bit_index = new Mesh<VertexPosition3, int>();
    Mesh<VertexPosition3, short> mesh_16bit_index = new Mesh<VertexPosition3, short>();
}

Obviously, there are a lot of things missing before these classes become usable, but I think this should work (hey, at least it compiles!) I also like the fact that indexed mesh formats have different types than non-indexed ones. What do you think?

Serializable attribute:
I think this would be very beneficial, not as a file format (though this is a possibility), but rather to share data over the network or from one process to another. Nailing the base format down is higher priority though.

Edit: I think it would be better to completely separate the mesh format from its consumers, i.e. the mesh format should not know anything about GL.DrawArrays or GL.DrawElements. This makes the format more flexible, and more future-proof.

Inertia's picture

hey, at least it compiles!
Indeed, and thank you for analyzing the problem. At some point you are so deep in the woods that you cannot see the forrest anymore because of all the trees ;)

Separating the Vertex from the collections is a good move, I just think there must be at least 1 default Vertex for model loader / generators to use. It should not be the generator's responsibility to figure out what attributes it can fill in the given Vertex, but rather fill a default structure. The user could either derive his custom vertex layout from the default used by the loader/generator, or create a whole new and convert between the 2 meshes.
A default vertex layout would also clearly define what attributes a model generator must supply, while Position and Normal are not optional, Color and UV can be very difficult to determine.

Regarding the hiding of DrawArray/Elements there is one problem. At least for the MS3D format, all mesh data in the model is stored to be used with DrawArrays. I'm converting this into an indexed trilist, but this must be somehow transparent to the programmer what kind of data he got. If you don't want 2 mesh formats, maybe a 3rd class MeshState could describe the current mesh, if you dislike the way I used members to track the state.
Same applies for BeginMode, the Torusknot can be generated as a triangle-strip or indexed tri-list (and it would be easy to add the other modes beside GL_TRIANGLE_FAN and GL_POLYGON ). Without the knowledge in what mode we're working you cannot properly draw the mesh. In this spirit, you can even go one step further and include the face winding (CW? CCW?) or Material like I did. This could all be hidden by passing those properties into the load/generate functions instead of tracking them with the mesh, but this would make a common file format for saving to disk rather useless.

At the very least the BeginMode must be known by the draw functions, you can still make assumptions if the mesh is intended for use with DrawArrays by checking the presence/size of the Indices array. The face winding should imho be assumed to be always the OpenGL default, it is very easy to revert the winding manually to CW, if you have an exisiting mesh with CCW faces.

the Fiddler.'s picture

[Default vertex formats]
Agreed, the most common ones like P3_N3_T2 should be provided by default. The model load/generator should probably target these, but then again nothing stops it from defining and exporting it's own vertex format to the its consumers.

[Mesh state]
After going through the PQTorusKnots(*) source to see how these structures are used it started to make a lot more sense.

So at the very least we need to know the type (triangles, quads etc), whether indexing is used (and the indexing type strip/list/fan) and vertex winding. BeginMode is a bit dangerous in that it (probably) is GL2 only, so it might be better to define a separate enum. Regarding the material, what information should that contain?

[Mesh]
What is a mesh? The way I understand it, it is a collection of vertices, indexed in some way, along with a set of attributes and materials. In this light, it makes sense to keep these data as separate as possible, to allow reuse e.g. of materials or indices between meshes. But I admit that my knowledge in this area is quite limited.

[Off-topic]
(*) PQTorusKnots compiles and runs (almost) fine under Mono 1.2.5 (a known issue with texturing), but crashes on Mono 1.2.6 preview 3. I suspect it's an issue with the vertex/index layout in memory, but I haven't been able to nail down the cause. Which is kinda problematic since the release is drawing close...

Inertia's picture

[vertex formats]
This probably should be closer examined what formats are good candidates for such a default, ofcourse this should never limit a consumer to this only format. The idea is just to have a lowest common denominator all GenerateBox, GenerateSphere, etc. functions can fill without any problems.

[indexing]
we already know from Beginmode (or an appropriate enum) which primitive type the mesh consists of, this information is needed no matter if we Draw-Array or -Elements.
Although it's a bit of a hack I start liking the idea to determine whether a mesh is for Draw-Array or -Elements by examining the Indices. You can safely assume that if there exists no Index-Array/IBO the mesh is not for DrawElements.

[face winding]
Like I said before, face winding is probably best solved by setting one of the possibilities as default. It's very easy for the user to convert between windings, but also makes it for programmers easier to write Generate* functions with only one winding in mind. If that winding is OpenGL default, this problem will be completely hidden from beginners.

[Material]
As an example, here the material supplied by MS3D:

public struct Material
    {
        public Vector4 Ambient;
        public Vector4 Diffuse;
        public Vector4 Specular;
        public Vector4 Emission;
        public float Shininess;
 
        public string TextureFileName;
        public string AlphaFileName;
    }

This isn't very usable in this state, but that's what is read from the file.

[Mesh]
Like you said, it's a collection of Vertices indexed by either order of appearance (DrawArrays) or by a separate list (DrawElements).

Just to make this more clear: In my definition, a Model (let's use a human wearing glasses as example) is described like these components:
-3 Meshes: The naked Body (Mesh 1), clothing and the frame of glasses (Mesh 2), the transparent glass (Mesh 3)
-3 Materials: 3 Materials in this case, one to display skin, one to display cloth/plastic and one to display glass.
-Optional: 1 Skeleton: The bind pose for skinning, a huge collection of Keyframes for animation for this skeleton.
-This can be extended now at heart's content, Matrix4 Orientation, Collision/Physics attributes etc.

I don't think sharing indices between meshes is something that will be used alot (it could be interesting for LOD though), but sharing parts of the material is quite desireable (e.g. different meshes might have unique textures but all require the same shader program bound). I haven't given Material/Skeleton abstractions much thought tho, for me a Material is 3 ints for GPU program and 2 Textures returned from Managers.

[OT]
Good to hear that it compiles under mono at all, my first OpenGL program I asked ppl to test didn't run on any ATi cards at all ;)

Inertia's picture

Here's some c&p from other file format's Material specifications. Wavefront's .MTL is very similar to what Milkshape supplies.

Wavefront .MTL (for .OBJ)

Ka <r: float> <g-float> <b-float>
    Defines the ambient color of the material.
    * default is {0.2, 0.2, 0.2}.
 
Kd <r: float> <g-float> <b-float>
    Defines the diffuse color of the material.
    * default is {0.8, 0.8, 0.8}.
 
Ks <r: float> <g-float> <b-float>
    Defines the specular color of the material.
    * default is {1.0,1.0, 1.0}.
 
d <alpha: float>
    Defines the alpha value of the material.
    * default is 1.0
 
Ns <shininess: float>
    Defines the shininess of the material.
    * default is 0.0.
 
map_Ka <filename>
    Defines the filename of a texture map to be used. The file should not be a regular graphics file, but just be an ASCII dump of the RGB values.

3DS Material

   0xAFFF : Material block
    -----------------------
      0xA000 : Material name
 
      0xA010 : Ambient color
      0xA020 : Diffuse color
      0xA030 : Specular color
 
      0xA040 : Shininess percent
      0xA041 : Shininess strength percent
 
      0xA050 : Transparency percent
      0xA052 : Transparency falloff percent
      0xA053 : Reflection blur percent
 
      0xA081 : 2 sided
      0xA083 : Add trans
      0xA084 : Self illum
      0xA085 : Wire frame on
      0xA087 : Wire thickness
      0xA088 : Face map
      0xA08A : In tranc
      0xA08C : Soften
      0xA08E : Wire in units
 
      0xA100 : Render type
 
      0xA240 : Transparency falloff percent present
      0xA250 : Reflection blur percent present
      0xA252 : Bump map present (true percent)
 
      0xA200 : Texture map 1
      0xA33A : Texture map 2
      0xA210 : Opacity map
      0xA230 : Bump map
      0xA33C : Shininess map
      0xA204 : Specular map
      0xA33D : Self illum. map
      0xA220 : Reflection map
      0xA33E : Mask for texture map 1
      0xA340 : Mask for texture map 2
      0xA342 : Mask for opacity map
      0xA344 : Mask for bump map
      0xA346 : Mask for shininess map
      0xA348 : Mask for specular map
      0xA34A : Mask for self illum. map
      0xA34C : Mask for reflection map
 
      Sub-chunks for all maps:
        0xA300 : Mapping filename
        0xA351 : Mapping parameters
        0xA353 : Blur percent
        0xA354 : V scale
        0xA356 : U scale
        0xA358 : U offset
        0xA35A : V offset
        0xA35C : Rotation angle
        0xA360 : RGB Luma/Alpha tint 1
        0xA362 : RGB Luma/Alpha tint 2
        0xA364 : RGB tint R
        0xA366 : RGB tint G
        0xA368 : RGB tint B
the Fiddler.'s picture

The 3ds material format is... interesting. (Mtl is just plain awful - or maybe it was the SGI Performer/CAVElib combination shudders)

Where do shaders fit in the grand scheme of things? Should they be attributes of the material format?

Inertia's picture

The way I see it, all of these material formats are rather useless for GLSL, Texture map filenames being the only usable bit. Actually I find this desireable, because the way a 3D Modelling app renders a Model is never exactly what you get in OpenGL (at least from my experience) and the Material settings must be tweaked anyways. Sure, you could write an app that puts all variables taken from the .3DS format to use, but the Material is clearly for a raytracer and for real-time applications many of these variables are just overweight. On the other hand the MS3D and MTL Materials are clearly for fixed-function lighting, providing a bare minimum.

The the points where the Model loaders/generators and Shaders have contact are:
direct:
-GL.BindAttribLocation and GL.VertexAttrib*
indirect:
-GL.UseProgram (and binding the appropriate Texture maps)
-GL.Uniform*

I really do not see how the model loaders/generators could give any hints what Vertex-/Fragment-programs should be used. It would be also problematic for GenerateSphere&Co to supply any Material at all (besides some pre-set default).
Like I said before, I believe the best solution is a middle-stage where the imported data from disk is converted into your custom formats, there's simply too many different directions one could go for a unified solution.

Edit: The default one could strictly emulate OpenGL 1 fixed-functionality behaviour. All Material-definitions mentioned so far can supply the required values.

I've no idea how this would look like in GL3 though, many built-in variables (gl_FrontMaterial? How about Lights?) will most likely disappear.

The lowest common denominator would be the struct used from GLSL internally to receive uniforms from OpenGL

struct gl_MaterialParameters
{
vec4 emission;
vec4 ambient;
vec4 diffuse;
vec4 specular;
float shininess;
}
uniform gl_MaterialParameters gl_FrontMaterial;

With a few string added, this could also hold Texture filenames too.

The problem with this kind of material is, it's not really useful for real-time rendering because Texture Maps can describe many materials at once, allowing the Model to consist of less Meshes, which really helps batching/instancing. The typical game 3D-object has only 1 Mesh and 1 Material, so you can draw multiple meshes in a row with minimum state changes.

Where does this stop though? For example .OBJ seems to store NURBS information aswell, while .3DS can describe a whole Scene with Lights, Helpers, Keyframes etc.

Inertia's picture

Maybe the approach so far was wrong, it could be much simpler. The .dds loader doesn't return you the image either, it returns you a valid Handle you can bind already, and a little information about the image dimension (Tex2D? Cubemap?). Why shouldn't the model loader/generators behave similar?

sealed class VboMesh
{
 public int VboHandle;
 public int IboHandle; // 0 == none
 public object Vertex&MaterialDeclaration;
 public eType IboType; // byte, ushort, uint, ulong?
}

I didn't really like using a collection/container for the Vertices, if you have the knowledge to write a function that can for example renormalize all normals in the VBO by using face normals, you should also know how to extract the data from the VBO and stuff it back in.

The "object" would need some formating, so you can figure out the offsets/strides to interleave the VBO correctly, but this approach might be much easier to handle for a new user than the old ones.
Any Thoughts?