Inertia's picture

DXT5 Parallax Mapping

This project was conceived as a proof-of-concept last week in the attempt to find an answer for the "geometry displacement vs. parallax mapping" question. Not sure if this is obvious or a stupid idea (never seen this used before in any app) so I'd like to hear some opinions about it.

In order to accomplish typical Parallax mapping, The shader requires these input textures:

RGBA8 Texture 1:

3 Channels: Diffuse Color R, G, B (called Dr, Dg, Db)
1 Channel: Specular Gloss (called Sg)

Order: [Dr][Dg][Db][Sg]

RGBA8 Texture 2:

3 Channels: Normal X, Y, Z (called Nx, Ny, Nz)
1 Channel: Parallax Height (called Ph)

Order: [Nx][Ny][Nz][Ph]

8 Bytes per Texel is quite a bit, so it would be desireable to use DXT5 compression to shrink the memory use. This works nicely for the first Texture, but the second Texture has notable artifacts when compressed. Storing the second Texture as RGBA8 is an obvious solution, but 'id Software' came up with the idea for Doom³ to reorder the second Texture like this so DXT5 can be used:

[E][Ny][E][Nx] (E=Empty, ie. 0)

Since we know the length of a normal, Nz can be calculated per fragment. However this isn't really helping with Parallax mapping, since the Red and Blue channels have quite poor precision when compressed so they cannot store the height properly.

My idea is to order the Channels like this to use DXT5 for both textures:

Texture1: [Dr][Dg][Db][Ph]
Texture2: [Sg][Ny][E][Nx]

This puts the height into the rather high precision Alpha channel of the first texture, and the Red channel from the second Texture appears to be sufficient for the gloss channel from my test textures. The blue channel is currently unused.

Free Image Hosting at www.ImageShack.us

Both screenshots used the same scene setup and shaders, the only difference is that the left one uses 1024kb+256kb .bmp textures, the right one uses 342kb+86kb .dds textures (which include mipmap levels down to 4x4). The diffuse+height texture is 512², the normal+gloss is 256² (image is probably (C)(R)(bla) by Typhoon Labs, it originates from their ShaderDesigner distribution).

The blue channel of the second Texture could be used as a multiplier for cubemap reflection contribution (might be nice to control the mirror strength independently from specularity), or use it as s-texture coordinate to look up a 1D texture (e.g. looking up specular exponent rather than passing that as uniform), or gate a simple if (tex2.b < 0.2) discard; alpha mask.

Any thoughts?

----------------------------------------------------------------------------------------------------------

Update: Added example application. Although the dds loader is included into the package, please discuss it here rather than using this topic. Thanks.

There's alot of room for improvement, don't expect to c&p this code and end up with Doom 5. The light does not consider attenuation and the Parallax effect is just a basic approach.

Controls probably require explanation too:

Mouse movement controls the light position.
Mouse wheel alters the camera's Y axis.
Q/A alter the Parallax' scaling.
W/S alter the Parallax' bias.
E/D alter the Material's shininess.
Space bar prints first GL error, Esc exits.
(See console window for feedback)

Edit: Added Download

AttachmentSize
Parallax.rar783.65 KB

Comments

Inertia's picture

So everyone reading this ran to the patent office seeing if they can make a claim, or where is my feedback? Or is the weather to blame? ?-)

the Fiddler's picture

I think it's the lack of a test app. ;) That, or the weather's too nice for programming :p

From the screenshots, it looks very good. The difference is hardly noticeable (I wouldn't be able to tell which screenshot is which, without the caption). Quite quite cool for the savings in memory.

Have you tested on other surfaces? Could this introduce visible artifacts on surfaces with a more "uniform" gloss map?

Another passing idea, didn't Ati introduce a specific texture format for normals, with the R500 series? Maybe it would be worth looking into that, although I don't know if it can reduce memory consumption more than bundling normal+gloss in a DXT5 texture.

Inertia's picture

Well, I do have a test app, but there's 2 problems why it's not attached to the post. 1) it requires the still-not-commited .dds loader and 2) uses rock and metal textures I have no license for. Shaderdesigner's license is not explicit about using the textures in other projects. Not sure who to contact, since Typhoon Labs is no more.

As far as I know, Ati's texture formats basically contain 2 alpha channels, so there's only 2 channels with similar precision per texel instead of 4 with DXT5. The biggest issue with that format is probably that nvidia cards older than GF8 don't support it. Texture 1 in DXT5 format and Texture 2 in ATI2N format only offers 6 channels to work with, not enough to store RGB, Normal, Height and Gloss. The common approach seems to be to use an uncompressed RGBA8 for the 2nd Texture, or forego either parallax or gloss.

teichgraf's picture

Really cool proof!
Maybe there are some other textures or tools out there to generate / edit such textures procedural, so you can post a license-free app (with source)? When will the DDS loader be commited? Is there a chance for 0.9.2?

Inertia's picture

Jacobo has given me permission to use the textures (thanks again!), so I'll clean and wrap it up these days and post the example app.

Not sure why the loaders aren't part of 0.9.1, Fiddler probably wants to make it more abstract :P

the Fiddler's picture

Actually, that's the reason. I feel it would be bad if there are two interfaces for textures (one for DDS and another for everything else).

Besides, you have to draw a line at some point and release what you have (almost 450 commits for 0.9.1!) No doubt the loaders will be available soon in the SVN repo.

teichgraf's picture

No doubt the loaders will be available soon in the SVN repo.
Sounds good!
And Fiddler & Inertia, great work so far!
Will the DDS loader have support for cube maps?

Inertia's picture

Updated initial post with the app, sorry for using rapidshare but it seems I've hit some upload limit.

You will have to change and rebuild the application in order to try out the metal texture. The version that will be included in the OpenTK examples will probably only include the rock texture since the depth is much better visible.

Inertia's picture

Here's something I think should be written down in case someone wants to develop the shader further and is looking for a possible improvement:

The Normal/Tangent Vertex attributes required by tangent-space bump mapping could also be replaced by a Quaternion. This is particularly interesting for huge meshes, where memory bandwidth could pose a problem.

Unfortunately searching the web for this topic didn't yield anything useful, I'd recommend handling this suggestion with great care as the required floating point operations for rotating the vertex will increase and might give worse performance on hardware below DX 10 level.

Edit: Stumbled across a paper on Quaternion based shading: http://www.ann.jussieu.fr/~frey/papers/scivi/Hast%20A.,%20Shading%20by%2...

flopoloco's picture

Could you please upload the file again? Link is broken. :P

Inertia's picture

The original post now has a proper file attachment including release build and source code (compiles against 0.9.1 release) .

objarni's picture

Nice of you to share!

Sadly, .rar is not available under Ubuntu (except as shareware for 40 days). Maybe .zip is the most "platform independent" format these days? I think it is readily available on Windows, Linux and OSX.

Also the OpenTK.dll.config was missing in the bin/Release folder.

When running using "mono Parallax.exe" I get the output attached in "log.txt". I'm using the OpenTK 0.9.1 config file, mono 1.9.1 and have a GeForce 7800 GT.

AttachmentSize
log.txt840 bytes
the Fiddler's picture

For rar, just sudo apt-get install unrar (no need for shareware).

This error seems to be caused by EOF differences between the platforms. Can you try replacing the StreamReader with System.IO.File.ReadAllBytes?

Inertia's picture

Is the issue resolved or should I look into it? What Fiddler said would have been my answer too, if he didn't beat me to it. (The first line is a comment, so there shouldn't be a compile error there :P)

I won't update the attachment for the dll.config, since this example should be on svn the second the .dds loader is there too.

objarni's picture

Have you updated the original .rar since my comment ?

Inertia's picture

No, because there's no feedback whether the problem is related to unpacking the archive or the file itself. (Fragment.glsl ist just plain text)

the Fiddler's picture

Objarni, can you try opening the file with gedit (or another editor) and saving with unix line-endings? Alternatively, can you try my previous suggestion to replace the StreamReader? I've been using File.ReadAllBytes for my shaders and never had a line-ending problem.

objarni's picture

There was no problem unpacking the archive.

I'll try Fiddlers suggestions when I get to my home computer (in a few days).