BlueMonkMN's picture

Improvement to text drawing.

In the Usage forum I posted a question about fonts being drawn anti-aliased when they shouldn't be (only large font sizes should be anti-aliased) so I investigated and found out that it was because of the behavior of MeasureString that characters were being drawn at non-integral positions. So I did a little more coding, and I think I found a solution that makes text look much better, and could be further enhanced to relatively easily support wrapping and alignment. I just hope .NET supports these operations on all platforms. Here are the changes I've made. Please tell me if some changes like this could be included in the main code base and/or what to do next.

TextureFont.cs

        public class CharLayout
        {
           public PointF pos;
           public char character;
           public CharLayout(char character)
           {
              this.character = character;
              pos = PointF.Empty;
           }
        }
 
        public CharLayout[] GetCharLayout(string str)
        {
            List<CharacterRange> ranges = new List<CharacterRange>(32); // Win32 GDI limitation
            List<CharLayout> result = new List<CharLayout>(str.Length);
            int unprocessed_idx = 0;
            for (int idx = 0; idx < str.Length; idx++)
            {
               if (!Char.IsWhiteSpace(str[idx]))
               {
                  if (ranges.Count >= 32)
                  {
                     GetCharLayout(str, ranges.ToArray(), result, unprocessed_idx);
                     unprocessed_idx = result.Count;
                     ranges.Clear();
                  }
                  ranges.Add(new CharacterRange(idx, 1));
                  result.Add(new CharLayout(str[idx]));
               }
            }
            if (ranges.Count > 0)
               GetCharLayout(str, ranges.ToArray(), result, unprocessed_idx);
            return result.ToArray();
        }
 
        private void GetCharLayout(string str, CharacterRange[] ranges, IList<CharLayout> result, int start)
        {
           StringFormat fmt = new StringFormat(StringFormat.GenericTypographic);
           fmt.SetMeasurableCharacterRanges(ranges);
           Region[] positions = gfx.MeasureCharacterRanges(str, font, new RectangleF(0, 0, 1024, 768), fmt);
           for (int idx = start; idx < result.Count; idx++)
              result[idx].pos = positions[idx-start].GetBounds(gfx).Location;
        }

TextPrinter.cs (inside PerformLayout function)

            if (alignment == StringAlignment.Near && !rightToLeft || alignment == StringAlignment.Far && rightToLeft)
            {
                OpenTK.Fonts.TextureFont.CharLayout[] coords = font.GetCharLayout(text);
                for(int char_idx = 0; char_idx < coords.Length; char_idx++)
                {
                    x_pos = coords[char_idx].pos.X;
                    y_pos = coords[char_idx].pos.Y;
                    font.GlyphData(coords[char_idx].character, out char_width, out char_height, out rect, out texture);
                    vertices[vertex_count].X = x_pos;                // Vertex
                    vertices[vertex_count++].Y = y_pos;
                    vertices[vertex_count].X = rect.Left;            // Texcoord
                    vertices[vertex_count++].Y = rect.Top;
                    vertices[vertex_count].X = x_pos;                // Vertex
                    vertices[vertex_count++].Y = y_pos + char_height;
                    vertices[vertex_count].X = rect.Left;            // Texcoord
                    vertices[vertex_count++].Y = rect.Bottom;
 
                    vertices[vertex_count].X = x_pos + char_width;   // Vertex
                    vertices[vertex_count++].Y = y_pos + char_height;
                    vertices[vertex_count].X = rect.Right;           // Texcoord
                    vertices[vertex_count++].Y = rect.Bottom;
                    vertices[vertex_count].X = x_pos + char_width;   // Vertex
                    vertices[vertex_count++].Y = y_pos;
                    vertices[vertex_count].X = rect.Right;           // Texcoord
                    vertices[vertex_count++].Y = rect.Top;
 
                    indices[index_count++] = (ushort)(vertex_count - 8);
                    indices[index_count++] = (ushort)(vertex_count - 6);
                    indices[index_count++] = (ushort)(vertex_count - 4);
 
                    indices[index_count++] = (ushort)(vertex_count - 4);
                    indices[index_count++] = (ushort)(vertex_count - 2);
                    indices[index_count++] = (ushort)(vertex_count - 8);
                }
            }

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
Inertia's picture

"isn't anyone else interested in font rendering?"

Didn't want to disturb the flow of your discussion. It's great that you are working on a sophisticated solution, but I've not much to add since this is way beyond my requirements for printing text on the screen. A greyscale&alpha spritesheet with hardcoded texcoords/width settings for each symbol is all I need (and trivial to implement).

If you want my 2c, use freetype and make that an optional plugin for util.dll. If anyone wants to have super-detailed-pixel-accurate-antialiased-text-printing-with-1000-options-from-an-arbitrary-font: they do have to pay a price for it.
Don't care about execution speed (can always be optimized later if it proves to be a bottleneck) but avoid massive garbage creation. This applies for textures aswell, you might be able to stuff it into some A8L8 format (or even something smaller) and use texture compression, but the price to pay is the continuous create/delete/binding of textures. For a game like Tetris this might be irrelevant, but for a RPG (usually text-heavy) this can become quite complex to handle.

BlueMonkMN's picture

I think I'm unclear on some of the subtleties that distinguish TextRenderer.DrawText from Graphics.DrawString. If Graphics.DrawString is faster, why would you want TextRenderer.DrawText?

If you have a chance to post the TextPrinter implementation you have, I might get a chance to look at it soon and see how it performs/behaves in my environment. I got a few things going on though, so I'm not sure when I'll get to it.

the Fiddler's picture

Actually, on Windows TextRenderer.DrawText can be up to an order of magnitude faster than Graphics.DrawString. The first uses GDI (which is hardware accelerated) while the latter uses GDI+ (which is not). Unfortunately, in our case we want to render to a GDI+ bitmap not the screen, and this is quite costly for TextRenderer.DrawText.

Why use TextRenderer.DrawText? The main reason is that its output matches that of the OS, so you could create an OpenGL GUI that matches the surrounding forms. The other reason is that it can render complex scripts when Graphisc.DrawString cannot.

I'll cleanup the code a bit and upload it tomorrow.

BlueMonkMN's picture

I'm ready to look at / work on code if you've uploaded it anywhere.
One more thought occurred to me, though: as far as I can tell, TextRenderer does not support alignment or word wrap. Since this usage of TextRenderer or DrawString does not see the performance benefits (and in fact sees a penalty, if I understand you correctly), wouldn't it make sense to use DrawString, not only for performance, but also for simplifying the implementation of word wrap and alignment? I for one don't care a bit whether the output matches that of the OS (I don't know why one would care); on the contrary, I would suspect that a cross-platform library should prefer providing consistent output across platforms independent of the OS, all else being equal. I guess the issue of complex scripts remains; I can't comment on that because I can't imagine what Graphics.DrawString can't do that TextRenderer.DrawText can do. I thought Graphics.DrawString could draw any string.

My main concern, however, is with the ability to properly/easily support word wrap and alignment, and it seems like more work than its worth to try to support that manually with TextRenderer.DrawText if it doesn't support it intrinsically like Graphics.DrawString does.

the Fiddler's picture

Both TextRenderer and Graphics support string alignment and word wrapping. Only the first supports complex scripts, Graphics.DrawString definitely cannot draw any string (this is a limitation of the underlying GDI+ library, which Graphics uses). Check this document which outlines their differences.

The current text rendering method (compositing individual glyphs) moves the burden of implementing alignment and wrapping to us. The proposed method (burn the whole string into a texture) gives us both (as well as better quality) at the expense of texture memory.

Results will always be different between operatins systems, as the underlying rendering libraries are different (FreeType on Linux, GDI/GDI+ on Windows, Quartz (?) on Mac OS X). There is no workaround for that - Windows 98 will render OpenType/TrueType fonts differently than Windows XP and both will render differently than Mac OS X. If what you look for is consistency, you can always burn a texture with the needed glyphs and render text from that (bye bye unicode! :) ) In fact, this should be easy to implement in, or on top of, OpenTK.

BlueMonkMN's picture

Oh, I wasn't meaning to say that I want or need the text output to be uniform across platforms, just that I wouldn't think that you'd want to strive for the opposite extreme if you had an easy choice with all else being equal.

In any case, after a little further investigation and experimentation, I was almost happy with the results I was getting with my experimentation with TextRenderer. Sorry I neglected to look closely at the TextFormatFlags enumeration. I took a quick glance and got it confused with StringFormatFlags which is for DrawString, and doesn't include the alignment options. But I see now that I can get word wrap and alignment with TextRenderer's TextFormatFlags.

However, after reading the article you linked, one line caught my attention:
Note that when text is rendered into a bitmap, transparency is lost.

So I performed another test and found some disappointing results. Anti-Aliasing doesn't look good on text rendered to a bitmap (image below -- sometimes they don't show up so I hope you can see it):
.

The top line is with TextRenderer and the bottom line is with Graphics.DrawString, both pre-rendered to a bitmap.

Did you upload the code? One thing I will need (and if you haven't implemented it, I can try to add it) is exposure to the information about the size of the text. I suppose I could access MeasureString directly for that, but why waste the time if OpenTK already knows the answer (which I assume it must if it is going to be copying a rectangle containing the text).

the Fiddler's picture

Yes, this is the biggest problem with TextRenderer, although this screenshot isn't exactly representative (there's no alpha blending). This problem is visible in the picture I uploaded to the first page, too.

Looking at the source code of Mono, it looks like we can work around the allocation problems of MeasureCharacterRanges by DllImporting gdipMeasureCharacterRanges directly. It also seems that Mono's TextRenderer uses Graphics.DrawString on Linux, so we are limited to GDI+ rendering in all cases.

In this case, MeasureCharacterRanges on top of the current system looks like the most attractive solution. This also means we won't be able to render complex scripts, but this can be addressed in the future if the need arises.

I haven't found the time to polish the code yet for uploading (it's not compatible with the current system), but it looks like that might not be needed after all.

BlueMonkMN's picture

I see... the first time you made the comment about alpha blending, it went a bit over my head I guess. Now I understand it better. I don't understand why the screenshot above isn't representative. The edges of the text are clearly jagged on the upper sample that uses TextRenderer compared to the smoother edges of the lower sample which uses DrawString. The bitmaps are drawn on top of a gradient background (which is not part of the bitmap) to clarify the problem.

But on to the next question. Have we switched places now? :) I'm kind of liking the pre-rendered cached idea, and you're back to MeasureCharacterRanges? I'm OK with either one, actually, but you mentioned working around the problem in Mono. Was the problem in fact only in Mono, and Visual Studio wouldn't be an issue? I thought the problem had to do with memory allocations occurring in the platform implementation underlying MeasureCharacterRanges, not the .NET framework implementation itself.

the Fiddler's picture

This screenshot isn't representative, because it doesn't perform any alpha-blending at all. The screenshot in the previous page on the other hand does perform alpha blending. How does it do this? It calculates an alpha 'mask' from the rgb values TextRenderer produces (the cached text is stored in an alpha-only texture). The quality *is* worse than correct alpha blending, but it doesn't look quite as bad as this screenshot.

Regarding MeasureCharacterRanges, it's problematic on both runtimes (.Net and Mono), due to memory allocations. I looked into Mono's implementation to see if this can be worked around somehow, and indeed it can (don't you love open-source software? :)) This means MeasureCharacterRanges is a valid approach to the problem.

I wouldn't go so far as to say we've switched places, it's just that it's best to consider all options ;)

Ok, summing up a few thoughts (I think better when writing stuff down :))

There are two parts we need to consider for the font engine:

  1. How to lay out and rasterize text (low-level stuff)
  2. How to display the result through OpenGL efficiently (high level)

Regarding the first, I can see three options:
a) Use interfaces that come with .Net (e.g. TextRenderer, Graphics)
b) Use OS interfaces through DllImport.
c) Rely on a native library non-system library, like Pango.

Of these:
* TextRenderer can layout complex scripts, but is slow and has problems with antialiasing on Windows.
* Graphics have good quality but cannot layout complex scripts.
* OS interfaces (e.g. GDI on Windows, ATSUI on Mac OS X) are more flexible, fast, can render complex scripts, but require platform-specific code (making ports more difficult).
* Native libraries hide the complexity of OS interfaces, but add a native dependency to OpenTK (which we do not want).

Pick you poison! :) I'd say "Graphics" plus "MeasureCharacterRanges" provide the best compromise between quality/complexity for now.

Regarding (2), namely displaying through OpenGL, we have two options:
a) Store individual glyphs in texture memory, and layout text as an array of textured quads.
b) Store whole strings in texture memory, and draw text as a single textured quad.

The first option makes better use of texture memory, but is more involved when performing layout (we have to position individual glyphs). The second option burns more texture memory, but we can delegate layout to the low-level handler (e.g. layout string "abc" in a 200x100 box, centered).

I honestly cannot say which option is better in the long run (input?) We already have (a) implemented, so we can keep using that for now.

objarni's picture

Here are some text rendering use cases I can come up with from the top of my head:

1) menus and static heads-up-text (does not change much ie cache-friendly)
2) fps (changes wildly, ie not cache-friendly)
3) logs/console/ingame-chat (changes often but not as often as fps)
4) "message boxes" like in RPGs (layout very important, does not change much, cache-friendly)
5) score (changes often)

How do choice (1) and (2) compare regarding these use cases?

Also, another case which I'm not sure if OpenTK intends to support at OpenTK.dll-level:

6) In-perspective text like signs in an RPG