Hangar's picture

Font rendering

[ TheFiddler: Topic split from http://www.opentk.com/node/236 ]

I get picky about text rendering and layout, which are two very different problems. Adding OpenGL, you have the additional problem of how to get the data into the graphics card. Here's are a couple good resources about advanced rendering: http://dmedia.dprogramming.com/?n=Tutorials.TextRendering1 and http://antigrain.com/research/font_rasterization/ The subject gets pretty complicated pretty quickly. I'll go over some things I'd like to do optimally, but by no means at first, and incrementally over time.

On the text rendering/OpenGL side, Freetype is the best library around. I know you're concerned about the external dependency, so what I'd like to do sometime is to have a separate Freetype driver that coexists with the OS-based drivers. Besides experimenting with what's discussed in the two links I posted, we'll need some way to find the fonts on the OS, a system for caching those loaded into memory. There's also the issue is applying styles like bolding and italics using either fonts in the same family or applying matricies. Another thing that matters is that fonts are optimized to fit to a specific pixel grid, so you have to query the OS for the screen pixel size and set a matrix based on that when you want to render the text.

Text layout is a completely separate problem. The simplest form is having fixed-width glyphs and trying to figure out where to break a line (so you don't have "exa" on one line and "mple" on the next). But it, too, gets much more complicated. For example, a lot of fonts have kerning, where two glyphs like an A and V are moved closer to each other because it looks funny otherwise. Internationalization adds a whole new layer of complexity. There are very few good libraries. Windows and OSX have built-in layout libraries. The only two good free ones are Pango and the ICU layout library. Both of the latter have their problems. I've found ICU much easier to work with, though it's written in C++, so it may have to be complied separately. Pango is written in C, and I've found it extremely difficult to use because all it has is a fairly terse API.

Before getting into Freetype and one of the free layout libraries, I would try to expand the complexity of the existing OS-based font drivers so that they could handle Unicode text, and maybe figure out some sort of tagging construct (XML?) to support bolding, italics, etc. The interface I'd like the client to see would be as simple as RenderText(font, text, rectangle). Maybe the font could be specified with an XML file.

To support this, we'd need access to something like Windows char messages that inform the program of Unicode text. Perhaps the best way to go about this would be to have a sort of text-entry mode. It would suppress normal key events except for those like the enter or escape keys so that hitting r wouldn't fire a rocket when the user is typing, and send character events instead. Note that there's a lot that can be going on here, too. Sometimes more than one keypress triggers a single character, and UTF-16 (what C# strings are made of) is sometimes only valid in pairs.

This is all ideas at this point. I don't want to take over your project, but I'd like to help if I can. This library has the potential to fix what I think are some of the problems with SDL.


Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
the Fiddler's picture

I get picky about text rendering and layout, which are two very different problems.
Wonderful, me too! :)

You raise interesting points here. Unfortunately, I don't have much time at this moment so expect a more thorough reply tomorrow.

First of all, check this screenshot which demonstrates the capabilities of the current font engine. If you haven't already done so, you should also check the two text and font samples bundled with the OpenTK distribution (execute Examples.exe inside the Binaries/ folder).

Here is a brief review of what is currently implemented, and how.

[Fonts]
OpenTK can currently load and use fonts through System.Drawing.Fonts. Individual glyphs are lazily rasterized and packed into OpenGL textures; small point sizes utilize antialiasing and grid-fitting for individual glyphs; larger point sizes only utilize antialiasing. This code supports *any* unicode font (including east-asian ones) - bold and italics are treated for all purposes as different fonts.

System.Drawing.Fonts rely on GDI+, which (IIRC) uses FreeType on Linux. By relying System.Drawing we remove the FreeType dependency from OpenTK itself, but we also lose the capability to use outline fonts. Personally, I am not bothered by this limitation (XNA doesn't support outline fonts either), so I'd put this feature in the "nice to have but not necessary at the moment" category.

The problem with native libraries that are not installed by default, is that they complicate deployment quite a bit. Ideally, OpenTK would run on any supported platform out of the box - which would be very hard once we add a dependency to a native library.

[Layout]
OpenTK currently supports simple LTR text layout with kerning. Grid-fitting, word wrapping, justification and RTL modes have been planned, but are *not* yet supported.

Text is "cut" into glyphs, which are positioned in a grid with proper kerning. The grid is cached as a Display List or a VBO, and can be rendered at any point. Word wrapping and grid-fitting isn't hard to implement (I think I may have implemented both already, but wanted to test more before commiting). The real issue is text movement with grid-fitting (the user must take care to position text into whole pixel boundaries at the end of the movement), which can be a bit annoying.

In any case, text layout is only planned for "simple" (i.e. latin) scripts; "complex" script layouts can be supported through external libraries, this topic is way too large for OpenTK to cover on its own. Once more, the issue with native libraries is that they hamper deployment. Thankfully there is a .Net implementation of of the Bidi algorithm for complex scripts (NBidi), which helps quite a lot.

[Text input]
This issue is orthogonal issue to the previous two. Since the font engine is already unicode aware, we just need to pass the correct unicode characters to the application.

How do we do this? The keyboard drivers already intercept OS key-press/release events, so it's quite simple to make them intercept char events. We'd basically emulate what System.Windows.Forms do for keyboard input ;)

Please, share your comments, thoughts, ideas, questions!

Hangar's picture

Wow, this engine has really gone beyond my expectations. Seems like you've already implemented at least half of what I've described.

Grid-fitting can be adjusted just before invoking the display list. If you query the current viewport and project the raster position onto the viewport, you can make a minute adjustment to fit it to the grid.

Treating different styles as different fonts is a bit difficult to work with. How would you bold a single word inside a paragraph? Maybe it's not generic enough for your engine, but this is the reason I thought of marking up text with XML. If the client wants to handle text differently, the client can translate his/her own method into the XML markup. Perhaps it'd be something like RenderText(Style, MarkupText, Rectangle), where Style would define different tags and MarkupText would apply the tags to text.

For text input, Char events are good. One thing I might like is an OS-dependent keypresses. What I might like to do in my project is have the user specify which keys they want to press to do a specific action. If it wasn't a key found on normal keyboards, I'd just know it as a system-dependent code. A useful feature might be to query key names as well. Though this is a completely separate issue.

the Fiddler's picture

[Grid-fitting]
Adjusting prior to invocation - never thought of this approach! :)

Ideally we would like the text to move freely and only "snap to grid" at the end of the movement, but I'm not sure how to implement this easily. My initial approach was to leave this to the user, but your idea may prove good enough for a workaround.

[Styles]
Please, don't take this personally, but this is a matter of focus: OpenTK is a fairly low-level game-dev library (not engine!), so we only need text rendering to be good enough for games, 3d apps and their GUI's. If this is at odds with e.g. generic text editor requirements, then so be it. (Who would want a text editor in OpenGL anyway?)

That said, you could split text at the word level (like OpenOffice/MS Office do), and apply styles per word. This would incur a speed hit (a big Display List is faster than many small ones), but styling would be much more flexible.

Text markup is also out of OpenTK's scope. I'd love to see such addons, but these don't belong to the core library. (Btw, the font engine follows the (style, rectangle) paradigm for text layout, so it would be relatively simple to write a RenderText frontend and use OpenTK.Fonts for the rendering backend).

[OS-dependent keypresses]
I'm not sure what you mean by this. Care to elaborate with a more concrete example?

Right now, OpenTK only reports key presses (not char events, or driver scan-codes). There are provisions to report extra keys (like volume keys etc), found on some keyboards, while unknown keys are currently reported as Key.Last. This can change, though: we could, for example, define Key.UknownXY tokens and map them at runtime to unknown key-presses (obviosuly the mapping should stay consistent between program runs).

[Key-names]
You mean key names as defined by the OS? I try to avoid exposing OS-dependent functionality as much as possible - you can always use the platform independent key names defined in the OpenTK.Input.Key enum.

Inertia's picture

Just to have this written down:
-games tend to ship with artist-created fonts as bitmaps in different resolutions. This has the advantage that default fonts can be tweaked at the pixel level to match the game's requirement. Example: For use in a chat overlay, a font often needs a black border to be legible no matter if the player is currently in a dark area (hidden in shadow, almost black background etc.) or in a bright area (well lit, maybe bright explosions etc.). Chosing a single color for a font - without any border to improve contrast - might be more work to tweak than just editing the font in your favorite 2D painting app and hardcode (or read from XML, or whatever) the font's kerning. It's not like that has to be changed once it's done properly and a game font seldom needs more than 100 unique characters for languages that use the latin alphabet (i.e. not asian, east european and north african languages).
-coloring text is also very useful to quickly distinguish the importance of multiple (unread) text messages. For example in Quake this is used to differentiate messages from the system (player join/quit etc.), from your team or the enemy team. This allows players to safely ignore certain messages solely by their color, without reading them.

the Fiddler's picture

It is my impression that games are slowly moving from bitmap fonts to truetype. Warcraft 3 comes as an example with excellent font rendering, as well as Vampire: Bloodlines.

You *can* color text with OpenTK - just call GL.Color4 prior to rendering the text. Glow effects are surprisingly easy to achieve, which more or less take care of the border issue (the code is directly translatable to OpenTK.Fonts/OpenGL)

That said, nothing is stopping anyone from using his own font-system with OpenTK. However, I do believe the bundled functionality is useful (font rendering was on of my biggest gripes with libraries such as AllegroGL, SDL, GLFW and Tao :) ) and will probably be used 9 out of 10 times.

One thing I hadn't thought till now is that it would be possible to provide an interface to support other font loading methods (the user would feed glyph textures and kerning pairs and OpenTK would handle the rest), but I'm not sure if that's worth the effort.

Inertia's picture

[Border/Glow]
I don't see that this deals with the alpha channel, but it's a good start. Especially when it comes to localizing an app for different languages, freetype sure is the better solution. Imho it is more complicated to create text like this though, both from artist and programmer side. (assuming you intend to create a custom font)

[Color]
It's not about coloring the whole line of text but allow to color single words/letters in the line, too. If the color isn't stored as floats, the memory consumption is neglectible. It is done in Tremulous (Quake3) like this:

You can color your name by using the ^ character, followed by a number:

^0 - Black
^1 - Red
^2 - Green
^3 - Yellow
^4 - Blue
^5 - Cyan
^6 - Pink
^7 - White
^8 - Black
^9 - Red

This also works for scripts:

bind F6 “say_team Base needs ^3Repairing!”

("Base needs" will be default color (cyan) and "Repairing!" will be colored yellow)

Hangar's picture

[Text Input]

we could, for example, define Key.UknownXY tokens and map them at runtime to unknown key-presses (obviosuly the mapping should stay consistent between program runs).

Yeah, this is exactly what I meant. The only reason for having OS-defined names would be to report back to the user the name of an unknown key. (Such as in a configuration dialog.) By its nature, it is somewhat OS-dependent, and I see why you are reluctant to expose it. It's not a high priority anyway.

[Styles]

I suppose a separate library would be appropriate. However, trying to fit layout on top of an existing font library has gotten me into problems before. Often, there isn't enough data from the font exposed in order to render it properly. If I try to develop an extension, I'll probably have to make some additions to the font interface as well. Maybe you'll accept those changes into the library.

I'm not sure if other font loading methods are worth supporting at this point, but other layout methods would definitely benefit from a clear division between font loading and text layout/rendering.

edit:

On the subject of colors, the XML format I was talking about is just a generalization of the ^1 ^2 colors. So, you'd be able to apply styles like <u>Underlined text</u> and separately define the u tag to add underlining to the text. Or you could make it apply the color blue, though <blue></blue> might make more sense. But perhaps it should be implemented as a separate library.

the Fiddler's picture

[Styles]
No problem with making additions to the interface, provided they don't complicate normal usage too much!

Font loading, text layout and text printing are already separate up to a point. Loading is handled through the TextureFont class, Layout through the TextPrinter and rendering through internal ITextPrinterImplementation "drivers". This way it will (hopefully) be simpler to add OpenGL3 support, once it becomes available.

The original idea was to allow the user to provide alternative implementations, but I haven't explored this area too much. Some interface changes will probably be required to make this possible, but nothing *too* complicated.

Sameera's picture

Hi,

Can you please give me a unicode sample code application in C.I need to design a soft keypad in hindi. Once I select any key it needs to be displayed in the edit box. This would require unicode font rendering.

Could you please provide more information on font rendering and if you can provide a sample application in C as to how to read unicode values and diplay, it would be highly appreciated.

Thanks & Regards,
Sameera