Max Kosenko's picture

OpenVG Proposal

Hello all.

I know that it's planned to support OpenVG later in future. And that's ok.
But before that is done I'd like to propose some thoughts and ideas.

OpenVG 1.1 is nice open standard for 2D rendering engine.
It's main target mobile devices, but it's also declared for engine for device independent UI framework.

One of such framework is Flash and that's even declared in OpenVG 1.1.

But since we are all in .NET world - we have another alternative - XAML based UIs - WPF and Silverlight. WPF and Silverlight don't need any new frameworks, since Microsoft doing that by themselves, but Mono needs new modern UI Framework. GTK# is nice, Windows.Forms also. But all those are really old enough. It's hard to compare them with Silverlight/WPF UI stack.

WPF for Mono stuck somewhere behind just because it's too large project. But Moonlight is well in very active development. Moonlight main target to do Linux support for Silverlight, but we all know that after all difference in Silverlight and WPF not that large. And with Moonlight that becomes possible to make WPF-Like applications based on full Mono framework access.

But Moonlight problems in that approach are performance and multi-platform compatibility. It's right now using Cairo for drawing and already hitting a wall of Cairo performance which is not that good at all in software mode, and even with ability to change backends - there is no sign of ability to construct fast HW accelerated UI on Cairo. Also currently that leads to a lot of unmanaged code and related problems.

Idea is following: extend Open Toolkit to OpenVG + make Managed OpenVG over OpenTK.OpenGL/ES. That is shown to be possible in Amanith project, but it's native and closed source. Later that can be expanded with using very nice agg-sharp project which is 100% managed Antrigrain rendering engine - fast and very high quality. But on the first step it's just ok to use software renderer from OpenGL. Preliminary tests shown that even software renderer of OpenGL should be faster than Cairo.

As we all know - main driving factor of open projects is in their usage. I think that such approach can lead to creating fast WPF-like UI for Mono and Moonlight faster than Silverlight and Flash. And creating such strong UI Framework for Mono can bring more developers to Mono platform itself. Especially with the upcoming support for Mono development inside VS2010. And after all that can bring new contributors to Open Toolkit since it can become important part of Moonlight and Mono project.

What do you think about that?

Max.


Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
triton's picture

What exactly do you mean by the "software renderer of OpenGL"?

Max Kosenko's picture

I mean OS based software OpenGL renderer used for fallback.

Max Kosenko's picture

Any questions/comments?

the Fiddler's picture

I absolutely agree that an open-source, OpenGL-accelerated implementation of OpenVG would be quite useful. Such an implementation could even be built on top of OpenTK.

You are also right that there are no immediate plans to add OpenVG support to OpenTK. This might happen in the future, but right now we have much more important things to spend our resources on. However, one thing is certain: our OpenVG support will be limited to a thin .Net wrapper over the unmanaged API. We will positively, absolutely *not* add an OpenVG implementation inside OpenTK!

The purpose of OpenTK is to provide

  • .Net bindings to Open[GL|CL|AL|...]
  • a platform abstraction layer

Things like UI frameworks or graphics engines belong *on top* of OpenTK not placed inside it.

With that out of the way, I somehow fail to see how OpenVG ties with AGG# and Moonlight. Moonlight is a Silverlight implementation based on Cairo. It is written in unmanaged code (I don't know the reasons, but I'd guess they are good). Porting Moonlight to OpenVG would be a monumental task with dubious results (OpenVG support is scarce). A port to AGG# or OpenTK is even more unlikely - both libraries are managed, while Moonlight is unmanaged. Finally, reimplementing Moonlight from scratch is simply out of the question: Moonlight contains man *years* of work and a dedicated, salaried development team. It is not something you can build in a couple of months as a hobby.

The Moonlight developers have indicated that there are opportunities to optimize the current implementation. I would argue that the sanest approach would be to take those opportunities. If the result is still too slow and benchmarks indicate that Cairo is the bottleneck, only then should the developers start implementing a different backend.

For the record, I think Clutter would be a much fit to Moonlight compared to either AGG or OpenVG.

Max Kosenko's picture

ok.

I understand your position to keep from any implementations at all. The problem here is that in such case it would be impossible to make managed implementation of OpenVG wrap it in OpenTK and make it use inside OpenTK.OpenGL.

In case you know how to make OpenTK wrapper of OpenVG the way in which it would be possible to make specific managed implementation and use it inside OpenTK - that can keep you out from implementation works and give that option to others.

When you are talking about the future - what would be your medium safety bet about time of OpenVG start of works?

As of agg-sharp. It's a good rendering engine and if you will be able to inject managedOpenVG implementation in OpenTK - that would be a great candidate for portable fast software render.

OpenVG can be used instead of Cairo. While currently it's real that Moonlight mostly done unmanaged it's still possible to see in future return to managed rendering engine. There are still a lot of managed things inside and it's not impossible to use managed rendering engine. There were some arguments to use unmanaged RE - lack of good managed one. I've seen that Moonlight team would like to have more managed code, but we all know it's resource intensive task. While Moonlight is just "Linux Silverlight" it doesn't have to be managed. Making it multi-platform benefits from more managed code.

I'm not sure that porting from Cairo would be monumental task - it will take a time, but not years.

OpenVG support on desktop mostly not exists, but that will change in a future and OpenGL support is much wider. Moving just to OpenGL would be wrong just because it means creation of rendering engine level. And it's better to use something like OpenVG. OpenVG looks more future promising and on some hardware it's already supported natively. Cairo doesn't have future in embedded use and Clutter not a project which life looks promising.

I'm absolutely agree with you and Moonlight team, that in terms of optimization it's better to do smaller steps first. But optimization is separate current task. I'm talking about future. If not to look ahead there wouldn't be any really fast multi-platform XAML GUI until someone will decide to do it.

I'm not a single developer, and I'm not searching for hobby work. And I understand what means a lot of man-years of work. We are the "self-salaried" team developing own complex product line. And I'm trying to figure out how to shift Moonlight from being slow Linux Silverlight to become best multi-platform GUI framework. That might need much more resources. But in case we can show to community reasons we can find much more contributors.

P.S. What's up with notifications on replies?

Max Kosenko's picture

btw - there is already good start with ShivaVG - open-source implementation of OpenVG over OpenGL. So if you will give an ability to inject managed implementations, I'm sure it's not too long to do managed OpenVG over OpenTK.OpenGL...

the Fiddler's picture

Ok, I think I understand your perspective a little better now: you wish to use Moonlight for standalone desktop applications, as a replacement of WPF. I can certainly see the appeal of that.

However, you still lose me somewhere on the roundtrip through OpenVG and AGG. If the point is to improve Moonlight speed and cross-platform compatibility, then why not file bugs against Moonlight directly and help fix the issues you have encountered?

Rewriting Moonlight to use OpenVG is hardly the answer. Creating a hypothetical managed implementation of OpenVG and rewriting Moonlight to use that... I somehow don't feel this is a good idea. :) Finally, an AGG or Clutter backend *might* help, provided Cairo is the bottleneck (which is not even certain at this point).

Which brings us back to OpenTK:

Quote:

When you are talking about the future - what would be your medium safety bet about time of OpenVG start of works?

If there is interest in OpenVG and we feel that OpenVG will be a good addition to OpenTK, we will provide OpenVG wrappers.

Quote:

So if you will give an ability to inject managed implementations, I'm sure it's not too long to do managed OpenVG over OpenTK.OpenGL...

This is already possible - if you are crazy enough to attempt it. OpenTK wrappers are routed through delegates: use reflection and you can reroute anyhow you like. If you are interested in trying such a thing, please be my guest. :)

However, just so this is clear, I will most certainly not write an OpenVG implementation for free. If your company wished to employ me in such a fashion, I would consider it - but for free? No, just no.

( No hard feelings, I hope :) )

Max Kosenko's picture
the Fiddler wrote:

However, you still lose me somewhere on the roundtrip through OpenVG and AGG. If the point is to improve Moonlight speed and cross-platform compatibility, then why not file bugs against Moonlight directly and help fix the issues you have encountered?

First of all - the problem not in issues. Problem in code portability and scalability. It's not good portable (it will require hand porting large parts of it). And it's slow. It's not only slow because of the Cairo problems, it's slow because it's software renderer.

the Fiddler wrote:

Rewriting Moonlight to use OpenVG is hardly the answer. Creating a hypothetical managed implementation of OpenVG and rewriting Moonlight to use that... I somehow don't feel this is a good idea. :) Finally, an AGG or Clutter backend *might* help, provided Cairo is the bottleneck (which is not even certain at this point).

I'm still not catching why it's not a good idea to use managed OpenVG over OpenGL in case that will appear. Rewriting to use AGG is bad idea since AGG C++, it lacks normal API, it's not supported, it's not standard based, and so on. OpenVG in contrast has large spec, reference implementation, set of tests and so on.

the Fiddler wrote:

If there is interest in OpenVG and we feel that OpenVG will be a good addition to OpenTK, we will provide OpenVG wrappers.

So there is no interest yet and there are no direct plans to do it?

the Fiddler wrote:

This is already possible - if you are crazy enough to attempt it. OpenTK wrappers are routed through delegates: use reflection and you can reroute anyhow you like. If you are interested in trying such a thing, please be my guest. :)

My feel is that having interfaces could be better approach than using reflection. Is there possibility to see that with OpenVG in future?

the Fiddler wrote:

However, just so this is clear, I will most certainly not write an OpenVG implementation for free. If your company wished to employ me in such a fashion, I would consider it - but for free? No, just no.

I was never proposing you development for free (but I'm not searching for salaried developers also). Something can attract you and that would be in line with what you already do. Others may find other things to do. That's a community effort to bring something valuable. I'm trying to see is there a possibility to shift a bit work of different teams related to each other to gain mutual success.

the Fiddler's picture
Max Kosenko wrote:
the Fiddler wrote:

However, you still lose me somewhere on the roundtrip through OpenVG and AGG. If the point is to improve Moonlight speed and cross-platform compatibility, then why not file bugs against Moonlight directly and help fix the issues you have encountered?

First of all - the problem not in issues. Problem in code portability and scalability. It's not good portable (it will require hand porting large parts of it). And it's slow. It's not only slow because of the Cairo problems, it's slow because it's software renderer.

I really don't know enough of Moonlights internal structure to judge its portability and scalability, but its developers have made it clear that they are focusing on features right now, not optimization. Moonlight 2 is an alpha-level project - it's far from complete.

Still, there's a good chance you have uncovered issues that Moonlight devs are not aware of. Filing bugs or feature requests against Moonlight is the best to bring those to attention (this includes stuff like hardware acceleration).

Max Kosenko wrote:
the Fiddler wrote:

Rewriting Moonlight to use OpenVG is hardly the answer. Creating a hypothetical managed implementation of OpenVG and rewriting Moonlight to use that... I somehow don't feel this is a good idea. :) Finally, an AGG or Clutter backend *might* help, provided Cairo is the bottleneck (which is not even certain at this point).

I'm still not catching why it's not a good idea to use managed OpenVG over OpenGL in case that will appear. Rewriting to use AGG is bad idea since AGG C++, it lacks normal API, it's not supported, it's not standard based, and so on. OpenVG in contrast has large spec, reference implementation, set of tests and so on.

Two reasons:

  1. There is no managed OpenVG implementation. Creating one from scratch is not a trivial task.
  2. As far as I know, OpenVG conformance tests cost a lot of money.
  3. Moonlight is written in unmanaged C++ and wouldn't be able to use managed OpenVG anyway.

I'm not saying a managed OpenVG implementation is not worthwhile, but I simply can't see how it would help as far as Moonlight is concerned. At best, it would take years before Moonlight would be able to use it! There are much more immediate ways to help Moonlight right now.

Max Kosenko wrote:
the Fiddler wrote:

If there is interest in OpenVG and we feel that OpenVG will be a good addition to OpenTK, we will provide OpenVG wrappers.

So there is no interest yet and there are no direct plans to do it?

Actually, you are the first one who's mentioned OpenVG. :)

While there are no direct plans for OpenVG right now, adding a simple wrapper is just a couple hours of work (its headers are by far the cleanest of any Khronos API). Obviously, we'd need to do more than add just a wrapper, so I am placing OpenVG into the 'nice to have post-1.0' category.

Obviously, if someone steps up to the task, I'll help get the deed done.

Max Kosenko wrote:
the Fiddler wrote:

This is already possible - if you are crazy enough to attempt it. OpenTK wrappers are routed through delegates: use reflection and you can reroute anyhow you like. If you are interested in trying such a thing, please be my guest. :)

My feel is that having interfaces could be better approach than using reflection. Is there possibility to see that with OpenVG in future?

Two questions:

  • Is it possible to call an unmanaged function pointer through an interface?
  • Will this be faster than calling through a delegate?

Can anyone answer those? (If the answer is yes to both, it would make sense to switch from delegates to interfaces throughout).

Max Kosenko's picture

>but its developers have made it clear that they are focusing on features right now, not optimization.

Yes. And that is optimal current position. I'm not talking about what happens just right now.

> 1. There is no managed OpenVG implementation. Creating one from scratch is not a trivial task.
There is a possibility to start that work of porting ShivaVG to managed world.

> 2. As far as I know, you OpenVG conformance tests cost a lot of money.
Best current OpenVG implementation not certified. Certifications costs. But that's not something really needed.

>3. Moonlight is written in unmanaged C++ and wouldn't be able to use managed OpenVG anyway.
Yes, but as Miguel have said in #Moonlight - it's not impossible or unreasonable to return back to be more managed in the future. Nobody saying that would happen this year at all. And nobody saying that would be easy. But it's reasonable to do. One of the reason why Moonlight is unmanaged is lack of managed renderer.

As of ability to later inject managed implementation of OpenVG (I don't see any other pretendent in OpenTK where it needs to be, but OpenVG one of it): I've studied your approach to calculate and establish set of Delegates. Well having interfaces with that approach would do +1 virtual call. Since you're using GetDelegateForFunctionPointer (which in fact 15% faster than direct P/Invoke, but still can be outperformed with ilasm I believe) that would be visible drop in hot places.

Changing delegates is not good - sometimes you're doing conversion of types to handle interop, which not needed at all with managed implementation. I will think about better approach. Don't want to propose something that will sacrifice performance or bloat code (even autogenerated one).