sharoz's picture

Webcam as texture?

Hello,

I’d like to input video from a webcam and use it as a texture. Cursory Google searches didn’t yield much, so I’m wondering if anyone knows how to do this with OpenTk (or Tao). Any links, libraries, or sample apps would be appreciated?

I’ve a Geforce 8800, so gpu capability is not much of a concern.

Thanks in advance.


Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
the Fiddler's picture

The difficult part is acquiring the video data from the webcam. A quick search revealed a couple of cross-platform libraries that do this job, like opencv and libvidcap. They don't seem to have any .Net bindings, but judging from the sample programs that should be easy to solve (e.g. opencv seems really simple).

Once you have the data for a video frame, you can simply upload it to an opengl texture with GL.TexSubImage2D.

objarni's picture

(sorry for off-topic question: when do you use GL.TexImage2d and when do you use GL.TexSubImage2d? The former is "slower" rumour has it; but why two methods doing the same thing but one just slower?)

the Fiddler's picture

GL.TexImage2D discards the old data and (possibly) allocates memory anew - you can use it to change the texture size, pixel format etc. GL.TexSubImage2D updates a region of an existing texture (that was previously allocated with GL.TexImage2D).

In this application, you'd create a texture at startup once with GL.TexImage2D, then repeatedly update its data with GL.TexSubImage2D.

Edit: Another thing to keep in mind is that this application will be bound by the video bus, not raw GPU power. Both AGP and PCI-E have more than enough bandwidth to cope with webcam frames, but you will probably see slightly increased CPU usage compared to outputing video through DirectShow or XV on windows and linux respectively. Going through OpenGL should be simpler, however.

objarni's picture

I can see some wonderful 3d-webcam-apps popping up in my head now :)

Imagine a downtown street where the TV-commercial on the big building in front of you is the webcam output! :)

the Fiddler's picture

It seems someone has already written opencv bindings to .Net: opencvsharp. Feel free to post here if you have any problems getting this to work.

sharoz's picture

Thanks for the fast response Fiddler. I found OpenCVDotNet, but it's not well packaged. I'm having a rough time getting it working. When (if) I get it working, I'll post some sample code.

I've got FBO's attached to a mesh, so I want to see what I can do with a webcam. Think shaders!

the Fiddler's picture

Don't forget to post screenshots!

sharoz's picture

Got it working!

I used OpenCVSharp. Its documentation is not in English, but the samples are fairly self-explanatory. I’m attaching a sample project that applies the captured image to a texture every frame and sends it through an edge detecting fragment shader (vertical edges are green and horizontal edges are blue).

Capturing the image is slow (7-8 FPS on my laptop). Commenting out the ‘SendToGL’ function has no impact on perf, so the capturing framework is definitely the bottleneck. Nevertheless, it’s fast enough for me.

A screenshot is attached.
The project was too big to attach, so here's a link:
http://vis.cs.ucdavis.edu/~haroz/publicfiles/OpenCVSharpSample%20with%20...

AttachmentSize
capture.png74.75 KB
Inertia's picture

Pretty cool, thanks for sharing :)

Question: Is there any problem with OpenAL, or are you simply not interested in capturing audio too?

sharoz's picture

I didn't bother with audio. You may want to check if OpenCV supports it if you're interested