Phonon + Quasar

Wednesday, August 13. 2008

I've got it working a long time ago, but since this Akademy it is finally available in KDE trunk: Phonon now can give the raw video frames (as RGB, YV12 or YUY2) to an application and Quasar gained a node to upload those frames as textures and uses Shaders to convert YUV formats to RGB and then do arbitrary effects with them. The cool part about that, obviously, is that the data intensive calculations are happening on the graphics card (you need OpenGL 2 for this to work). To demonstrate the power of this I wrote a simple example that connects the PhononInputNode to an EdgeDetection node and then to a RenderNode. I did a screencapture of the result, but that looks a lot worse than the real performance, which is totally smooth and, of course, doesn't show much CPU usage.

You can also see me adjusting the threshold value of the edge detection filter with a keyboard shortcut.

Here's the code:

m_input = new PhononInputNode();
connect(m_input, SIGNAL(needUpdate()), SLOT(updateGL()));
m_compo.addNode(m_input);
m_displayNode = m_compo.createNode("/output/render");
m_saturationNode = m_compo.createNode("/color/edgedetect");
Quasar::link(m_input, "output", m_saturationNode, "texture");
Quasar::link(m_saturationNode, "output", m_displayNode, "texture");
Phonon::createPath(m_media, m_input);
m_compo.prepare();
connect(m_media, SIGNAL(finished()), m_media, SLOT(play()));
m_media->play();

Things to note:

the PhononInputNode is both a Phonon::MediaNode and a Quasar::Node
the Quasar API is not final, especially wrt. PhononInputNode

Posted by Matthias Kretz in Phonon at 13:49 | Comments (11) | Trackbacks (0)

Trackbacks

Trackback specific URI for this entry

No Trackbacks

Comments

Display comments as (Linear | Threaded)

"Phonon now can give the raw video frames (as RGB, YV12 or YUY2) to an application"

Hehe - I know I only asked you this on Saturday, but in light of this: would a Phonon-ised video kio_thumbnail be possible, now? It would be great if so, and amazing how quickly a "not possible" can change into a "possible"

#1 SSJ on 2008-08-13 14:31 (Reply)

Yes, implementing a video thumbnailer using Phonon::Experimental::AbstractVideoDataOutput is the right thing to do. With this you can do some heuristics on the first frames to find a frame that might show a bit more than only black or the title of the movie (which is a white line in a thumbnail

#1.1 Matthias Kretz (Homepage) on 2008-08-13 15:27 (Reply)

Excellent news; nice work!

The old xine-based thumbnailer has this logic already built-in, IIRC, and replacing the xine calls with Phonon stuff should be reasonably straightforward

#1.1.1 SSJ on 2008-08-13 16:14 (Reply)

Wow, this is good news.

I'm currently working the get webcam support in KMess operational. Does this mean it would become really easy to capture webcam data?

(I'm not working on the capture part yet, still on the protocol code).

#2 Diederik van der Boor on 2008-08-13 14:36 (Reply)

I just bought a webcam because i will travel 10 weeks to calgary. Now i would like to do something with my webcam. Could quasar and phonon be used to create a webcam app similar to the one on osx (don't know the name, but it looks nice and has real time effects)

#3 Beat Wolf on 2008-08-13 15:01 (Reply)

A webcam app, perhaps something like this:
http://www.kde-apps.org/content/show.php/MyFace?content=77565

#3.1 Morty on 2008-08-14 11:33 (Reply)

Regarding webcams: This new class is one part. The other (still missing) one is the capture source.

But once you can have a webcam as video source you're free to do any kind of realtime effects using Quasar with that video data.

#4 Matthias Kretz (Homepage) on 2008-08-13 15:30 (Reply)

Wow -this is really great news.

I'll for sure make a video shape for KPresenter now. (Though it's too late for the KOffice 2.0 release, which has entered soft freeze)

#5 Casper Boemann on 2008-08-13 15:44 (Reply)

Just saw this running in the Phonon BOF, and it is really quite impressive. Any chance you could post the complete program you were using, Matthias?

#6 Casey Link (Homepage) on 2008-08-13 16:30 (Reply)

Phonon can give me raw frames? I have an algorithm working on QImages and was wondering how I can get individual frames out of my videos. So how is it done? How do I get raw frame data? I could not find anything in my QT4.4 doc.

#7 Kraut~salat on 2009-01-18 10:28 (Reply)

I'm also very interested in modifying video frames while playback.

Can you supply me a code snippet showing how to access those raw frames ?

My goal would be to overlay some information within those frames.
( Mostly XORed over the original bytes. )

#8 Martin Schröer (Homepage) on 2009-02-28 15:56 (Reply)

Add Comment

Name
Email
Homepage
In reply to
Comment	E-Mail addresses will not be displayed and will only be used for E-Mail notifications To prevent automated Bots from commentspamming, please enter the string you see in the image below in the appropriate input box. Your comment will only be submitted if the strings match. Please ensure that your browser supports and accepts cookies, or your comment cannot be verified correctly. Enter the string from the spam-prevention image above: HTML-Tags will be converted to Entities. You can use [geshi lang=lang_name [,ln={y\|n}]][/lang] tags to embed source code snippets Standard emoticons like :-) and ;-) are converted to images.
	Remember Information? Subscribe to this entry