Matthias

So, there's much interest in the Phonon + Quasar code - as I had hoped

To reproduce the result on your computer you need to get current phonon, current phonon-xine (with libxine >= 1.1.12) and current quasar from SVN. For those that don't know where Quasar lives, you can find it here: http://websvn.kde.org/trunk/playground/base/quasar/. To compile quasar you need to set the KDEDIR environment variable to the prefix where you installed phonon and then run qmake and make in the source dir. Then go to the examples/phonon subdir and run

./phonon /path/to/video.ogg

Have fun detecting edges in your videos ...

I've got it working a long time ago, but since this Akademy it is finally available in KDE trunk: Phonon now can give the raw video frames (as RGB, YV12 or YUY2) to an application and Quasar gained a node to upload those frames as textures and uses Shaders to convert YUV formats to RGB and then do arbitrary effects with them. The cool part about that, obviously, is that the data intensive calculations are happening on the graphics card (you need OpenGL 2 for this to work). To demonstrate the power of this I wrote a simple example that connects the PhononInputNode to an EdgeDetection node and then to a RenderNode. I did a screencapture of the result, but that looks a lot worse than the real performance, which is totally smooth and, of course, doesn't show much CPU usage.

You can also see me adjusting the threshold value of the edge detection filter with a keyboard shortcut.

Here's the code:

Things to note:

• the PhononInputNode is both a Phonon::MediaNode and a Quasar::Node


• the Quasar API is not final, especially wrt. PhononInputNode

As and there will surely be a few more people traveling the same way, I wanted to ask if there's interest in traveling together - or to share a car ride. I've already talked to Martin Gräßlin who lives in Mannheim (I'm in Heidelberg, in case you didn't know already), and we'd probably take a train together. Neither of us owns a car, but if you do and drive by Mannheim/Heidelberg (A5) and want company and people sharing the high fuel cost with you, let me know. If not, you still might want to try to meet up in the train... in that case also let me know.

For the train I'd probably take the ICE 610 (12:35 Mannheim - Köln) and ICE 14 (14:44 Köln - Bruxelles-Nord) on Fr, 8th, of course. But sharing a car ride would probably be cheaper.

Now back to exam preparations...

After a month of no KDE coding I'm back to do a few things in KDE again - though I really want to (and need to) do more for University if I ever want to get done with it...

Anyway, what I wanted to show is how to do simple animations in the UI that should help the user to understand how the different UI elements play together. I have one such case with the Phonon KCM where I now tried the following animation to clear things up a bit:

more_ui_feedback.ogg

The way I achieved this effect was to add a special Animation widget that sits above the other widgets and is not added to any QLayout. Reimplement the paintEvent to draw the animation frames and use a QTimeLine member to call repaint(). It needs two pixmaps: one of the old content of the QListView at the right and one with the new content. The paintEvent then draws the old content over the QListView so that it seems the widget contents have not changed yet and draws the new content depending on the frame number of QTimeLine (with 50% opacity). QTimeLine::finished() is connected to the deleteLater slot which removes the animation widget.

Here's the code:

Today I checked my mails and saw a few Phonon-related commits. I always read those commit messages with priority... and I was a bit surprised. I knew this day would come. But I didn't know when and I didn't know whether it's going to be GPL+commercial or LGPL. And I found the commit messages all said "[License: LGPL]". *dance*

Now what does that license mean? It means everybody can contribute without having to sign a copyright assignment. That just makes it so much easier to handle contributions!

Now I know what code I'm going to read and play with the next days.

Read the rest here and here

Yesterday on IRC I was reminded of the nice abstractions Qt has with QPaintDevice. (There are many useful paint devices, but most important for this discourse are QWidget, QPixmap and QImage.) So there are several things Qt4 allows you to do:

• QPainter::setRedirected
• QPixmap::grabWidget
• QWidget::render

Inspired by the video from Mirco Müller I wanted to see what could be done with those functions. I started with setRedirected but couldn't make it work immediately. So I did what the docs said and tried QPixmap::grabWidget. And without much effort I was able to draw a widget someplace else. But then I wanted to transform the image which needs a conversion to QImage. Expecting that Qt might optimize a bit if it renders directly into a QImage instead of a QPixmap and then converts to QImage I used QWidget::render to render a widget directly onto a QImage. Piece of cake. Now I had a QImage, transformed and painted into a different widget. Last missing piece is to update the image if the original widget changes (e.g. mouse or keyboard events). QObject::installEventFilter makes it easy to listen for paint events on the other widget and update accordingly. And to make it a fully featured class I added the ability to look for child widget paint events and made it repaint only as little as needed.

The result is this code:

Video

Today I wanted to see whether this eye-candy can be put into KDE at some place and I have not succeeded. Perhaps somebody can make a mockup of how this could be used without making you feel "something is wrong".

Anyway more pictures:

Notice that all this is just fooling around with what Qt provides. This is neither useful nor going into KDE 4.0

For the interested people, the implementation of WidgetMirror looks like this:

While working on assignments for the Computergraphics course I'm taking this semester I was missing signals and slots. Using Qt for the assignments is not possible (well actually I didn't ask explicitly whether I could use it), so I felt challenged to code the convenient things I'm so used to...

I knew that boost had some implementation of signals and slots that used templates but I had never actually taken a look at it. And I didn't want to, yet. I wanted to think about the problem myself first. Basically I came up with that a signal needs to be an object where as a slot may not have to be an object, but must also work as a member function. After a few iterations I came up with a header that now allows me to do the following:

The output of that program is:
A::operator()
A::f
g
B::operator() 5
B::f 5
g 5
as you probably expected. Notice that

• You can use 0 to 10 types in the template parameter of the Signal class.
• You can connect to functions, member functions and objects that have an operator() with the right signature. This allows to connect one signal to another one, like:
  Signal<const std::vector<int> &> s1;
  Signal<const std::vector<int> &> s2;
  s1.connect(&s2);
  s2.connect(&someFunction);
  s1(std::vector<int>());
  This will call someFunction(std::vector()).
• Signal will only allow "slot" connections and operator() calls with the correct signature. I needed to make Signal have different member functions depending on the number of its template arguments. Took me some time to realize but this was possible using inheritance.

Now you probably want to see the code for Signal:

After playing with the mixer API a bit I feel confident to be able to go forward a propose a design.

class diagram

I'm happy for people to look over it. Don't think about implementation details but whether this API would be easy to understand, use and still powerful enough for all our desktop uses.

After a phonon/phonon-xine debug session (where I got stuck at KIO playback showing inexplicable memory access violations inside xine-lib) I had to do something creative again. So I tackled the mixer.

First I started with trying out ALSA's softvol plugin, and whether that would have less latency than the software volume control inside xine-lib. As it's close to impossible to measure I have to rely on my senses and that's telling me there's no audible difference. Ok, so for AudioOutput's volume property I could stay with xine-lib's offering, but seeing softvol work for the first time that got me started on exploring the ALSA mixer.

When an application uses an ALSA "device" that contains the softvol plugin then this will add a new "integer control" to the mixer. KMix and also alsamixer do not show the new control without a restart, and the control does not disappear again when the application is done and no app uses the softvol anymore.

From there I went to look at kmix (trunk)'s abilities wrt. custom layout definitions and whether it would be possible to group KDE made softvol controls in a sensible way. This was not getting where I wanted without touching the code so I finally started libphononmixer.

When you want to use libasound (ALSA's user space lib) to fulfill a task it is, IMHO, best to not go from the doxygen generated documentation (which partially on their website is empty anyway) but open an editor in the libasound code and look at the header and source files. As the doxygen comments often don't tell much more than the function and parameter names already do the code is really the best to go with. And now I have a Mixer class that can be constructed from a Phonon::AudioOutputDevice object, a QString or an enum value. This installs a QSocketNotifier on the poll_descriptors libasound gives me and reads out all the events when the filedescriptor is ready for reading. With this I get all change notifications of a given mixer, even when a softvol control is added.

From there I'll go and make a nice high-level mixer API that lists all available mixers, gives notifications when a mixer appears/disappears, lists all the elements of a mixer, makes those elements give value change notifications and so on. I'll look into the possibility of removing softvol controls from the mixer, and whether it's possible to identify their origin and group them correctly. Once this is all ready it'll be the perfect base for a Plasma Data Engine and then a Plasma Applet for controlling your sound.

Personal Note: I'm close to being done with my sickness and probably today I'll go back home (where I'm alone because my wife is far away in Spain ). I stayed with my parents for the time I was sick. It's great to have them, not only as fallback solution, but in those cases it becomes so much more obvious just how much a great family is worth!

It's been a month already since I was at the Trolltech office in Oslo, but ever since I wanted to write about it...

Trolltech invited me to be able to discuss the design of Phonon or, more in general, what their needs are for a multimedia API that they want to provide with or in addition to Qt. The first three days (Wed.-Fri.) we've been whiteboarding the whole day, which was quite exhausting but OTOH exactly what I always wished to be able to do when I was designing Phonon in the beginning. The outcome was that we agreed that we want to have an API that provides all the flexibility of a graph based approach while simplifying the concepts as good as possible so that non-experts have no trouble using the API for simple tasks without getting any details wrong.

Before

This is where we needed a small change in the Phonon frontend API to make this possible. Before August Phonon allowed "graphs" of the form:

Path changes

Now the AudioPath and VideoPath objects are replaced by a value based class called Path, which is hidden for simple cases:

vs.

or in code:

I wanted to keep the Path concept to make it easier to support the most common "graph reconnection": to insert/remove effects. With the common graph you'd have to disconnect media and output, connect media to effect, and effect to output and also somehow tell the objects that they should not lose any signal data while they're still disconnected and that playback needs to continue seemlessly after the effect insertion. With the path object it's a simple

Mixing

Another change is that is not allowed anymore. The implicit mixing of audio data is removed. (For video it was never allowed anyway). The plan is to add a special class that handles mixing after the 4.0 release. Implementation note: This is close to impossible to implement using libxine.

What's the gain?

Perhaps you've already seen it, but why is this now more flexible than what Phonon provided before? The answer is that you can now define arbitrary graphs like . In a later version we also want to have an object that can "contain" multiple MediaObjects in order to be able to control them in sync. This is basically the concept of a media graph containing multiple source nodes. But if you have only one source, which is the case for most apps, the MediaObject defines the graph implicitly. This hides the graph concept for most developers using Phonon, which IMO helps to make Phonon easier to use.

Trolltech

Now what's Trolltech's role with regard to Phonon? They want to provide multimedia API with Qt 4.4. And they decided to set their bets on Phonon. This means they will work (with me and the rest of the community that is interested) on continuing the design of the frontend API to support use cases like capture, transcoding, VoIP apps, Video chat, ... And they will work on backends for Windows, MacOS X and Linux (including embedded). KDE will ship the xine backend with 4.0, but that one has several problems:

• libxine is GPL-only, so they cannot tell commercial customers to simply use Phonon on Linux for multimedia


• libxine does not support mixing of audio streams, and I tried to separate the clock and ticket system in libxine to make it more flexible, but I feel like this is a dead end. libxine is designed for simple playback tasks and that's it.


• no capture or encoding with libxine

On another note, 2h per workday, while I'm working on Phonon, are now sponsored by Trolltech. This additional money is very welcome, especially now that we'll have to pay the rent for a flat in Heidelberg and another one in Sevilla.

I'll be gone for 10 days to visit Sevilla, Spain. My wife will spend a semester abroad (and I'll stay in Heidelberg) at the University of Sevilla.

As finding a place to stay is not as organized as in most other (european) countries, the first task we'll have to accomplish is finding a flat that is not overpriced and not "interior" (meaning without windows). Until then we'll stay at a hostel and hope to also find time to do a bit of vacation.

The weather is supposed to be too hot for my taste, but just right for Flora.

I'll not be reading mailinglists, but if you send a mail directly to my address you have a small chance that I will see (and perhaps answer) it before Sept. 11th.

NMM Release 1.0.0 "April"

I'll be taking a closer look at this release the next weeks and will be more actively looking into the NMM backend again.