Multimedia content has become part of our everyday life. Video and audio reports from current events broadcasted on TV and the web, audio books, short video clips and music consumed for entertainment, and video lectures for education are part of a long list of possible examples. The life cycle of multimedia content is typically hidden from the consumer, starting (in a very simplified model) with capturing followed by post-production and distribution. To consume content users have to find content by searching, browsing or recommendation of others, and as soon as a choice is made multimedia streams have to be adapted and delivered.

The project Self-organizing Multimedia Architecture (SOMA) aims to capture the whole life-cycle of multimedia content in a single architecture for large distributed multimedia information systems. In SOMA we focus on scenarios where events, which we understand as “limited time periods of special importance”, are a central concept. Examples for such scenarios are sports events stretching over time, where start, finish or critical parts of a race are possible events, or traffic monitoring, where events like traffic jams or accidents have to be reported and investigated.

Within the project we investigate the behavior of small but efficient (computing) units working self contained but collaboratively. A network of smart sensors reports events to a distribution network – captured in multimedia data units. In the distribution network events are analyzed, processed, stored, and prepared for delivery. Events and related continuous data are either pushed to users on a subscription basis or consumed by users based on pull mechanisms. Based on the consumption behavior and user feedback, popularity and relevance of delivered content is assessed and reported to the distribution and the sensor networks.

Self organization takes place on different levels: The sensor network “configures” itself and adapts to changing environmental conditions while the distribution layer optimizes resource management in terms of storage, processing, adaptation, and delivery. On the user level SOMA provides a platform for users to (implicitly) organize content. Relevance is computed by estimating popularity and interestingness, whereas the latter is investigated in context of the users’ intentions. In between the different levels – introduced as layers in SOMA – events, multimedia data, and metadata are exchanged. Hence, sensors can react to events pointed out by users, distribution takes sensor capabilities and usage patterns into account to harmonize and orchestrate input both from users and sensors, and users are able to follow the reports in near real time while not fully relying on automated methods. These cross-layer effects are a central research challenge in SOMA.

Within the proposed three years project we aim to develop the three layers in parallel and in-tegrate the efforts at the end of each year in a major scenario test and evaluation. In addition to the cross layer effects, research challenges within layers are pointed out and research questions are formulated in the proposal. Furthermore, we plan to develop an applied research scenario with our industrial partner ASFiNAG Mautservice GmbH. In this application scenario, we extend our existing research and development results of our previous joint FFG project “LOOK” towards a quality-assured multimedia-based traffic message system on the basis of SOMA.

At the end of the three years we plan (i) to make major contributions to research in the area of distributed multimedia systems and (ii) to present – with the help of our industrial partner – a working prototype entitled “LOOK2” showing the practicability of SOMA in a real world scenario. We also aim to extend our research in this field by subsequent projects in both fundamental and applied research.