PHAROS: Lighting the way for European audiovisual search
01 December 2008
THE EMERGENCE OF DIGITAL AUDIOVISUAL MEDIA
Worldwide, the volume of stored information grows exponentially at one percent per week with a rapidly increasing share of audiovisual content. This growth is one of the major challenges for businesses and organisations today which find themselves at a loss on how to unlock the value found in the exploding data volume. Search has become the de facto way of interacting with textual data, e.g. on the internet. On the other hand, search in audiovisual data until today is pursued rather simply: creators or providers of data attach some descriptive attributes or annotations to the files (metadata), and search engines match in these descriptions, but not in the content itself.
This is the major obstacle for true audiovisual search: how to automatically map the ‘opaque' media content to a searchable format that helps compare it with query phrases or with one another for similarity. Today, if a searcher indicates a liking for a specific song of a heavy metal band and asks for recommendations of similar songs, a search engine would probably use ‘heavy metal' as a keyword and recommend a number of other heavy metal bands. However, if that specific song is a slow one, the searcher may also want the system to recommend similar songs of a similar style with probably the same music instruments. That is hard to achieve today, if all this information is not explicitly attached in textual format to all files. But that would be the ultimate goal: searching media based on the content itself.
PHAROS - BRINGING TOGETHER HIGH-PROFILE EUROPEAN PLAYERS
Search-based technologies which can tackle the challenges of tomorrow's requirements are essential for the competitiveness of information societies. Here in Europe, the PHAROS integrated project, which is funded under the EU Framework Programme 6, is building a next generation audiovisual search platform, which is designed, developed and applied jointly by a consortium consisting of high-profile academic and industrial players with proven track records in innovation and commercial success. One of the leading project partners is FAST Search and Transfer, the Microsoft subsidiary headquartered in Oslo, Norway, which is one of the leading-edge companies in search technologies. “FAST is the biggest technology contributor to the PHAROS project, responsible for the development of the PHAROS frameworks for content processing as well as query analysis and search, based on FAST's NextGeneration technology portfolio,” explains Stefan Debald of FAST. As technical director of the PHAROS project, Debald is responsible for the project's technology direction.
CALL FOR INTERDISCIPLINARY COLLABORATION
Matching rich media objects to the information needs of a human user is commonly referred to as the semantic gap. Reducing this gap requires the best possible semantic understanding of media which can be achieved by applying automatic analysis tools. New types of media often go along with new kind of user interfaces, displays, devices and their use in new kinds of applications. Addressing search in rich media therefore requires interdisciplinary collaboration. “The domain of audiovisual search covers many diverse research topics ranging from information retrieval to research in analysis of rich media, combining various audiovisual modalities (image, speech, music, video) and their multimodal combinations, and finally, social media research because rich media consumption patterns and recommendations require a solid understanding of the user aspects,” explains Stefan Debald of FAST. He points out that the PHAROS project consortium reflects this broad set of unique expertise, and that this kind of coming together of software and computer science expertise with media production and telecommunication service providers is of tremendous value for the future of Europe's audiovisual sector.
THERE IS NO ‘ONE SEARCH SOLUTION FITS ALL' ANYMORE
Web search versus Enterprise Search
Whereas Web search, covering indices of approximately 10 billion documents, is often perceived as the world's largest search application, from a business application perspective enterprise search providers face far greater challenges with customer installations exceeding 30 times the size of the web index, spanning petabytes of data. There may be standard scenarios for so-called information workers in business and enterprises, but as a matter of fact the number of specific application scenarios one might think of today and in the future is beyond one's imagination.
The user in the centre
What regularly frustrates users is the fact that the search engine does not consider the specific context of the user and the query to provide personalised results. There are many aspects to a user's context, including culture and the user's background. For example, if a user is focusing her search for locations in the Northeast of the US, it is likely that she is looking for Potsdam in New York State rather than Potsdam in Germany. Users want to be able to prioritise the information that is relevant to the task at hand and users want the option to interact to identify the right information even when they are not sure what they are seeking.
Flexible search platform
What this all means is that there will be no one single search engine capable of handling all kinds of search scenarios. On the contrary, the range of requirements calls for search platforms which provide maximum flexibility to accommodate for the uncertain future and to adapt to a range of application and enterprise scenarios.
THE FUTURE: Q&A WiTH STEFAN DEBAlD, FAST
What makes the PHAROS project unique, not only in Europe but worldwide?
PHAROS is a focal point for integrating technologies arising from various research domains, and all feeding into the project to accelerate the innovation process towards creating a platform for audiovisual search. PHAROS collaborates with European and national initiatives, as well as with some of the largest and most sophisticated media producers and telecommunications service providers, ensuring industrial relevance and world-wide application reach. PHAROS involves 12 partners in 9 different European countries.
Stefan Debald, Manager Research Programs, FAST
Why a search ‘platform'?
PHAROS intends to bring up a search platform rather than another search engine. The difference is that the design of the platform is modular so it serves flexibly as the basis of a range of applications, for industry and business-like service and content providers to build applications with audiovisual search capabilities. The architecture of PHAROS allows a great level of flexibility and ease of integration of new technologies. That means that business partners could take the platform and augment it with additional modules to add new capabilities, which means accelerating their innovation cycles.
The availability of a robust and scalable industry-strength search platform for audiovisual search would also help academia integrate and commercialise their research results in the search domain more easily and rapidly.
Could you give an example for modules that would be an added value for the platform?
The content processing framework of the PHAROS platform allows for extensions of arbitrary rich media analysis technologies which could be best-of-breed products or emerging prototypes from academic research. This openness ensures the ability to adapt to a broad variety of possible future application scenarios.
How do you engage with the broader community in Europe?
PHAROS aims to take future user and search requirements as the key design principles to ensure that the PHAROS platform is well positioned to power real-world application scenarios. Therefore, the PHAROS Federation has been established as an interest group outside of the consortium with key stakeholders of the audiovisual industry to review requirements and follow-up with project results.