Therefore, more flexible user interfaces are needed that allow users to sketch a query image by hand drawings and to dynamically select regions of interest from a given query image. If such reference images are not available or if the information need is covered only by parts of the query object, the result usually does not meet the user’s expectation. These reference images must be close to the final result so that the user can take them to express her information need. However, such queries require that one or even several reference images are available prior to the start of the search process. Our solution is fully general and we describe how it uses concepts of templates and variable content elements to reduce redundancies and increase the flexibility in developing paper-digital applications.Ĭontent-based retrieval has become a very popular and also powerful paradigm for searching in multimedia collections, especially in large collections of images. We present an advanced publishing solution that is based on an authoring rather than programming approach to the production of interactive paper documents. Although various frameworks and tools are available to support the development of paper-digital applications, these tend to either require some programming skills or focus on specific application domains. Although a wide variety of technical solutions and applications have been proposed, they all rely on some means of specifying links from areas within paper pages to digital services where these areas correspond to elements of the document's artwork. Over the last decade, there has been an increasing interest in paper-digital systems that allow regular paper documents to be augmented or integrated with digital information and services. Our innovative architecture for multimodal interaction encourages the use of software engineering principles such as modularisation and composition to support a growing set of input modalities as well as to enable the integration of existing or novel multimodal fusion engines. Our solution is based on a central fact base in combination with a declarative rule-based language to derive new facts at different abstraction levels. We present Mudra, a unified multimodal interaction framework supporting the integrated processing of low-level data streams as well as high-level semantic inferences. However, there is a lack of multimodal interaction engines offering native fusion support across different levels of abstractions to fully exploit the power of multimodal interactions.
Existing multimodal fusion engines and frameworks range from low-level data stream-oriented approaches to high-level semantic in\-fer\-ence-based solutions. In recent years, multimodal interfaces have gained momentum as an alternative to traditional WIMP interaction styles.
0 kommentar(er)
