The IRMA Community
Newsletters
Research IRM
Click a keyword to search titles using our InfoSci-OnDemand powered search:
|
multimodal language processing using nlp Approaches
|
|
Author(s): Fernando Ferri (Istituto di Ricerca sulla Popolazione e le Politiche Sociali, Italy), Patrizia Grifoni (Istituto di Ricerca sulla Popolazione e le Politiche Sociali, Italy)and Manuela Tersigni (Istituto di Ricerca sulla Popolazione e le Politiche Sociali, Italy)
Copyright: 2007
Pages: 5
Source title:
Managing Worldwide Operations and Communications with Information Technology
Source Editor(s): Mehdi Khosrow-Pour, D.B.A. (Information Resources Management Association, USA)
DOI: 10.4018/978-1-59904-929-8.ch132
ISBN13: 9781599049298
EISBN13: 9781466665378
|
Abstract
People usually communicate through multimodal dialog. In fact multimodal interaction is flexible and natural because it uses all the five senses in parallel. For this reason we need to consider multimodal languages definition and processing, according to techniques and approaches used in Natural Language Processing (NLP). Here we describe the characteristics of a multimodal language by means of NLP according to the fact that the speech modality appears the most complete one (it is considered to be the predominant modality). Users communicate and interact referring to a set of key-concepts. These concepts can be expressed with different modalities and/or by means of more than one modality simultaneously. In the multimodal language definition, these key-concepts have to be extracted. Such concepts are then processed with a natural language processing approach: any concept expressed in any modality can be “translated” into natural language. This implies that speech acts like a “ground layer” all the modalities refer to. We propose a tool to define multimodal languages, it allows the user defining the language according to his/her personal way to express concepts of a particular domain in the different modalities.
|
|