Table of contents -- Acknowledgements -- Introduction -- Multimodal issues -- In between modes -- 1.. Multimodality - the late discovery of the obvious -- 2.. How many modes is `multi'? -- 3.. Mode distinctions -- 3.1. Semiotic properties -- 3.2. Cognitive orientation -- 3.3. Semantic potential -- 4.. Mode integration: Overlapping and mixing -- 5.. The textual interface between language and image -- 5.1. The typographic image -- 5.2. The verbal image and the language-image-link -- 6.. Semiotic principles across modes -- 7.. Conclusion -- Notes -- References -- Modelling multiple semiotic systems -- 1.. Introduction -- 2.. Language vs. semiotic systems -- 2.1. Stratal organisation and grammar -- 2.2. Semiotic systems and metafunctions -- 3.. Mode and the relationships between semiotic systems -- 3.1. Mode in SFL -- 3.2. Mode as the primary activator of semiotic systems -- 3.3. Fractal principles between semiotic systems -- 4.. Conclusion: An example using gesture -- Notes -- References -- Appendix -- Notation conventions -- Problematising `semiotic resource'* -- 1.. Multimodal semiotic resources -- 2.. Nature of a semiotic resource -- 3.. Visual images as semiotic resources -- 4.. Extending the claim of arbitrariness -- 5.. Basic building blocks: Words and icons -- 6.. Conclusion -- Note -- References -- Multimodality and empiricism -- 1.. Introduction -- 2.. An example of interpretative analysis -- 3.. Multimodal annotated corpora -- 3.1. The origin and representation of annotated corpora -- 3.2. Annotation problems with complex data -- 4.. The GeM model: Layering for classification -- 4.1. The GeM model -- 4.2. Designing and populating a multimodal corpus -- 5.. Examples of using a GeM-annotated corpus -- 6.. Conclusions and directions for the future -- Notes -- References -- Analyses and applications -- On the effectiveness of mathematics*.

1.. Introduction -- 2.. Wigner on the effectiveness of mathematics -- 3.. A systemic functional approach to multisemiosis in mathematics -- 4.. A historical perspective of mathematics -- 5.. The Cartesian project -- 6.. Newton's view of the world -- 7.. Meaning in modern mathematics -- a. Language -- b. Visual images: Graphs and diagrams -- c. Mathematical symbolism -- 8.. Inter-semiosis in mathematics -- 9.. Complexity, chaos and Wigner on the effectiveness of mathematics -- Notes -- References -- Multimodality in language teaching CD-ROMs* -- 1.. Introduction -- 2.. Premises -- 3.. Examples of visual/audible `learning aids' -- 3.1. Pronunciation and acoustic phonetics -- 3.2. Text and image -- 3.3. Speech and gesture / lip movement -- 4.. Conclusion -- Notes -- References -- Reviewed language teaching CD-ROMS -- The multiple modes of Dirty Dancing -- 1.. Introduction -- 2.. Constructions of class -- 3.. Constructions of gender -- 4.. Uses of multimodal discourse analysis -- 5.. Multimodal configurations and issues of genre -- 6.. Results: Towards a theory of multimodality -- 7.. Conclusion -- Notes -- References -- Multimodal text analysis and subtitling -- 1.. Introduction -- 2.. Translation studies -- 3.. Creative texts -- 4.. Subtitling -- 5.. The multimodal approach -- 6.. Conclusion -- References -- Multimodality in the translation of humour in comics -- 1.. Introduction -- 2.. Multimodal humour and its translation -- 3.. From playing with words to playing with signs -- 3.1. Linguistic word-play -- 3.2. Plays on words reinforced by non-verbal signs -- 3.3. Plays on signs depending on a multimodal combination -- 3.4. Non-verbal plays on signs -- 4.. Comic pictures -- 4.1. Culture-specific pictorial elements -- 4.2. Pictorial intertextuality -- 4.3. Pictorial allusions -- 5.. Comic aspects of typography -- 6.. Summary -- Note -- References.

Source texts -- Secondary texts -- Multimodality in operation -- 1.. Introduction -- 2.. Theoretical background -- 2.1. Systemic functional theory -- 2.2. Multimodality theory -- 2.3. Linguistic and multimodal analyses of museums -- 3.. The questions to be explored -- 4.. An overview of Salzburg's Mozart-Wohnhaus Museum -- 4.1. The floor plan of the museum -- 4.2. Zones in The Dancing Master's Hall -- 5.. Analyses -- 5.1. Linguistic analysis of the text -- 5.2. Visual analysis of the family portrait -- 6.. Synthesis and discussion -- 7.. Conclusion -- Note -- References -- Floorplans and images -- Appendix 1 -- Zone 2 - Processes in the original Residence text -- Drawing on theories of inter-semiotic layering to analyse multimodality in medical self-counselling texts and hypertexts -- 1.. Introduction -- 2.. Paving the way for a deep-analysis of multimodality in contemporary text corpora -- 3.. Text versus hypertext: Do we need a new conception of text? -- 4.. Approaches to theories of inter-semiotic layering -- 4.1. The model proposed by Hoek (1995) -- 4.2. Practical application of the model on sample (hyper)texts -- 5.. Challenges for the future: Intelligibility -- 6.. Conclusion -- Notes -- References -- On the multimodality of interpreting in medical briefings for informed consent -- 1.. Interpreting in hospitals -- 2.. Interpreting and multimodality -- 3.. The use of diagrams in medical briefings for informed consent -- 4.. Briefing for a gallstone operation -- 4.1. Introducing the organs in the source language -- 4.2. The organs in the target language -- 4.3. The suspected diagnosis in the source language -- 4.4. The suspected diagnosis in the target language -- 5.. Conclusion -- Notes -- References -- Index.