3. Semiotics

Semiotics, or semiology, is a highly complicated field that would need its own literature review to even come to a partial understanding. Semiotics is also related to, or perhaps even inseparable from, the field of structuralism (Culler, 2004, p. 56). Two of the field’s best known theorists are Charles Sanders Peirce and Ferdinand de Saussure. Part of the difficulty in understanding semiotics is that Peirce and de Saussure come from different backgrounds and their work is not always in accord (Deely, 1990, p. 4). While a discussion of their differences in thought is outside the scope of the current literature review, their respective works are invoked on different issues in the course of this literature review. Since I am not trying to argue for the existence of a visual syntax, I do not intend to give priority to a close study of syntactic aspects of semiotics, or other aspects which play an important role in the related field of linguistics. This particular literature review is admittedly biased to the visual. As such, a review of linguistics, while pertinent, is not forthcoming at this stage.
However, in terms of the immediate literature review, it is both the sign-signified-signifier relationship outlined by de Saussure and the icon-index-symbol typology of Peirce which are most relevant (Deely, 1990, pp. 46-47; Culler, 1981, p. 23). Peirce’s typology is particularly important to the discussion of images, both now and historically. The iconic sign bears a semblance to that which the sign represents, the signified, such as a drawing. The index sign has an actual, direct physical connection to the signified, such as smoke, a sound, or a photograph. A photo, however, can also be an iconic sign, since it bears a resemblance to the signified. Finally, the symbol sign is arbitrary and works by social convention. Words are an example of symbolic signs. A semiotic analysis might suggest that Vidtionary is simply a play in semiotics. A Vidtionary video definition edits together video recordings, sound recordings, still photographs, clipart, and animation in order to represent a word. The word itself is often freed from typography, becoming a graphic designed to have a semblance or some relationship to that which it represents. In semiotic terms, Vidtionary uses a combination of icon, index, and symbols to point to a sign, which is the word. In the Introduction section, I stated that my goal is to link the word to the world, but a semiologist might argue that a Vidtionary video definition is a sign representing a sign. A Vidtionary video definition will no doubt be challenged to not become a ‘mockery of signs’ (Baudrillard, 1994, p. 92) or one of Barthes’ myths (Barthes, 2004, p. 82).
In his writing on images as court-room evidence, Richard Sherwin (2008) has suggested that images possess a potency which is much greater than that of words. He outlined three reasons why images can persuade better than words:

First, because photographs, films, and videos can appear to resemble reality, they tend to arouse cognitive and especially emotional responses similar to those aroused by the real thing depicted … [2] because images appear to offer a direct, unmediated view of the reality they depict, they tend to be taken as credible representations of that reality. [3] When images are used to communicate propositional claims, at least some of their meaning always remains implicit. Images cannot be reduced to explicit propositions. (pp. 184-185)

Sherwin’s three reasons owe much to the icon-index-symbol typology. When Sherwin argued that the image is more powerful than the word as evidence, it is precisely because the image can function as both iconic sign and index sign that it seems nearly inscrutable. Sherwin went on to argue that images “arouse cognitive and especially emotional responses similar to those aroused by the real thing depicted” (p. 184). But Nicolas Mirzoeff (1999) has suggested that photos and recordings have lost their ability to index reality, because “everyone knows they can be undetectably manipulated by computers” (p. 8). Following Mirzoeff’s way of thinking, an index sign is compromised once it is manipulated, but I tend to think it depends on the intent and degree of manipulation. Legally speaking, the day that images can no longer be used as evidence in a courtroom will be the day that they lose their power as indexes.
In Things That Talk, Lorraine Datson (2004) has described the controversy that arose when photographs were first used as courtroom evidence in the United States. She has written, “Opponents contended that the photograph was a pale substitute for firsthand evidence, the ‘hearsay of the sun’” (p. 13). Detailing the feelings of horror an exhibition of glass flowers produced in 19th century viewers, Datson has described the “fear of the counterfeit usurping the place of the real” (p. 13). One can look much further back in history to the Christian iconoclasts, or even to the early 21st century when the Taliban destroyed 1500-year-old Buddhist statues, and see that throughout history images can be regarded with suspicion and be at the center of controversy. In the first chapter of his book Visual Thinking, Rudolf Arnheim (1997) has suggested that society generally distrusts the visual realm and visual perception. He traces this distrust back to Plato. Plato considered the sensory world to be illusory, and suggested that truth lay beyond the realm of the senses (p. 9). Arnheim cited other famous examples of distrust in images. There is the stick dipped into water which appears to look broken, the distant object which looks small, and honey, which tastes bitter to some and sweet to others (p. 5). Arnheim saw the end result of this distrust as the neglect of the arts in education. He wrote, “The arts are neglected because they are based on perception, and perception is disdained because it is assumed not to involve thought” (p. 3). In his 1945 Phenomenology of Perception (2007), Merleau-Ponty wrote that, “We must not, therefore, wonder whether we perceive a world truly, we must instead say: the world is what we perceive … For if we speak of illusion, it is because we have recognized illusions, and we have been able to do this only in the name of perception” (p. 63).
Arnheim (2007) has suggested that our society and education system are verbally biased. He writes, “Roughly half of our population, business, and office workers, teachers, lawyers, civil servants, and salesmen, spend their working days handling references to things, products, and services rather than producing or employing these things themselves” (pp. 203-204). He pointed out that people might fail intelligence tests, not because they do not understand a physical phenomenon, but because they might be unable to understand certain verbal phrasings in the question (p. 203). In other words, the intelligence test is simply a language test. He was critical of the education system’s push towards abstraction, moving away from the realm of sensory perception to a realm where verbal modes of expression become substitutes for real-world sensory stimulation. Arnheim warned against language becoming detached from reality. He wrote, “This harmful alienation occurs in people largely concerned with things that stand for other things” (pp. 203-204). He considered verbal thinking to be sterile if it forgets the perceptual experience from which, he argued, language is derived (p. 232).
When Arnheim referred to the ‘things themselves’, he may or may not have been intentionally referring to the field of phenomenology, which often uses the phrase ‘the things themselves’. Merleau-Ponty (2007) has written:

To return to the things themselves is to return to that world which precedes knowledge, of which knowledge always speaks, and in relation to which scientific determination is abstract, significative, and dependent, as is geography in relation to the countryside in which we have learnt first what a forest, a prairie, or a river is. (p. 57)

The phenomenologist wants to throw off the shackles of language and go back to the things themselves, pre-language and pre-philosophy. Klaus Held (2003) interprets Edmund Husserl’s phenomenological method to suggest that “objects have an existence that is independent of both subject and situation” (p. 18). Writing before Husserl, Nietzsche had also used the phrase ‘the things themselves.” Nietzsche wrote:

We believe that we know something about the things themselves when we speak of trees, colors, snow, and flowers; and yet we possess nothing but metaphors for things – metaphors which correspond in no way to the original entities … Every word instantly becomes a concept precisely in so far as it is not supposed to serve as a reminder of the unique and entirely individual original experience to which it owes its origin; but rather, a word becomes a concept in so far as it simultaneously has to fit countless more or less similar cases – which means, purely and simply, cases which are never equal and thus altogether unequal. Every concept arises from the equation of unequal things. (as cited in Medina, 2005, p 122-123)

Merleau-Ponty (2007) includes Nietzsche along with Hegel, Kierkegaard, Marx, and Freud as part of the phenomenology movement before it arrived “at complete consciousness of itself as a philosophy” (p. 56).
It might not help the cause of the language learner, for which Vidtionary strives to become an aid, to invoke philosophical traditions which call into doubt the meaning of words and the system of language in general. Nevertheless, by pointing back to things themselves, it is hoped that Vidtionary can remind that the things exist in the world, whatever the word is that is ascribed to them. Vidtionary embraces phenomenology’s task “to reveal the mystery of the world and of reason.” (Merleau-Ponty, 2007, p. 68)