Semiotics of the Visual: On Defining the Field. (Mihai Nadin, Ed.) = Semiotica, Vol. 52, No. 3/4 (1984), Amsterdam: Mouton, 1985.
Introduction to The Semiotics of the Visual – On Defining the Field.
A contradiction (it would sound better if I called it a paradox) dominates the definition of the subject: Since semiotics is an integrative approach, to section off a particular area of perception and declare it the valid object of semiotic research means to deny semiotics its essential quality of interdisciplinarity. Therefore, from an epistemological perspective, the semiotics of the visual is not possible, and the attempt to define its boundaries is alien to the nature of the concepts and methods of semiotics. One more argument against the subject: Various pre-semiotic theories followed a common pattern. That is, they attempted to apply concepts of great generality (such as form and content, space and time, and more recently, structure and function) to the main spheres of sensorial perception, sometimes producing pretentious statements or apparent laws. (It is sufficient to cite here Fechner’s law concerning the auditory, or the laws referring to color in respect to visual phenomena.)
Semiotics has not put any of these laws under discussion but has questioned the legitimacy of specializing knowledge, of introducing distinctions that only a gnoseologically rigid, clear-cut model of sensorial perception founded on the psychological model sustains. Obviously, the ear does not see in the way the eyes do as a sight organ; neither do fingertips hear; nor does the nose feel, etc. But the fact is that each time a sign is perceived, a semiosis begins in which the ‘absent’ accompanies the present. In other words, the sign presents sets of complex mechanisms – the mechanism of memory in the first place – that restore the syncretism of the real. The dominance of one sensation (visual, auditory, olfactory, etc.) cannot be denied, but not to the extent of eliminating all others or especially their integration in the whole idea or feeling or both together. Semiotics does not introduce an integrating principle from the sphere of the known into the sphere of the cognizable, but starts out from the pragmatic observation according to which everything is in motion and everything is interaction.
The interdisciplinarity of semiotics is thus an epistemological condition deriving from the need to consider interaction in its complexity. Semiotics proposes a heterogeneous, instead of homogeneous, model and tries to describe it not through reductions but through eliminating them. In the case mentioned above, the elimination is of psychological reductions; but elimination is valid for sociological, ideological, historical, etc. reductionist models.
Considering semiotics as the theory and practice of mediations (Nadin 1981), I naturally had reservations when initiating semiotic research in the visual. Although my research follows the line of Peirce’s semiotic, it integrates results from Saussure’s pre-semiotic, structuralist model and from his successors, especially the French semiologues. No matter how much we think we can determine our own existence, there comes a moment – such as the one when I started my work in America – when certain adaptations to reality are necessary. Working at an excellent school, whose object is design (which many in and outside the school consider a visual domain par excellence), I came to understand that the chance to verify my own model was given a rebours. Either the visual could be separated from the continuum of the semiotic field – and in this case, semiotics would follow its traditional course of specializations, denying its universality as envisioned by Morris – or this separation would be impossible and research would be justified through its negative results (in the Hegelian sense of negation). These appeared to be the logical extremes.
I was still unaware of the fact that semiotics, in the sense founded by Peirce, transcends the dualistic model, that a third is possible, and that this third is itself a semiotic result: in the sphere of the visual, the nonvisual whole is rediscovered. Different hierarchies between the visual and nonvisual proposed in various cultural contexts or anthropological models correspond to semioses dictated by pragmatic reasoning. The written word exercises a social action of stabilization and simultaneously evidences an important political function. Under conditions in which media are diversified, the word’s role changes; images less associated with language play a more and more important role. The written word blocks interaction but invites interpretation in time. The image, transmitted with the aid of communication systems supported by computer technology, reduces interpretivity but permits interaction in a way never before attempted or utilized.
The fact that the visual, like all other components of the semiotic field, is a crossroads for all that the visual is not does not automatically mean that specialization is reconfirmed as a paradigm of knowledge. The way in which the nonvisual is known and understood from the perspective of the visual is different from the way verbal language is known and understood by linguists, or space by a geometrist, color by physicists, texture by chemists, etc. Mediations in the field of the visual are nonvisual by implication: word associations, olfactory, tactile, or other associations. Interpretation of the sign by sign – in a process in which we ourselves become, in the act of interpretation, signs and elements of mediations – brings the concrete visual back in the universal perspective.
These methodological observations are intended only to explain the general framework in which the articles that follow were conceived, the framework of research, more precisely. It was inevitable that each author start out from his or her specialty: linguistics, design, art history, philosophy. The result is a departure point. The research, whose results are partially presented in these articles, does not propose to negate semiotics from within by application contrary to its nature, but to establish what happens when for methodological or other reasons (such as those acting in a modern society that is extremely fragmented due to specialization) one attempts to section the object of study of semiotics.
Anticipating several results to be presented herein, I can say that they converge toward the conclusion that the visual, as an integral part of the general semiotic field, evidences ever-greater importance in our time. The continuous deterioration of language – which many, confusing cause and effect, still associate with low-quality education – has as one of its necessary results the transition from word-dominated to image-dominated communication. It would be excessive to enter here into the details of the semiotic processes that mark the transition from the civilization of literacy to what I call ‘the civilization of illiteracy’ (Nadin 1983). The social division of labor is only one of the factors that need to be considered, and specialization (which the criterion of productivity makes necessary) is a consequence of labor division. Thus in the field of the visual, new specializations lead to segmentation that is deeper, harder to overcome, making the integrating procedure of semiotics all the more necessary – but all the more difficult.
Actually, nothing would justify this entire project if not the facts that professionals in the visual – graphic designers, architects, film/video artists, newspeople, painters, computer graphics professionals, and others – sense the fragmentation of their specialties and that mediation through visual signs often escapes their control. Obviously, the solution is not the return to pictographic culture or mytho-magical images, but the integration of complementary perspectives, such as those of Western and Far Eastern cultures. In practice, the techno-scientific activities in the two cultures often meet and continue to influence each other more than their competition in the marketplace leads us to think.
Working on the language of television (a nontraditional area for an art historian), Gregor Goethals attempts to see to what extent semiotic concepts are merely a new name given to traditional concepts in art history or if the former represent a means of investigation and evaluation better adapted to her object of study. It is easy to understand her insistence on the category of function, which she applies almost in the structuralist sense used by Mukarovsk&Mac253;, while aiming to place the entire discussion in the semiotic perspective. (Peirce is invoked as a terminological orientation point.) From its beginning, television has had problems with formal categories. After McLuhan, it became quite evident that this new medium presupposes its own evaluation criteria, and that the influence it exercises would be extended to manifestations that are not telegenic through their own nature. Politics discovered television’s mediating (i.e., semiotic) action before semioticians determined the rules to be applied. Electoral campaigns have been decided through television. Facts and events are hidden or omitted. On the other hand, facts and events occurring outside our direct sphere of action become familiar. As a medium, television has assumed new aesthetic functions and will exercise direct influence on our future life and work in its relation with computers (image manipulation, retrieval from immense data bases). Although Gregor Goethals limits her study to the relationship between the semiotic tactics of religious and political ‘communicators’, the possible implications in other spheres are easy to discover and consider.
Having preoccupied herself for several years with the aspects of written language, Naomi Baron extends the semiologic model of the sign, also introducing the semiotic function but from a perspective different from that of Gregor Goethals. She distinguishes between various levels of representation while pursuing her argument, according to which iconicity is not a property of the sign itself but a relational concept. She states: ‘Iconicity in any system of representation – be it language or art – can only be defined modulo another variable: the people producing or perceiving the sign.’ Baron’s definition of representation is very encompassing. The bottom line is: ‘Words in human languages represent experience.’ Since the definition is so general, she must keep under control the concepts used in discussing differences between various forms of representation. In future attempts to use the components she identifies in representation (content, shape, participants), it will have to be proven that they are not a remake of Saussure’s distinctions or of similar semiologic paradigms. Naomi Baron understands that representation is tested in communication. Applying her model to the typology of artistic representation, she makes an attempt at interdisciplinarity, which is actually the only characteristic shared with the other contributions to this issue. Here, I would like to point out that while divergent in premises and conclusions, the results of our activity are based on this shared understanding of interdisciplinarity. It might not be enough to configure a school of thought – and there is no need to regard this issue as representative of a newly formed group – but it is a critical characteristic.
The questions that Nikhil Bhattacharya raises are of principle: How adequate is verbal language for communication in a world of individual, subjective experiences? What are the shortcomings of visual representations? Will understanding visual language help us better understand verbal language? He relates his inquiry to the increasingly important role of computers and their languages and the need to understand the referential, iconic, and symbolic aspects of verbal and visual constructions. While occasional semioticians do not have time to explore basic issues, Bhattacharya positions himself in the philosophical realm. It is not useless to say that, whether following Saussure or Peirce, it is impossible to understand the perspective from which they work without understanding the philosophical foundation. Removed from the philosophical context, either of the two systems is only a collection of strange words. Philosophical foundation should not be seen as a goal in itself, and this makes Bhattacharya’s contribution distinctive. When talking about iconic elements in visual language, he discusses the role of convention in order to discover that iconicity is a relative quality of representation. Previous discussions (especially Eco’s on iconic representation) have missed this point. It is no accident that, although computer scientists use icons for making their machines more user friendly, Bhattacharya asks whether iconicity is available in human language and furthermore, how iconic encoding-decoding takes place. No doubt that for someone unfamiliar with “computerese’ or what are called ‘buzzwords’, his discussion of the typical ‘garbage collector’ will present some problems. It just happened that when LISP, the artificial intelligence computer language, was first presented to the public, the expression ‘garbage collector’ produced a big laugh. As a formal language device meant to help in controlling the amount of memory used, the ‘garbage collector’ presents not only technical aspects but also very important semiotic aspects. The example that Bhattacharya discusses belongs to meta-language. Once again, interdisciplinarity was accepted as a necessary premise.
An important segment of the entire research has concentrated on the various aspects of graphic design education and activity. It should be pointed out that Thomas Ockerse (in collaboration with Hans van Dijk) elaborated a course in applied semiotics that has been a requirement in the Graphic Design Program since 1977. Since its inception, the course has integrated the results of research carried on in the United States, Europe, and Japan and has in turn become a source of research, an experimental laboratory, and a viable context for testing results. Thomas Ockerse is an example of an artist with high semiotic awareness. The greatest part of his creative work deals with sign processes. Ockerse has very good conceptual, semiotic discipline and an original way of involving it in generating images with poetic value. One of his concerns is the relation between various types of signs – visual, verbal, musical. Attracted by the Peircean semiotic approach, Ockerse not only classifies signs but also shows how classifications can be broken, proving that dynamic continuity (what Peirce called synechism) is just as important a part of his semiotics as his definition and typology of sign. It is, I believe, rewarding to see how semiotics becomes alive. Even if, at the level of the metaphor, conceptual discipline is no longer entirely possible, we can learn so much more about semiotics.
Claire Taylor’s study concentrating on ‘noise’ in visual communication is not surprising. Again, we have the occasion to observe that what we call noise is, among other things, the interaction between visual and nonvisual, between various codes of a given culture. The expression of the conflict of values in this idea, together with the suggestion of the formative role played by noise (to a certain extent, culture is the product of restrictions – conscious or not – imposed by noise), is pursued mainly in the printed media. The conclusion (slightly provocative) is not the result of accepting noise as a disturbance in communication, but of understanding the potentiating function that noise exercises. Instead of a septic, sterile, monotone image in which the personality of the author, photographer, or illustrator disappears behind depersonalized typography, Claire Taylor suggests expressive spontaneity, the graphic ‘accident’, ‘imperfection’ (the latter graphically controlled). Extending her research to other forms of visual expression (for example, Times Square and ‘noise’), she observes the semiotic phenomena through which noise is integrated in the message and accepted as a cultural value.
Contrary to the tendency of many researchers to produce new taxonomies adapted to the medium analyzed, Bethany Johns asks whether certain explicative models of nonvisual origin (in this case coming from the analysis of poetry) can be applied to the visual. Her procedure is obviously integrative, but it could not be anticipated to what extent concepts so different in nature from the visual (such as those concerning metaphor) can be applied. To read Vico, for instance, after Peirce’s semiotic model is understood and his theory of logical nature applied to the symbol means not so much to confirm Vico’s explanations but to integrate important results that have been ignored, if not rejected, by semioticians. At the level of reciprocal action between constituent parts, Bethany Johns points out different levels at which the image is constituted, the relation between the visual and nonvisual, the openness of processes through which associations, superimpositions, disassociations, etc. are produced. It is a domain of junction in which semiotics’ quality as meta-domain becomes especially clear.
Finally, from among the research projects I have been engaged in since the establishment of the Institute for the Semiotics of the Visual, I selected one of the themes everyone talks about, but not always with enough professional discipline. The meaning of the visual is part of the comprehensive subject called the meaning of the sign, and it is not possible to avoid, as a premise of research, the definition of the sign’s functions as they derive from the definition I adopted. Semiotics has been literally invaded by all sorts of specialists, scholars who never really succeeded in their respective domains. They recycle some of their older articles or lectures, introducing two or three terms with a semiotic flavor. (The magic word ‘meaning’ always shows up.) There is no intention to deal with this phenomenon, first of all because I believe that the best way to defend semiotics is to continuously improve the quality of our research and of everything we decide to publish. However, not to be aware of the confusion shadowing semiotics is as detrimental as contributing to the problem. This issue would not be possible if the participants in the session The meaning of the visual: On defining the field at the VII Annual SSA Conference held in Buffalo had not taken part in discussing our theses as generously as was done. It was our intention to allow as much time as possible for discussion and the three hours open for discussion provided us with the feedback from our colleagues that we wanted and looked for so much. Our special thanks go to David Lidov, Wendy Holmes, Donald Preziosi, and Elaine Nardocchio, to mention a few. The discussion encouraged us to propose to Thomas A. Sebeok publication in a special issue of Semiotica. The fact that he overcame an initial circumspection, as expressed in our beginning correspondence, is just one reason for thanking him for making publication of this issue possible.
Mihai Nadin (1981). Zeichen und Wert. Tuebingen: Gunter Narr Verlag. – . (1983). The civilization of illiteracy, Semiotics 1981. Deely and Lenhart (eds.). New York: Plenum.
- Human-Computer Interaction
- Post-Industrial/Post-Literate Society
- Ubiquitous Computing & Digital Media
- Art & Design
Semiotics of the Visual: On Defining the Field. (Mihai Nadin, Ed.) = Semiotica, Vol. 52, No. 3/4 (1984), Amsterdam: Mouton, 1985.