A Cognitive Semiotic Study of Students ' Reading a Textless Image versus a Verbal Image

This study explores fourth year college students' content retrieval from reading textless versus verbal images. Furthermore, it examines the extent to which the respondents comprehend and understand them .The procedures include selecting an image from the internet, designing a written test with its rubrics and exposing it to jury members to ascertain its face validity. The data of this study were analyzed according to the adopted of Peirce's model (1931) of ' icon, index and symbol'. The researchers find out that most of the results were in favor of the verbal images, captions restrict readers' creativity to infer any additional message.


Introduction
It goes without saying that language is the best means of communication that human beings use in all walks of life.Communication, thus, involves a message whether oral or written or visual that is sent from an addresser to an addressee .Holme (2009, p. 11) argues that "language can be seen as a mirror of mind and more as one evolving response to how we conceptualize the world through mind and body."Language is one of many ways that knowledge is represented.The present paper investigates how the readers can read and interpret both textless and verbal image, via answering the questions which read as follows: What are the key signs/ objects that help to comprehend and interpret the literal meaning of textless and verbal image?What is the connotation of textless and verbal image?And to what extent do the respondents comprehend both textless images versus verbal image?
Visual and verbal domains constitute cognitive semiotics.Bal (1991, p. 19) mentions that these domains ''interpenetrate, influence, and inform each other.''They are linked with each other by intersemiotic translation which is defined by Jakobson (1971, p. 261) as '' an interpretation of verbal signs by means of signs of nonverbal sign systems.''Written texts and visual arts represent different ways of producing signs and meanings.Those ways do not contradict each other, but they complement each other and have the same activities (Grishakova, 2010, p. 175).Rhetorical items are elements that assist understanding the different ways of message production (Frascara,p. 38).Hall (2003, p. 87) states that modes of representation made the ambiguity, even if the image is supported by a detailed essay or a small caption.As images have form, which is the text's instrumental aspect, whereas content is ' the core of meaning ', which is anything left after the formal devices (Collins, 1991, p.113), transferring meaning is a process between the encoder (image) and the decoder (the reader), the image has a 'content' and the reader must do ' interpretive work' (Huhtamo, 2003, p. 1).A study presented by Adham S. Ahmed (2012) among several previous related studies under the title " A Semiotic Analysis of the Iconic Representation of Women in the Middle Eastern Media", which examines how advertisements psychologically influence its audience comparing it with the way that linguistic function of speech requires action from its receiver.This study follows Kress and Van Leeuwen's Framework.But the present study follows Peirce Model in his trichotomy of signs.

Cognitive Linguistics
Cognitive Linguistics is concerned with investigating the relationship between human language, the mind and sociophysical experience" (Evans et al., 2007, p. 2). Bednarikova (2013, p.14)  -language is not an autonomous cognitive faculty.-knowledge of language is from language use.These three hypotheses (an) is omitted oppose to the dominant approaches of syntax and Semantics represented by generative grammar and truth-conditional Semantics (Croft & Cruse, 2004, p.1).Evans (2015, p.2) describes human beings as "minded creatures in nature "who save their thoughts in their memory to use it through communication via language.Making thought overt is the cairn of communication.Language on occasions is imperfect in conveying thoughts perfectly well, because it depends on meanings as the core of communication.Another crucial principle is that "human language is symbolic in nature because it arises from the association between a phonological representation and a semantic representation" (Langacker, 1987, p. 11), there is a drastic difference in handling the symbolic nature of language by Langacker's approach and that of saussurain in that, although Cognitive Linguistics agreed on that the copula between form and meaning is slightly arbitrary, it recants that language is " totally structured arbitrarily " ( Johnson 1987as Cited in Ibarretexe & Rojo, 2013, P. 11).
Cognitive Linguistics refers to the "crucial role of intermediate informational structures in our encounters with the world" Geeraerts and Cuyckens (2007, p. 5) maintains on the same page that language is regarded as "repository of world knowledge, a structured collection of meaningful categories that help us deal with new experiences and store information about old ones".This archive helps human beings to deal with new meaningful experiences.
As Cognitive representations have codes, content and transferring mediums, Semiotics deals with signs, they share the same concerns (Herzfeld & Lenbart, 1980, p. 493).

Semiotics
The true sign theory goes to St. Augustine (AD 354-430), though he did not use the term Semiotics.He divided signs into three divisions: a natural sign which is literal found in nature; and a conventional sign that is related to humans like gestures and words, then, he adds the scared signs like miracles which are all message from God. John Locke (1632-1704) was the one who introduced formally the study of signs into philosophy and called it Semiotics for the first time.Then, this idea was completely ignored till the late nineteenth century as Ferdinan de Saussure and Charles S. Peirce paved the way to the understanding of Semiotics and signs (Danesi, 2004, pp. 7-8).Kim (1996, p.1) states that when someone looks around himself, he will find many things [sun, sky, tree, paints, photographs...etc.],all these things are signs even man himself, and by those signs we understand our world as each sign stands for something.Fiske (2002, p. 40) shows that in Semiotics the receiver or reader plays an active role.He aptly stated that: Semiotics prefers the term 'reader' (even a photograph of a painting) to 'receiver' because it implies both a greater degree of activity and also that reading is something we learn; it is thus determined by the cultural experience of the reader.The reader helps to create the meaning of the text by bringing to it his or her experience, attitudes, and emotions Sebeok (2001, p. 3) says that semiosis as a phenomenon that can distinguish live things from inanimate things.Semiotics studies the functions that are applied by signs ranged from simple bodily signals to advanced symbolic structures as words, those functions are; to signal the existence of sign, to communicate messages within the species, and to model incoming information from the external world.

Sign
The center of Semiotics is sign.Hence, the study of sign is called Semiotics and some people called it Semiology.They are used interchangeably.According to Pierce (1955) a sign may be termed as an icon, an index, a symbol.An icon is "a sign which would possess the character which renders it significant, even though its object had no existence", an index is "a sign which would, at once, lose the character which makes it a sign if its object were removed, but would not lose that character if there were no interpretant", a symbol is "a sign which would lose the character which renders it as a sign if there were no interpretant" (Peirce, 1955, p. 104).(1955, p. 104-105) states that any representamen that can substitute its object is an icon.A sign can be iconic when it primly represents its object by its similarity regardless of its mode of representation.An iconic representamen can be classified as a hypoicon, when there is should be a substantive.Any kind of material image, such as painting, has a conventional mode of representation, but without label it can be termed as a hypoicon.

Pierce
He adds that hypoicon can be virtually classified depending on the mode of firstness that they share as, images which share simple qualities; diagrams which has a completely dyadic, or regarded as dyadic relations among the parts of one thing; metaphors which " represent the representative character of a representamen by representing a parallelism in something else".Peirce proposed that communicating an idea directly is through an icon, while establishing indirect method in communicating it should rely on the use of an icon.

b. Index
Pierce (1955, p. 107) it is a sign which does not so much refer to its object by any similarity or analogy, nor by associating with the general object's characters, but as it has a "dynamical connection" with both of the singular object and with the sense or the person's memory which works as a sign.
Pierce (1955, p. 108) indicates that indices have three characteristics that differentiated them from other signs; they do not own important likeness to their objects; they refer to singles, single unities, single combinations of unities, or single continuum; they have blind indenturing way to give attention to their objects.If is not impossible, it may be arduous to give an infinite pure index or to find a sign without having the "indexical pure index".Peirce furthermore adds that "Subindices" are signs that presented by real relevance with their objects such as "proper name, personal demonstrative, or relative pronoun or the letter attached to a diagram".As an index is a representamen which has its representative character because of its individuality, so if it is not an individual, it is not an index for example, a man who carries a rolling gait is an indicator that this man is a sailor.So, any sign directs the attention is an index.

c. Symbol
Pierce (1955, p. 114) clarifies that symbol is a sign which its "representative character" composes exactly in its interpretation is determined by its rule.It is a law, or "regularity of the indefinite future", and its interpretant should have the same prescribing, the absolute "immediate object, or meaning", and that law should be incarnated in individuals, and describe some of their qualities.Thus, a symbol may consist of an index, or icon.A child walking with a man and the man points by his arm to the sky and says "there is a balloon".The main portion of the symbol is the pointing arm, and without it the letter will have no information.But if the child says, "What is a balloon", and the man answers, "It is something like a great big soap bubble", he puts the image as a part of the symbol.
Pierce argues that icon does not own a dynamical contacting with the object it denotes, i.e. it is not connected with its object, but they share some qualities.On the contrary, the Index is actually associated with its object, but association has nothing to do with interpreting mind, except signaling it.The Symbol is associated with its object according to "the idea of the symbol-using mind".

Meaning: Denotation and Connotation
Barthes is the first linguist who first set up the analysis of meaning by the "orders of signification".Denotation is the first order of signification (Fiske, 2002, p. 85).Denotation and connotation are two levels of meaning based on signs, codes, and are culturally relative.Denotation is less culture specific than connotation.Knowing how denotation and connotation work is important for the analysis of graphic design (Barnard, 2005, p. 35).Denotation is" the dictionary meanings of a word...It is the primary sign of a natural language" (Marten & Ringham, 2006, p. 62-63).It beckons a simple, un ambiguous( explicit) and direct nexus between a sign and its referent.It occurs when a signified is so obvious to everyone in the same manner (Kim, 1996, p. 24).The denotation of image is what literally found in the image as colors, shades, the number of the elements that are found, and so on ( Nicholas & Price,1998, p. 3).For Barnard (2005, p. 35) connotation is greatly a powerful type of meaning.It is the dictionary meaning which gains an additional cluing resulting from the context in which it is applied (Marten & Ringham, 2006, p. 51).It is the subjective meanings that are arbitrarily added to signs, because of interpreters idiosyncratic experiences and feelings ( Kim,1996, p. 24), i.e. it is the interface between the interpreter and the culture when the values and beliefs of culture interact with the text or image to generate meaning (Barnard,2005, p. 35).In images, Allen (2005, p. 29) clarifies that the denotation of images is the first level of signification : the image itself as the signifier and what is the picture about is the signified.Connotation is a second level of signifying system which utilizes the first level as its signifier and binds another signified to it.Connotation can be described in even one word such as (romantic, purity ...etc.), on the contrary, denotation can take many words to be described.The connotative of image is the different interpretations associated with the image as white color has a number of possible connotations (peace, purity, calmness, and so on).
The difference between denotation and connotation is that denotation is concerned with "what", while connotation is concerned with "how".Although all signs have both denotative and connotative meanings, the denotation to the connotation proportion is varied according to the type of communication (Kim, 1996, p. 25).Roland Barthes developed the work of Hjelmselv who stated that the same concept would have different meaning if it expressed in different ways, denotation and connotation do not relate to language only, but relate to images also, saying that images, too, have two strata of meaning and for him visual denotation refers to concrete things, and places, and visual connotation refers to abstract concepts which is culturally shared meanings (as cited in, Leeuwen, 2005, p. 37).Semiotics allows us to describe the process of connotation, and also directs us beyond the field of semiotics, because we can study messages, the receivers of messages, and context in which significations take place, but we cannot study the referent because it is outside semiotics domain (Leeuwen, 2005, p. 31).Hence, the researcher agrees with Fiske (2002, p. 86) who states that: "connotation is the human part of the process; it is the selection of what to include in the frame, of focus, aperture, camera angle, quality of film, and so on.Denotation is what is photographed, connotation is how it is photographed".Hence, connotation is subjective as we are un aware of this level.

What is an image?
An image is "a mental picture that you have of what sb/sth is like or looks like... a copy of sb/sth in the form of a picture or statue" (Joanna, 2010, p. 775).It is difficult if it is not impossible to define image, Mitchell (2005, p. xiii) defined image as "any likeness, figure, motif, or form that appears in some medium or other", Mitchell wrote that Jaroslav Pelikan refers to the use of icons as "a social movement in disguise" where doctrinal vocabulary is used to rationalize an essentially political conflict.Today, images have a great power in our world.Heidegger (1977) proposes that we live in the age of the world picture, which does not mean a picture of the world but the world is conceived and grasped as pictures (as cited in Mitchell, 2005, p. xiv).Hence, "the images is a simple, unstructured collection of qualities which are sufficient for the sign formed by the image to function as a sign" (Jappy, 2013, p. 113).
Images can denote equally a mental imaginary entity as memories and a physical object as painting or superscription.It can imply the visual and verbal arts or demonstrate a verbal motif, a metaphor, or the formal totality of a text as a verbal icon.Images have quasi-logical situation in Peirce's sign formation "icon", "symbol", and "index" forms the semiotic relationships (Mitchell, 2005, p.2).
Images differ from the objects in significant ways, in how they are conjoint.In this sense, Mitchell (2005, p. 51) maintains that Clarcke suggested two types of images; sensory image which refers to "a representation or iconic natural sign of the 'external' object", and material image which refers to " a naturally occurring image which is a public object observation by more than one individual and causally related to the object of which it is an image" (Mitchell, 2005, p. 55).Ryle argues that sensory image is not itself an object of perception because in this case, it requires a second image to perceive it then a third one and so on, these images will not be as signs interpreted as indicators of the existence or similar to the object, if what is perceived is the object; as table or chair not the sensory image.To interpret a sign as representing a certain object, it is necessary to make such a comparison between sign and its object, e.g. a table and a visual image of it (Mitchell, 2005, p. 51-52).Material images include mirror images, shadows, T.V. images, and photographs, they share the feature of having straight compatible to the objects of their images, and by this feature we can differentiate material from sensory images (Mitchell, 1994, p. 503).

Images and words
In the late 1960s, many experiments performed to find that the left hemisphere is the controller of all sequential, logical, analytical, verbal thinking, while the right hemisphere is the controller of more intuitive processing and nonverbal holistic thinking.But recent researchers recognize that both hemispheres dominated most activities (Olshansky, 2008, p. 20).Mcclanahan (2000, p.15) states that the work of the writer and the painter begins with careful observation, the painter tries to instantiate both vision and mood through visual elements layers with deep observation, and so the writer tries to captivate image, sound, smell, taste through words layers also with deep observation.Regarding reading and writing as the most complex mental processes which is linear in nature, videlicet, that the reader moves from left to right, from individual letters, words, sentences to paragraph in inductive way, i.e. from parts to whole thinking.On the contrary, reading image is a nonlinear activity, which means that the reader of an image follows the deductive way, from whole to parts to whole again i.e. reading the whole image before wandering to the image parts to gain their full meaning and their contribution to the whole (Olshansky, 2008, p. 21).
Olshansky adds that images can be treated as a universal language to each culture or region, and the language of image does not need translation, so both of them used to make meaning but they function differently, though we can use one to perform the other (Olshansky, 2008,p. 22).

Methodology of the study
As far as methodology is concerned, the following procedures are conducted: 1. Reviewing literature that is concerned with textless images and verbal images; 2. Surfing the internet so as to select 1 social image from the website to be exposed to jury members; 3. Exposing these images to jurors in College of Fine Arts, and University of Baghdad to check the adequacy of these images to the students (see appendix B); 4. Constructing a written test and a scoring rubric and exposing both of them to jurors so as to check their suitability, reliability and validity; 5. Conducting a pilot study on a sample of fourth year college students.The reasons behind the choice of the fourth year college students are they are presumably reached a good level in reading comprehension in general and reading any images (textless and verbal images) in particular and they suitably represent the population of the study; 6. Exposing the sample of the study to a written test so as to highlight the different readings and interpretations of the two kinds of images among students; 7. Laying out a practical schedule for obtaining the sample and using the instrument of the study; like getting permission from the Department of English concerning timetabling and administering the test in its final form; 8. Administering the final form of the test on the population; and 9. Scoring and analyzing the results of the final test (see appendix A). 10.Adopting a qualitative and quantitative study in analyzing students' responses to reach the results.

Analysis
The denotation and the connotation level of this image is achieved through a test of two parts.The first part consists of textless image and the second one has the verbal one.There are four questions all of them have the same wording except the second question which has a different one for each part.The questions results are treated via scoring rubrics (see Appendices A).The questions are as follows: The aforementioned questions are given to the respondents so as to collect data for the subsidiary question/s, and to provide the researcher with the denotative/ literal meaning of images, through identifying the main signs in this image.
Then, providing the connotational meaning through evoking the right emotion of this image whether it is positive or negative; then, they support their answers with reasons.In question four, they attempt to infer an additional massage which could be applicable to the image.The researchers analyze the image and each question in each part separately.The analysis of the image explains the right choice of emotion and then analyze the reasons behind that choice.

Data analysis
The main signs of this image are a bird with one leg.According to Peirce's model the suggested analysis for the third image is put on (Table 1); icon refers to chick, index indicates direct relation and the symbol is life.Whereas, Icon refers to one leg, the index indicates direct relation and the symbol is disability.The respondents who identified all the main signs at the first part is 66%, whereas 20% of the respondents identified them partially and 14% failed to identify them.As for the second part, the researcher found that 48% managed to identify them fully, 52% has partial identification, and no respondent failed (see Figure 1).So, in this image, the percentages were in favor of textless images, by comparing 66% to 48%.46% of the respondents suggested a suitable title for the image, whereas 32% estimated relevant titles (see Table 2), and 22% gave irrelevant titles (see Table 3).The titles given by the respondents are written as it is, and as follows: Table 2.The respondents' relevant titles of the image Relevant titles 1.Strong bird.2.You should continue your life and challenge the difficulties.3. Hope and volition.4. The strong leg. 5. Hope and ambition.6. Challenging the disability of a bird.6. Life with hope is the best.7. A strong fighter.8. Fall seven times, stand up eight.9. Hope is something beautiful.10.Never give up.11.No disappointment with this life.12. Hope in life with one leg.Table 3.The respondents' irrelevant titles of the image Irrelevant titles 1. Wounds of a child.2. Freedom.3. A determined small bird.4. Resistance of a bird. 5.The impoliteness to the inflection and continuation in life.6.The poor bird.7. The hopeless bird.
In the second part, 16% of the respondents managed to pick up the right word from the caption that suggests a title to the image as 'hope ', whereas 10% the respondents chose to write all the wording of the caption or a part of it.But 14% of the respondents failed to choose a word from the caption or chose words outside the caption.Due to the given percentages, (see Figure 1).Hence, the second part has the lowest level of interpreting/understanding.As far as the third question is concerned, and regarding the first part, 84% of the respondents managed to evoke the right connotation, whereas 16% of them failed, among which 18% failed to give relevant reasons and 82% managed to give relevant reasons.As for the second part, 88% of the respondents succeeded to get the right connotation and 12% failed, among which 46% failed to give relevant reasons and 54% managed to give relevant reasons.In this part i.e., verbal image, the relevant reasons given by the respondents influenced by deciphering the meaning of the textless image and the verbal one, as they alternated between choosing linguistic key signs or non-linguistic key signs, i.e., choosing signs from the image or from the caption (see Figure 2) As to the fourth question, which requires the respondents to give an additional message to the image, in the first part, they achieved 74%, 16%, and 10% regarding N/IC, CC, and RC respectively.While in the second part, they achieved 98%, 2%, and 0% concerning N/IC, CP, and RC respectively (see Figure 2).These results show that most of them cannot infer any additional massage from this image.The caption restricts their imagination and thoughts.2012), who maintains that messages vary in classification in accordance with the social status as well as sexual one.

Appendices
considers Cognitive Linguistics a discipline of the Cognitive science that describes "the mental structures and processes connected with language knowledge."In the last decade, a huge number of research emerged under the name of Cognitive Linguistics, most of them focused on Semantics, Syntax, and Morphology.Croft and cruse (2004, p.1) proposed three major hypotheses that guide the Cognitive Linguistics to language.They are as follows:

Q. 1
Identify the key signs/ objects that describe the image.(Both parts) Q.2 Suggest a suitable title for this image.(First part) Pick up a suitable word that suggests the title for this image.(Second part) emotion evoked in you when looking at this image is a. positive, why? b. negative, why?Q.4 Give an additional massage that you can infer/highlight from this image.

Figure
Figure 1.Percentages of denotation meaning