Language Engagement at the Level of Syntax: Assessing Metatalk and Task Types in SLA

While task-based instruction is considered as the one of the most effective way to learn a language, it is oversimplified on various grounds especially in teachers’ implementation of the approach in practical terms. Different variables may affect how students are engaged with the language and also with the task. The present study was designed to investigate language related engagement on the basis of metatalk; talk about the language, and task typology. To this end, 80 intermediate participants were assigned to four homogeneous groups on the basis of their proficiency level. The groups were given four different types of the tasks namely; jigsaw, dictogloss, test reconstruction, and translation in order to examine the role of metatalk and task-typology in creation of language engagement opportunity. Participants’ language related engagements were measured by evaluating syntactic devices used in language related episodes in their performances. The statistical analysis revealed that there were significant differences across groups. Specifically, the translation task had the most potential for creating language engagement opportunity and jigsaw task created the least language engagement opportunity. It is concluded that task implementation and task design affect learners’ language engagement at the level of syntax. Implications for testing and material development are provided.


Introduction
By reviewing the literature one would see that the trends in research on classroom discourse analysis has undergone a change.The input-output driven approach in which teachers' input and feedback and the kind of interactional modification occurring within the course of an interaction were seen as the driving force for learning to happen.In the holistic approaches, language has a semiotic role that brings all relevant factors into account including society, the kind of scaffolding and collaboration provided in the task performance and the metatalk and intersubjectivity created as students oriented towards task with different motives and social backgrounds to make the joint ownership of task.Critical approaches which present new horizons for researchers involve locating the classroom in the larger context of society as a whole and assessing how classroom particulars are formed by not only the instructional intentions, but also by larger social, economic, political and cultural aspects (Mirzaee et al., 2010).
Learner language production is affected by many factors.According to the sociolinguistic approaches and activity theory, by having the opportunity to create the joint ownership of task, the way that learners may operationalize the same learning tasks might be different from one learner to another since learners with different goals orient themselves toward the learning task in a different way.Hence this may lead to the creation of different learning activity on the side of the learners (Mackey, 2006;Nita & Gardener, 2005;Nassaji & Fotos, 2004).
There is a distinguished history on the efficiency of form focused tasks in engaging students in language learning.While the efficiency of focused tasks is well established, the efficiency of unfocused task in promoting learner's interlanguage is open to research which was the purpose of the present study.Specifically, this study aimed to evaluate which kind of task provided more opportunities for metatalk as it is believed that metatalk has a facilitative role in learning since it primes acquisition processes by creating opportunities for noticing (Ellis, 2003;Schulz, 2001).
Tasks have been highlighted in teaching English as foreign language (EFL) and English as a second language (ESL) as a pedagogical tool that provides a medium for the language learners to notice natural and communicative use of target language by negotiating the meaning they intend to make.The reason is that through negotiation of meaning, the learners notice form and meaning relation and notice the gap between their current interlanguage and the target language they are learning which is achieved by the negative input and feedback which in turn provide noticing opportunities (Pica, 1994).However, several studies suggested that by focusing on communicative aspects, one cannot guarantee a sufficient development of target like competency (Mackey, 2006;Ellis, 2001).Learners could convey meaning but with non-target like morphology and syntax.Long (as cited in Mayo, 2002) proposed a reactive focus on form (FonF) intervention in which certain language features can be noticed when students engage in meaning based tasks and noticing these language features are inevitable for meaning conveyance.Other researchers have favored a proactive FonF which is advanced reform of the task by the teacher so that certain language forms are noticed by the students (Ellis, 2001).There were various attempts to evaluate the potentials of different types of tasks in creating opportunities for noticing to take place.Whereas other studies worked on the type of task by itself, the present study focuses on potentials of different task types (text reconstruction, dictogloss, jigsaw, and translation) on creation of opportunities for the occurrence of metatalk.
Metatalk is defined as a talk about language and the knowledge behind it, is called metalinguistic knowledge (Mayo, 2002) Metatalk is operationalized as learners' attention to form through the identification of language related episodes (LREs).LRE is defined as any talk in which students are engaged with commenting on the language they are producing, ask about their language use and correct or have others to correct their use of language.Ellis (as cited in Mayo, 2002, p. 201) defines LREs as the cases in which students talk about language forms explicitly.This talk is created when students are performing meaning based tasks which means they negotiate about the efficiency of the language forms in conveying the intended meaning in an accurate and coherent way.
Studies show that not only different classes and schemas of human cognition but also linguistic forms are prototype structures.Studying prototype structures can provide fruitful insights on the role that the metalinguistic knowledge has in learner performance.According to Hu (2002), inner structure of the prototype categories influences acquisition and the use of such knowledge because metalinguistic knowledge is explicit knowledge of linguistic forms, classes and form-based meaning relations within classes.There is evidence that prototypicality of a category is evaluated by relative frequency and saliency of members of a category.The prototypical categories are learned more easily, processed more quickly and accurately.This study aimed at evaluating the potentials of different task types in creating noticing opportunities of these prototypical categories.

The Research Question
Considering the points mentioned in the preceding section about the metatalk opportunity and task typology and their role in learner production, the following questions were set: 1. Is there a significant difference between the targeted syntactic devices used by each group with regard to the type of tasks and metatalk opportunity?2. If there is any difference, which task leads to more metatalk opportunity?

Participants
The participants in the present study were 80 non-native male speakers (NNS) at the intermediate level of language proficiency whose L1 was Persian.They were all between 18-23 years old.The participants were approximately at the same level of language proficiency as their oral ability was measured on a) an interview which was designed on the basis of the curriculum through which they learned English and b) according to Foreign Service Institute (FSI) rating scale (Flucher & Davidson, 2007).Participants with the score of one standard deviation above the mean and below the mean were selected as the target participants.The reason why intermediate participants were selected was that in intermediate level, form and meaning compete with each other to have the attention of the learner (Hu, 2002) and within this competition, meaning is given much more attention (Hu, 2002).By manipulation of task conditions (task type and metatalk opportunities), it is possible to influence L2 learners' priorities in attention allocation.

Interview
FSI rating was used to ensure homogeneity of participants on the basis of language proficiency, so any significant differences could be attributed to group membership variable rather than preexisting differences.Students with one standard deviation above and below the mean were considered as intermediate participants.In order to have a reliable source for decision making, experienced raters were selected and briefed on the procedures of conducting the interview.Two raters were asked to attend the interview session and evaluate participants' performances at the interview session.The inter-rater reliability was also calculated.The interview was designed on the basis of the content of the materials that students had covered during their education in order to increase its content validity.The participants' performances were measured and evaluated according to the FSI rating scale.In order to assess the homogeneity of the subjects of this study, an interview was conduced by two trained raters.Participants' performances were measured according to the FSI rating scale to test participants' speaking skill.The data on interview are provided in Table 1 and Figure 1.

Inter-rater Reliability in the Interview Assessment
The scores given by each rater on interview were entered into SPSS program and a correlation coefficient was calculated between them.These are estimates of the reliabilities for each set of ratings as they were assigned by the raters in the interview sessions.Since these estimates are the reliability of each single set of ratings, and since two or three sets of raters are likely to be higher in reliability when taken together, an adjustment to find the reliability of larger numbers of ratings taken together would be logical, possible, and advisable.The Spearman-Brown Prophecy formula can be used for this purpose (Brown, 1996).This result gave a conservative reliability estimate (that is, it is safe and not likely to be an overestimate).Since the reliability estimate is close to one (0.98) as the ideal estimate, it could be concluded that the interview and the rating procedure adapted by trained raters could be a sound source for reliable information and decision making.
Since the mean score in the interview was 66.11 and the standard deviation was 5.34 (Table 1), the scores of one standard deviation above and below the mean were selected as the scores representative of intermediate level participants in the following way: X+ 1SD → 66.11 + 5.34 = 71.45X -1SD → 66.11 -5.34 = 60.77 Thus, the participants whose scores were within the range of 60.77 to 71.45 were selected for the purpose of this study.
Since the focus of the study was to examine task type and metatalk opportunities they create, the tables related to reliability assessment of the interview are not reported.

Task
To reach the objective of the study, four tasks were designed observing the following parameters:

Task Evaluation and Selection
Five experienced teachers were asked to evaluate the sample tasks that the researchers designed on the basis of form and meaning related episodes which participants encountered in their course books to ensure that participants all were familiar both with forms they encountered within the designed tasks and discourse management strategies required in performing the tasks.This was done to ensure that participants had the explicit knowledge required to run metatalk so that any difference in participants' performance could be attributed to task conditions rather than participants' language background.The same teachers were given a questionnaire (appendix 1) as a checklist prepared based on the guidelines provided by Nunan (1989) for evaluating the appropriateness and efficiency of the tasks.The tasks that the teachers agreed on as being the appropriate and efficient tasks on the basis of the questionnaire were selected for further analysis (Appendix 2).

Evaluating Task Complexity to Ensure Comparability across Tasks
To establish a common qualitative measure for complexity of tasks, the researchers had used a checklist taken from Geldenbuys (2011).This checklist (Table 2) was used to ensure that the tasks were of the same complexity and to ensure comparability of performance across task types.The same teachers in task evaluation and selection stage were asked to rate the tasks.The questionnaire had 6 items which were answered by the same five teachers acting now as raters.They were asked to rate the tasks on the basis of their qualitative judgment guided by the checklist (Table 2) mentioned on a 5 level scale ranging from the simplest (1) to the very complex one (5).The inter-rater reliability was taken as an index of reliability in qualitative evaluation of task complexity.Since this evaluation is based on the correlation between the rates given by the teachers on each task, the obtained index of correlation (0.632) was taken as a token for the degree of reliability.The Spearman-Brown Phrophecy formula can be used for this purpose (Brown, 1996).Using the statistics provided in Table 3 and putting them in the Spearman-Brown Phrophecy formula, the researchers calculated the reliability index of 0.89 for ratings in task complexity evaluation.The 0.89 degree of correlation between the five raters displayed a close to one reliability index in the decisions made by the raters over the complexity of the tasks presented to students.

Task Content Consistency
In order to compare the effect of the four tasks on creating a quantitatively different medium for matatalk, the researchers designed tasks with a same content.The topic of the tasks was selected on the basis of participants' answers to a questionnaire designed to check topic familiarity.First, the input for the dictogloss was provided and its difficulty level was calculated using readability index to ensure comprehensibility of the input for the participants and its appropriateness to participants' level of proficiency.Then, the input for the dictogloss was deprived of the most frequent linguistic devices which occurred in LREs in Mayo' study (2002) to form the input for text reconstruction task which the students were supposed to reconstruct collaboratively by engaging in metatalk that this research intended to elicit.The input for dictogloss was translated to Farsi to construct the translation task.For the visual jigsaw task, the content of the translation task was transformed to pictures or ideograms so that the same content is elicited as the other three tasks.Since visual jigsaw task was a kind of task that promotes meaning negotiation in nature rather than focus on form, to make the comparison across the tasks possible, students were ask to write a summary of what they talk about as they performed the jigsaw task.The purpose of this summary writing was probing students to focus on language which was the focus of this study.

Tape Recorder
The researchers tape recorded the participants' interactions.Because the presence of tapes and audio-taping students' performance might influence their performance, the teacher was asked to run a warm up to reduce students' stress.

Procedure
Prior to conducting the research, the participants were given a checklist (topic familiarity questionnaire) in order to determine their background knowledge to find the tasks they were more confident at.First, the sample tasks were taken from the materials that the participants had studied.Second, the teachers were given the sample tasks and asked to evaluate the tasks according to the checklist described in the task evaluation and selection section.Third, five teachers were asked to rate the complexity of tasks according to Table 2 to make cross tasks comparability possible.Each group was given one of the tasks (dictogloss, text construction, translation task and jigsaw).Participants were instructed to perform the tasks.To avoid misunderstanding in the task procedure, the researchers provided instructions needed for the tasks in participants' mother language (Farsi).The tasks were accomplished using English.Similarly, since students had very limited ability to carry out metatalk and since the presence of metatalk opportunities were important not the medium (language), researchers allowed the participants to use their L1 during the task performance.So their occasional use of L1 was not excluded from the study as far as they are considered as LREs.The researchers did not impose any time restriction as long as the learners completed the task in a fifty-minute class period.The expectation was that learners would use language forms and lexical choices to convey their meaning clearly, coherently, and precisely as possible.According to Ellis (2003), this talk about language (metatalk) supports second language learning and the process of internalization which is moving from intermental to intramental activity.The performances of the four groups were audio recorded for further analysis of syntactic devices used in their interaction.

Design
Based on the transcriptions, the researchers analyzed the data in terms of the frequency with which different syntactic devices (Table 4) were used by participants in each group.One way ANOVA was used to see if there are statistically significant differences in the use of targeted syntactic devices in each group.If there is any difference, the difference could be attributed to task type and their potentiality in creating opportunities for metatalk.
Tasks performances were analyzed for the distribution of turns which is the number of turns allocated to discussing LREs.The turns in which LREs were produced were selected.These turns were recognized by students' linguistic engagement in the following taxonomies.The episodes taken from the data of the present research are as follow: 1 Asking for meaning of word and expressions Asking for English equivalent Punctuation (Since the spoken data were used to answer the research questions, the punctuation was nil.) Note: The list of categories is taken form Mayo (2002) Having this background and acknowledging the open area of research on the type of task and the metatalk and their facilitative role in second and foreign language acquisition, the researchers designed this study to shed lights on the mentioned areas.

Results
The main objective of this study was to find out whether there was any statistically significant difference between language related engagements across four tasks that were designed to create opportunities for metatalk.
The review of the descriptive analysis of the syntactic devices across four tasks types, as summarized in Table 5, reveals that jigsaw had the lowest mean (i.e.2.50) and Translation task had the highest mean (i.e.16.15) with a corresponding difference in related standard deviation values.So one way ANOVA was used to analyze participants' performances across four task types.The purpose was to investigate if there is any statistically significant difference in tasks' potentiality in creating a medium for the language engagement by engaging participants in the use of different syntactic devices.By reviewing the p-value of the test in Table 6, the researchers found that there was a significant difference between the task types in terms of syntactic devices participants used in different task types.That is, there is statistically significant difference in participants' language engagement at the level of syntax.
Consequently, the hypothesis that there is a statistically significance difference across four tasks types in terms of syntactic devised used was confirmed.But to find the exact locations of the differences, the researchers ran a post hoc comparison.As shown in Table 7, a post hoc comparison was conducted to answer the second research question by evaluating the differences among the tasks in terms of their potentials in creating noticing opportunities.There was a significant difference in the mean score of noticing opportunities across tasks.The post hoc comparisons using Tukey HSD test indicted that the mean score for translation task (M=16, SD= 4.05) was significantly different from the mean score of all other tasks.The post hoc analysis indicted that the p-values for all comparisons except for the comparison between dictogloss and jigsaw were significant since the values were less than 0.05 (p< 0.05).So, there are significant differences across all tasks except for jigsaw and dictogloss comparison in terms of their potentiality in creating a medium for the use of different syntactic devices.
By checking the mean differences, the researchers found that translation task (with the mean value of 16) and the jigsaw task (with the mean value of 4) had the most and least potentiality in creating a medium for language engagement at the level of syntax.These two tasks are the two extremes of the language engagement at the level of syntax continuum and the test reconstruction and dictogloss are at intermediate levels as they had the mean of 10 and 5 respectively.To rank the tasks in terms of their potentiality in engaging learners linguistically in syntactic devices, the translation would be at the first rank, test reconstruction would be at the second rank, dictogloss would be at the third and jigsaw would be at the fourth rank.
Although the results presented show that the difference obtained in frequency of syntactic devices across tasks were unlikely to occur by chance, it does not tell us much about the magnitude of this difference.One way to find out about the magnitude of the difference is to calculate an effect size statistics (Pallant, 2001).The procedure for calculating and interpreting Eta squared (one of the most commonly used effect size statistics) can be obtained by dividing the sum of squares for betweengroups (1757.650)by the total sum of squares (2491.950).The resulting eta square value is 0.70, which in Cohen's terms (as  Pallant, 2001), would be considered a large effect with substantial difference in the amount of potentials of each task type in creating noticing opportunities in the form of metatalk.

Discussion
Language engagement at the level of syntax analysis aimed to evaluate language related episodes in which students were engaged in as they tried to understand others and make themselves understood.In this kind of analysis, the number of turns participants produced on the syntactic devices that frequently occur according to Mayo's study (2002) were measured.
Having analyzed the data, the first research question was answered as the researchers found a significant difference in the mean number of the metatalk turns on syntactic devices across different task types.In order to answer the second research question, the mean differences were analyzed in post hoc comparison by cross checking of the significance levels.The researchers found that translation task with the highest mean and the jigsaw task with the lowest mean were the most and the least potential tasks respectively in creating a medium for language engagement at the level of syntax via metatalk in English and the text reconstruction and dictogloss as intermediate levels with the mean of 10 and 5 respectively.
In line with the results of the present research, Faerch & Kasper (as cited in Machida, 2011) declare that Translating from L1 to target language may also support the language acquisition.When learners write in SL/FL, they use their interlanguage (IL) strategies such as generalization, systematic construction, or paraphrase (Faerch & Kasper, 1983cited in Machida, 2011).In so doing, learners try to find the equivalent linguistic forms in their current interlanguage.When learners are engaged in translating form L1 to target language, they pay close attention to L1 text and analyze and try to fill the gap by an appeal for help outside of their own interlanguage resources.They may look up dictionaries to find words, discuss with other learners about the text or they may appeal for help from the teacher or even use websites or other books to get more information or make up for the lack of background knowledge.This interaction with various recourses expands the learning opportunities students may have.Therefore, translation activities provide opportunities for the learners to notice how form and meaning are related.Translation can provide opportunities for learners to learn about the language, and help them to bridge the gap between their L1 and target language.Translating helps learners to notice and observe the linguistic systems in L1 and target language, and also how the two languages convey meaning (Machida, 2011).
Text reconstruction was the second mediator in language engagement in this research.In line with what the researchers discovered in the data analysis, Thornbury (1997) argued that in text reconstruction, shortage of learners' current linguistic competence is displayed as they reconstruct the text in target language.Thornbury (1997) concluded that translation activates bottom up processes that are needed for attention to form; the processes are absent in comprehension and communicative activities.According to him the matching that occurs in text reconstruction has the most effective benefit because in this task learners compare their performance with the model.This provides them with positive evidence of the language features are not acquired yet.This noticing process changes input to intake and cause the learners to have a restructuring in their developing linguistic competence.
Dictogloss task was the third mediator in language engagement in this research.Findings support the idea that dictogloss involves both top-down and bottom up processing.In top down process, learners decode the input by paying attention to content words and by activating background knowledge, whereas in bottom up process learners encode the input by grammaticizing the lexis which requires lower level syntactic processing.Another advantage of the dictogloss is that different learners will notice different language elements depending on their current interlanguage, their interest and motivation (Thornbury, 1997).
The reason why jigsaw task has been found as the task with least potentials for the occurrence of metatalk opportunities is that in jigsaw task the input is visual and not verbal.In this kind of task, learners use the areas of their interlanguage system in which they feel confident and they avoid the problematic areas.So they have used the structures that create no problem in their meaning conveyance.Since the research was about the nature of different task types, it becomes clear that the more focused the task is, the more metatalk opportunity there would be.
The results of the study revealed that there was statistically significant difference across task types in terms of their potentials in creating meta-talk and noticing opportunities for language engagement at the level of syntax.Investigating the task potentials only in terms of the level of language engagement would mask an important issue which was task engagement.
One of the problems with any descriptive or experimental research is that the researcher codes the relevant or desired data which support the research questions or uses research data to illustrate an analytical category or support an argument.So, the researcher presents the segments of discourse data that show the manifestation of the use of particular syntactical form or peer scaffolding in task accomplishments (Norton & Toohey, 2004).But, according to Norton and Toohey (2004), these types of data do not elegantly and neatly fit in with the theoretical construct of task-based language use.The participants messily engage with the task, sometimes attentively and sometimes perfunctorily.At some moments, some members of the group appear to be dealing with other matters, they appear to work together sometimes and not at other times.There seems to be varying degrees of engagement and involvement with the task collectively and individually; in short, there is a continuum of 'on taskness'.This is open to research to see how the representational choices and selections we make from our collected research data unavoidably involve us in making direct or indirect statements of our interest and our view of reality.
The need for language awareness and the approach to implement this acquisition facilitator device have been a challenging issue in ESL and EFL.Providing opportunities for learners to challenge their interlanguage is out of question since this challenge makes learners go beyond the extremes of their current language proficiency and form various hypothesis in their minds about the way target language works through trial and error process.By language awareness, learners test the hypothesis that they have made about the target language.This also helps learners in reflecting on use of the target language metalinguistically, which, by itself, can have potential effects on how they use what they already know to discover what they do not know.By the new approach towards learning, learners are encouraged to take the responsibility in learning so that they take an active role rather than reactive role.But teachers do not leave the learners alone in this discovery.One of the ways in which teachers can facilitate what seems at first to be too difficult is through evaluating the medium by which they are paving the way.One of the mediums by which the language learning way can be paved is pedagogic tasks.
In line with the findings of the present research, different tasks have different potentials in stretching the learners' interlanguage and providing opportunities for new adventures in language learning.One of the areas which is still open to research is assessing how different kinds of tasks lend themselves to create noticing opportunities for suprasegmental features including stress and intonation patterns.Since Iranian students are exposed to segmental features including vocabulary and grammar, it can be a sound piece of research to expose students to suprasegmental features and their functions and study the ways by which this exposure (either through explicit or implicit instruction) could be used to promote learners' current interlanguage .Then, a step can be taken ahead and the degree of potentiality of each task in creating more metatalk opportunity and attention on intonation or other suprasegmental features of the learners' target language could be evaluated.
Task attainment is the end result of not only functions but also the interactions between various cognitive, metacognitive learning strategies, and motivational strategies.In SLA, motivational and cognitive issues and aptitude determine students' overall learning success.(Dornyei & Koromos, 2000, p. 278).As mentioned, one of the limitations of this study was individual differences including the idea of multiple intelligences and language learning strategies which may contaminate the results of the study.It is highly recommended to conduct a set of studies to find the effect of the individual differences on the cognitive factors including noticing taken place for syntactic devices.
As mentioned before, task-based instruction has been oversimplified on various grounds.Mere application of the tasks in English learning classes will not guarantee the accomplishments of the objectives of task-based instructions.Various factors may affect the end result of language learning programs to different degrees.The design and the type of different tasks not only affect the quantity of learner language and the degree of language engagement on the side of learners but also the quality of engagement with the task itself.Since tasks have been used as an instrument for language learning, manipulation of the tasks affects the amount of noticing learners have over different language elements.Besides, tasks can engage learners in such a way that the language learning becomes the side product of their engagement and definitely their enjoyment.

Figure 1 .
Figure 1.Descriptive statistics on the interview

Table 3 .
Correlations for calculating inter-rater reliability on task complexity

Table 5 .
Descriptive statistics on syntactic devices across four tasks types

Table 6 .
One way ANOVA on syntactic devices across four task types

Table 7
in turn, creates opportunities for meaningful language use and language development.Non or partial engagement is not generally discussed.However to dismiss or ignore the messiness is, ironically, to do what some see as a problem in experimental psycholinguistic research which may give a portrait of classroom reality that is different from what really happens, (p.260).
Norton & Toohey, (2004)believe that:Naturally occurring data of the kind do not lend themselves to easy representations of classroom reality.Pedagogic tasks are meant to generate participation and engagement which