Assessing Iranian Student-Teachers’ Problem-Solving Ability: A Comparison between Conventional and Problem-Based Assessment

The advent of problem-based assessment has opened the question of whether or not conventional approaches to assessment are truly a representative of students’ knowledge (Lampert, 2001). In an attempt to find out about the suitability of problem-based tests in evaluating student-teachers’ knowledge of teaching English, a problem-based test was designed and administered to 33 MA students who were selected through the quota approach at Payame Noor University. Comparison of the results of the problem-based test and conventional test through paired-samples t-test revealed that teacher-students gained higher scores in the conventional tests; however, the results of the semi-structured interview with the participants revealed that they considered problembased test a true measure of their knowledge and favored it over conventional assessment due to its relevance to content and context of the course, its motivating nature, and its dynamic structure. This study can have pedagogical implications for language teachers and material developers.


INTRODUCTION
Assessment is in no certain terms a significant aspect of education so much that Farhady, Jafarpoor, and Birjandi (1994) pointed out that successful education is incomplete without proper assessment.Recent approaches to learning and assessment such as problem-based learning have aimed at solving problems which had dominated traditional and conventional forms of education (Savery, 2006).Conventional forms of education as stated by Boud and Feletti (1997) targeted students' memory rather than comprehension; therefore, retention of newly learned materials was in spectrum.In line with conventional teaching, conventional assessment evaluated students in terms of what they remembered rather than what they could perform (Ansarian, Adlipour, Saber, & Shafei, 2016).Problem-based learning (PBL) as an innovative approach to learning stood against traditional approaches to learning which harshly shoved down a load of content to the throats of the students (Kwan & Lee, 2009).Consistent with PBL, problem-based assessment attempted to measure what students could perform rather than what they had memorized for a short period of time.
Performance rather than competence is of prime significance to student-teachers.In this study, student-teachers are students at postgraduate level who are getting prepared for teaching at educational systems in future.Based on the regulations of Iranian universities, these students should pass a course entitled 'Practical Teaching' to qualify for teaching English at the Iranian educational system.Through the course, the students should demonstrate their ability to handle authentic teaching issues that occur in language classes.However, whether the structure of the course per se leads the student-teachers to comprehension rather than memorization has not been delved into.Therefore, the researchers were motivated to compare the performance of these students in terms of traditional assessment and problem-based assessment.The students were also interviewed for their perception about problem-based assessment.
Assessment is a delicate concept in education.It cawn determine the future of the individuals and give direction to their goals.Therefore, it is of prime significance to assess students as precisely as possible.However, there seems to be a problem with the approach adopted to assess student-teachers' teaching skill.The student-teachers' performance is conventionally measured through giving lectures about teaching methods and taking multiple-choice-item tests, among others.It has already been proved that multiple-choice-item tests are not suitable for assessing one's ability to solve problems, and considering that practical teaching in language classes is replete with occasions where teachers should be able to think of a way to solve problems; therefore, the validity of current testing system is under question.Inappropriate tests, as they seem to be, may lead to selecting teachers who are not capable of teaching language classes; therefore, the new generation of language learners may be affected.As this issue requires immediate attention and action, this study was conducted.

Purpose of the Study
This study aimed at investigating the issue of assessment in the course 'Practical Teaching' from two different perspectives.Firstly, by measuring student-teachers' current performance through a multiple-choice test and then by comparing it to their performance in problem-based tests.Secondly, the study aimed at delving into student-teachers perception about both conventional assessment and problem-based assessment to find out which is more suitable for them.

Research Questions
The following research questions were proposed for this study.1.Is there any significant difference between the student-teachers' performance on conventional university test and problem-based tests with regard to the course 'Practical Teaching'?2. What is the perception of student-teachers about problem-based tests for the course "Practical Teaching"?

Significance of the Study
It is very significant to select students as teachers who are indeed capable of delivering effective education.This would not be possible without proper assessment.In addition, it is important to hear the voice of young generation of teachers to find out what they think about the merits and demerits of conventional and problem-based assessment.This can help teachers to prepare more effective tests for the students.

REVIEW OF THE LITERATURE
In general, problem-based learning has a long history in education.Its pioneers at McMaster University in Canada began implementing this approach in 1970s (Hmelo-Silver, 2004).Primarily, problem-based learning belongs to medical education though it was soon adopted by many educational systems with slight alternations.By definition, problem-based learning is an innovative and collaborative approach to learning in which students attempt to learn content and master learning skills (Hmelo-Silver, 2004).Students learn content by making use of their higher order thinking skills (Saver, 2006), a process which often leads to retention rather than memorization.One of the main underlying assumptions about problem-based learning was that the materials learnt through higher order thinking skills have a higher chance of retention.Boud and Fletti (1991) assumed that 90% of the content learnt by the students through conventional education is subject to being forgotten.This indicates a need for an approach which could solve the problem with retention of knowledge.In a problem-based process, students learn by analyzing and decoding a real life problem (Larsson, 2001).They generate hypothesis about the solution and prepare a plan to reach the solution.The role of the teacher is diminished to a tutor who usually helps with learning skills rather than content.
The concept of problem-based assessment should be studied from two various perspectives.On the one hand, stu-dents' knowledge who go through problem-based learning should be assessed through problem-based tests (Kwan & Lee, 2009), one the other hand, problem-based assessment targets students skills and performance rather than what they have accumulated in their brain.Therefore, problem-based assessment is considered to be a true measure of what students can really do (Savery, 2006).In problem-based assessment, test takers receive problem scenarios instead of question.They should view the problem scenario as real-life issue and attempt to solve it by considering the context of the study (Hung, 2006).
In general studies that have accorded focus to problem-based assessment have reported lower scores among the students in immediate posttest (Barrow, 1986;Boud & Feletti, 1997;Savery, 2006); however, in terms of delayed posttest students with problem-based education have outperformed those in conventional classes.This issue signifies the importance of problem-based education.

Research Design
This study has a mixed-mode design, as it includes a quantitative and a qualitative part.The quantitative section of the study has a quasi-experimental design (due to non-random selection of the participants through quota approach).It also has a within-subject design.The participants' performance was measured using quantitative data (scores).The qualitative section of the study made use of purposive sampling method.Semi-structured interview questions were prepared by the researchers based on Hung's (2006) 3C3R model in problem-based learning.

Participants and Setting
An intact class of master's level students who were in the process of the course 'Practical Teaching' in the current academic semester (Semester 1, 2017) were used to pursue the purpose of this study.Running power analysis, the researcher realized that at least 33 participants are required for the quantitative section of the study.Gender, age, religion and socio-economic background were not among the variables in this study and focus was accorded on participant at one particular setting-Payame Noor University.One of the limitations of the study is that the researchers could not collect data from various types of universities in Iran, though the course practical teaching is being taught in most types of universities, i.e., state, private, and distance universities.To solve this problem, we assumed that the participants in the study are similar to those in other universities in the Iranian setting, as they all enter different universities through the university entrance exam.Therefore, quota sampling cab best describe the sampling procedure of the study.
As for the qualitative section of the study, 10 participants were selected and interviewed.As consistencies were found in the answers and as the researchers reached the pint of data saturation, no more participants were interviewed.

Assessing Iranian Student-Teachers' Problem-Solving Ability: A Comparison between Conventional and Problem-Based Assessment 43 Instruments
Three main instruments were used in this study.The first one was the final achievement test administered to the participants as the course's final term exam.This multiple-choice-item test was used as the basis of comparison.inorder to make sure the test has construct validity two experts in the field were asked to review the test.
A piloted version of a researcher-made posttest was used to assess student-teachers' problem-solving ability related to the real-life situations they encounter in the language classes.Item analysis was conducted to check item discrimination (ID) and item facility (IF) of the test items.In addition, construct validity of the test items was checked by a) using two experts in the field, b) selecting the questions based on the topics covered in the course book and running factor analysis.
The last instrument is the semi-structured interview.The interview questions were selected based on the constructs of problem-based learning approach as stated by Hung (2006).This could ensure that the interview questions have construct validity.

Procedure
Consenting participants were distinguished from non-consenting ones by distributing consent forms in order to conduct an ethical research.The researchers collected these participants' final term scores for the course 'practical teaching'.One day after the final exam, the participants were given the problem-based test.The results of the problem-based test were compared to the final term scores (as administered by the university) in order to answer the first research question.
As for the qualitative section of the study (second research question), Grounded Theory (GT) was used to collect samples of the participants utterances about the method.This inductive approach to qualitative data analysis is based on Corbin and Strauss (2008).The participants were interviewed after taking the problem-based test.Using the semi-structured interview, the researcher delved into the participants' perception about the new assessment procedure, i.e. problem-based tests.

Exploratory Data Analysis Results
As an assumption of parametric study, distribution of data was checked to find out whether or not parametric tools should be used in the study.
As can be observed in Table 1, ratios of skewness and kurtosis over their perspective standards are within the range of +/-1.96;thus, normal distribution of scores was assumed.
In the next stage reliability of the tests was gauged through Cronbach alpha formula.The results of the Cronbach alpha test was equal to α=.81 and α=.79 for the conventional test and the problem-based test respectively.
As for the results of inter rater reliability test, there was a very high and significant correlation between the two raters who rated the samples (r =.913, p =.000); that is an acceptable agreement percentage of 83.3% between the two raters.

Comparison Between Conventional and Problem-Based Test Results
The following null hypothesis was postulated based on the first research question: H0.There is no significant difference between the student-teachers' performance on conventional university test and problem-based tests with regard to the course 'Practical Teaching'.
The participants scores in the conventional test was compared to their scores in the problem-based test using paired samples t-test, as the study had a within subject design.
As can be seen in Table 3, there is difference between the conventional test scores (M= 22.75, SD= 3.06), and the problem-based test (M=12.24,SD= 12.30) The results of paired samples t-test (t(32)=4.78,p=.000 ≤.005, SD=12.61,[6.0430-14.987]indicate that the difference between the scores on two tests is meaningful; therefore the null hypothesis was rejected, and it can be assumed that there is a significant difference between the student-teachers' performance on conventional university test and problem-based tests with regard to the course 'practical teaching'.

Perceptions About Problem-Based Tests
The second research question of this study was: • What is the perception of student-teachers about problem-based tests for the course "Practical Teaching"?In order to delve into the participants' perception towards problem-based assessment, Corbin's and Strauss's (2008) Grounded Theory (GT) was used to analyze the data.GT is an inductive approach to data analysis.Samples of participants' utterances were selected.Based on the samples the main themes were extracted.
As for the interview questions, we made use of 3C3R model suggested by Hung (2006) to design the interview questions.This model attempts to optimize the constructs of problem-based learning and assessment by making sure all the constructs have been included in the test.The constructs include content, context, connection, referencing, research, and reasoning.Therefore, the semi structured interview questions were designed regarding these constructs.
The participants believed that the questions given to them at the posttest well represented to the content of the course book.However, mostly they were unaware of the way the theories could be met in practicum.
"What we read was in test, but I did not know that it could be the answer to the question." It seems that the participants expected to encounter a test which would measure their memory span; however, problem-based tests assess students' ability rather than memory.
"I had memorized a list of language teaching methods and their features.Useless for me in the test." In terms of context, the participants agreed that the problem-based test was well-situated in the context of language classes: IJELS 5(3):41-46 "The situations in the test are exactly like real classroom situations.For example, selecting a technique to correct students." Some learners also mentioned more detailed features of the test which were observed in the test such as culture.
"Speaking English is different from Farsi; The two cultures are different.For example, you talked about writing a letter to friend to keep my dog.This is related to Western culture.I enjoyed it." On the other hand, some participants considered contextualization a demerit and favored lesser degrees of contextualization.
"If I did not answer the question correctly, it is because I did not know the context…very technical for me." Seven participants acknowledged that there was a logical connection between the questions and the outside world.
"If the test was a real language class, it could test experience and intelligence." As for researching, the participants believed that a problem-based test should not be confined to the boundaries of conventional tests in terms of time and research tools.
"We should open books to find our answers.Even in class, when I have problems, I ask friends." They also believed that they needed to know their goal before they started the search for the answer.Thus, decoding comes before research.
The participants favored the collaborative nature of assessment in problem-based setting, and considered it a form of real-life assessment: "In real life you ask people questions; you search online.In this test I could do it.That is, I can express a blessing, a necessity."

DISCUSSION
As it was observed in this study, there is a difference between the results of the student-teachers' scores in conventional tests and problem-based tests.The participants' scores in the conventional test were significantly higher than their scores in problem-based test.It is generally assumed that students gain lower scores in problem-based tests (Barrows, 1986).Waters and McCracken (1997) assumed that traditional approaches to assessment, e.g.multiple-choice items and true/ false questions cannot discriminate poor and good students from each other, whereas problem-based assessment distinguishes students based on what they can do based on their knowledge.Indeed, conventional forms of assessment are accused of being narrow and scores are result of rote memorization (Koschmann, Glenn, & Conlee, 1999).Many students who simply pass in conventional forms of assessment fail to solve the ill-structured problems in problem-based tests.Krajcik, Blumenfeld, Marx, and Soloway (2000) pointed out that one of the merits of problem-based tests is that they are contextualized.Consistent with Krajcik et al. (2000), Savery (2006) stated that problem-based assessment guides the students in certain domains.In this study, the participants perceived the problem-based test as contextualized; they stated that the questions were based on real-life situations that may occur in language classes.Hung (2006) argued that the problem scenarios should be context-and goal-specific; otherwise, the students will move from one context to another to find the answers.
Problem-based assessment is a collaborative approach to solving problems.As a result, dealing with a large number of students in a test is cumbersome (Waters & McCracken, 1997).However, Hays and Gupta (2003) noted that the collaborative nature of PBL is one of the most significant constructs of this method.Collaboration is a source of inquiry for the students.The current study dealt with rather a small number of students (n=33); therefore, no problem was felt in dealing with the participants.In addition, the participants favored the collaborative nature of problem-based assessment and considered it an innovative approach to assessment.
Fletti and Boud (1997) mentioned that although many students are more used to conventional approaches to assessment, they usually have low retention of knowledge.As a One of the issues mentioned by the participants in the interviews was the connection between the questions in the problem-based test and with the context of the study.Hung (2006) assumed that connection is a significant dimension of problem-based learning and assessment which makes it meaningful.Authentic problems that students are exposed to in their real life are connected to the context and real life issues.The participants in this study acknowledged this issue as an issue which made solving the problems interesting for them.In line with this study, Hmelo-Silver (2004) stated that intrinsic motivation is the result raising students' interest by giving them meaningful and authentic problems to solve.

CONCLUSION
This study attempted to solve some of the problems observed in conventional approaches to assessment.As stated earlier in this paper, conventional assessment through multiple-choice items is subject to low retention.In order to solve this problem, more innovative approaches to assessment should be used.To this end, PBL was selected in this study.The results of the study through quantitative analysis revealed that although student-teachers' scores in problem-based tests are reduced significantly, they consider their scores a true indicator of their knowledge.In addition, the results of the qualitative section of the study revealed that in general the student-teachers have a positive perception about problem-based assessment and believe that conventional forms of assessment are not a true measure of their knowledge.
This study can have pedagogical implications for test designers.Test designers are in charge of creating accurate and valid tests that can target the constructs of the course.It seems that previous and traditional forms of assessment do not assess test takers' knowledge in terms of what they can do.Therefore, test takers can adopt PBL as a possible approach to making tests.Language teachers should also remind the learners of the significance of proficiency and skill-based learning over memorization.This can guide learners throughout their language learning experience.To achieve this goal language teachers can create authentic problem-based tasks in the classes and use them to increase learners' proficiency.
This issue can be subject for new research; it is significant to know how PBL can affect proficiency vs. competency, an issue with rare empirical evidence in the field of applied linguistics.It comes highly recommended by the researchers that the focus of future research should be on examining the effect of PBL on various (e.g., incidental vs. intentional) vocabulary learning forms.Future researchers may also wish to study the dynamics of collaboration among language learners in PBL classes.

Table 1 .
Distribution of the scores

Table 3 .
Mean score comparison

Table 4 .
Paired samples t-test; conventional test vs.problem-based test