Testing the Predictive Validity of the IELTS Test on Omani English Candidates ’ Professional Competencies

This study has investigated the relationship between IELTS testing and Omani English teacher trainees’ professional competencies by adopting a quantitative method for data collection. A total number of 94 graduate freshmen Omani English teachers’ IELTS, CGPA and their teaching professional competencies are collected. The results reveal a moderate significant relationship between IELTS and CGPA but a weak relationship between IELTS and teaching competencies. This study could contribute to the growing body of literature that aims to assess the construct validity of IELTS, and attempts to do so in the new terrain of teaching competencies. This study puts forwards recommendations for IELTS proficiency test in the Omani context.


Introduction
The IELTS and TOEFL tests constitute one of the requirements to gain admission to Anglophone universities.In Oman, the proficiency test has been made mandatory on potential English language teachers after they complete a 5-year English education program.The International English Language Testing System, known for short as IELTS is "designed to assess the language ability of candidates who need to study or work where English is used as the language of communication" (IELTS, 2001, p.1 cited in Feast, 2002, p.76).In other words, IELTS was initially designed as a proficiency test to gauge the prospective entrants' ability to communicate effectively within their new, English-speaking environment.However, since its inception, researchers in education and assessment have raised serious concerns regarding the IELTS's ability to yield accurate, reliable assessments of the non-native students' proficiency in the English language (Dooey & Oliver, 2002).The broader use of this test by higher education institutions in the Anglophone countries as a key requirement for the admission of non-native students suggests a possibly inadvertent attempt to use IELTS as an indicator of academic potentials.This situation has prompted Woodrow (2006, p.52) to call upon researchers "to investigate the extent to which current IELTS levels are predictive of academic performance in specific academic settings".A third and unprecedented use of this standardized examination appears to take roots in the Sultanate of Oman where an IELTS band score of 6.0 has been imposed by the Ministry of Education on any education graduate before they matriculate as professional English language teachers.This is in spite of the fact that these candidates would have successfully completed a five-year English Education program where English was the language of instruction and including a full-year school-based supervised practicum.So, this procedure, while ascertaining that the candidates have the adequate language skills, raises serious questions about the validity of such a standard examination in gauging the candidates' teaching skills or pedagogical knowledge.It appears to build on an unsolicited assumption which equates between the mastery of a language and the ability to teach it.
The misinterpreted purpose of the IELTS test is the motivation for this study which will further examine the issue of English proficiency tests, in particular, IELS and its accuracy as a predictor of English teachers' professional competencies.The extant literature on this topic will be reviewed in this study.Thus, I will begin with a review of the literature around the IELTS test and its predictive validity with regard to on students' academic performance.In addition, the wash-back effects of IELTS to the learning and teaching outcomes will be reviewed.Then, I will examine the research around the definition of English teachers' professional competencies.I will conclude this literature review with a focus on the research questions which will guide the study.

The predictive validity of IELTS test and its wash-back effect
There has been a remarkable increase in the numbers of expatriates seeking higher learning opportunities in universities in English speaking countries such as Australia, Canada and the United Kingdom.Regardless of the international circumstances that have placed the English-speaking world in the leading seat and the various motives behind this evergrowing educational tourism, the trend has brought to the fore the standardized English language examinations and turned them into a lucrative commodity and a subject of controversy.
One such standardized language assessment tools, is the International English Language Testing System (IELTS), jointly developed by the British Council, IDP (International Development Program) Australia and Cambridge ESOL (English for Speakers of Other Languages) Examinations (www.ielts.org).The test consists of four modules: listening, reading, writing and speaking respectively.Speaking may sometimes be examined before the three modules depending on the condition of where the test takes place, although Ahmadi, Ozek & Demirel (2012, p.17) suggest that IELTS testtakers "prefer having modules which they believe to be easier or more difficult either at the beginning or at the end of the examination", as it could affect their performance.In addition, the test is available in two versions: (a) academic, which is used for university admission purposes; and (b) general training, which is used for vocational training and immigration purposes.A score of 6.0 to 7.0 is commonly accepted at universities as adequate evidence of international students' proficiency for admission, although some colleges and departments may require higher levels.
Investigations to date question the eligibility of the criteria adopted by tertiary education institutions to measure nonnative students' proficiency in the English language (Dooey & Oliver, 2002).With caveat in mind that IELTS is used as an entry measure, it is of significance as Woodrow (2006, p.52) states "to investigate the extent to which current IELTS level are predictive of academic performance in specific academic settings".Predictive validity is defined with respect to the extent to which IELTS can predict academic success as measured by course grade.
The findings of these studies are far from uniform.Elder (1993), investigating the relationship between the language proficiency of 32 overseas students, as measured by IELTS, and their postgraduate diploma of education courses' performance, found a weak but significant relationship between them.The study offered no conclusive evidence about the predictive validity of IELTS.It, however, did "confirm evidence from previous studies that it is at low levels of proficiency that language makes a difference" (p.72), as it found that "the strongest level of agreement between test prediction and academic outcomes occurred at band 4.5 casts some doubt on the recommendation that a band score of 6.5 be a minimum requirement" (p.87).This result indicates that the required band score of 6.0 which has been imposed by the Omani Ministry of Education lacks a solid, research-based foundation.
In contrast, Tonkyn (1995) argued that "there is plenty of evidence that language proficiency is an important issue with regard to the academic performance of overseas students" (p.14).This finding implies that students who obtain high scores on any English proficiency tests such as IELTS or TOEFL are more likely to succeed in their future academic study.
In a 1998 study at the University of Tasmania, Cotton and Conrow conducted a pilot project with 33 international students to investigate the relationship between IELTS and academic outcomes.Data revealed no significant correlation between IELTS global scores and academic performance overall.This study appears to be unique for its inclusion of a number of subjects with IELTS scores lower than the band 6.In fact, two students whose scores were below 6 passed the academic year while several students who had scored 6 and 7 failed the year.This finding confirmed Elder's (1993) previous idea that the lower the band score level, the more significant the relationship between language proficiency and academic outcome.
As IELTS is made of four components, some researchers explored the relationships between these individual modules and the academic capabilities of the candidates.Cotton and Conrow (1998) concluded that "the reading subset scores, in particular, were best able to predict subsequent academic performance" (p.109).Another study by Kerstjens and Nery (2000) of the relationship between IELTS scores and students' subsequent academic performance found similar results.The finding showed that IELTS reading subtest (accounting for 8.4% and 9.1% of the total variance) positively attributed to English proficiency as measured by IELTS.This finding is supported by Dooey and Oliver's (2002) study on the predictive validity of IELTS.It looked at the relationship between the English language proficiency measured by IELTS and the academic performance of 89 students in business, science and engineering at Curtin University.The researchers found no significant relationship between IELTS overall and academic success across the disciplines, however, "the reading module is the best predictor of academic success and it was the only subtest of the four macro-skills to achieve a significant correlation" (p.51).In the same year, Feast conducted a similar study using multi-level analysis, and got a set of contradictory results.The results of her study show that there is a significant but weak relationship between English language proficiency measured by IELTS and international students' performance measured by their Grade Point Average (GPA).Thus, it is important to note that IELTS overall is not a good indicator to gauge students' academic performance, albeit its slight significance.This finding has been concurred by Woodrow's (2006) conclusion that English language proficiency as measured by IELTS was moderately predictive of 82 international students' academic achievement.Unlike Dooey and Oliver's (2002) work, the study found no significant relationship between the candidates' reading and their academic performance, but did find significant correlations between writing, speaking and listening subtests and the students' GPA.Similarly, the findings reached by Wilson & Komba (2012) confirm the existence of a weak significant positive relationship between English Language Proficiency (ELP) and academic achievement of 89 Tanzanian graduate students, who will shortly be joining universities locally or internationally.
The second area of research literature I am reviewing for this study focuses on wash-back and teaching/learning outcome.Wash-back refers to the impact of an assessment procedure on the teaching which takes place prior to its delivery.Both students and teachers tailor the classroom activities to the demands of the test.The present study focuses on IELTS preparation courses as it is crucial for students' future academic success.Research suggests that the IELTS students' learning activities are test-oriented in terms of (a) content; (b) format; and (c) strategies, as the IELTS teachers focus more on test content and techniques accordingly.The IELTS preparation courses, nevertheless, did not improve the students' language proficiency as stated by the teachers in Iran (Erfani, 2012).Therefore, it is crucial for the teachers to objectively aim to develop the language proficiency of students rather than merely passing the IELTS test.Extensive research concurs that the focus of both the teachers and students is on the test task when they are teaching IELTS preparation courses rather than the development of academic language proficiency (Berry and Lewkowicz, 2000;Lumley and Stoneman, 2000;Hayes and Read, 2004;Hawkey, 2006 andGreen, 2007).Such studies raise the issue of the extent to which teaching to the test, in this case IELTS as a preparation course, can be linked to the deterioration of the students' assessment scores, and consequently to the language proficiency development (Turner, 2004;Morton &Moore, 2005 andGreen, 2007).Therefore, IELTS preparation courses require investigation on this test's validity and reliability as it could significantly impact the students' obtained score, and consequently affect their future lives.In the Omani context, Rustaq graduate students are given IELTS preparation courses to be prepared to pass the IELTS test.

English teachers' proficiency/ competence
Teacher's proficiency can be judged by three conceptual dimensions, namely teacher competence; teacher performance; and teacher effectiveness (Westera, 2001;Passo, 2009).Teacher competence is not a straightforward construct that is generally understood from two perspectives: theoretical and operational.The theoretical perspective conceives competence as a cognitive structure that facilitates a specified behavior.The operational perspective, on the other hand, views teacher competence as encompassing their ability to cope with complex and unpredictable situations.In addition to the knowledge, skills and attitudes, this perspective covers metacognition, strategic thinking and presupposes conscious and intentional decision-making (Westera, 2001).According to Huntley (2008), teacher competence necessitates five components: "demonstration of thorough preparation, a sound knowledge base, effective classroom management, professional communication with a range of stakeholders, and an accurate sense of self-awareness" (p.125).The Omani participants in this study are pre-service English language teachers.For this group, linguistic proficiency is an added expectation of their teacher competence.Linguistic proficiency is defined by Elder (2001) as: encompass[ing] everything that normal language users might be expected to be able to do in the context of both formal and informal communication as well as a range of specialist skills.These specialist language skills include command of subject specific/metalinguistic terminology, on the one hand, and the discourse competence required for effective classroom delivery of subject content, on the other hand (p.152).Cummin (2001( , cited in Butler, 2004) ) adds to these ideas proposing that language needs to be understood in global contexts and local school discourse domains.Thus, in addition to the above-mentioned requirements of teacher competence, one can add language knowledge and a global perspective as expected competencies of the pre-service English language teachers.Based on the ideas outlined above, teacher competence involves teachers' proficiency in a range of attributes.Epstein and Hundert (2002) sum up the notion of teacher professional competence when they suggest that teachers require "the habitual and judicious use of communication, knowledge, technical skills, clinical reasoning, emotions, values, and reflection in daily practice for the benefit of the individual and community being served" (p.226).Turning to how teacher professionalism is assessed, Whitty and Willmott (1991) claim that teacher competence is firstly characterized by its ability to perform a task satisfactorily according to predetermined criteria.Secondly, it is characterized as encompassing intellectual, cognitive and attitudinal dimensions, as well as performance.These different characteristics to assessing teacher competence attracted considerable criticism, particularly in relation to their potential to narrowly concern about outcomes at the expense of performativity and accountability.There are some attributes in teacher competence which can be readily recognizable while others are difficult to recognize, for instance intellectual, and cognitive dimensions.They can be tested at present and at an appropriate level in individuals.Consequently, competence has been criticized as too atomistic; too regulative and too aligned to outcome-based approaches (Fullerton & Clements, 2002).Based on literature review, no study, to the researcher's best understanding, lists IELTS or any other standardized language examination as a tool to measure teaching competencies.Such tests are meant to measure both academic and general English Language proficiency.IELTS developers explicitly caution institutions that "IELTS band scores are not predictors of academic success or failure" (UCLES, 1999b, p.8).Thus, to examine the impact of IELTS on graduate students' academic performance, and explore the adequacy of having IELTS as indicators of English teaching proficiency for Omani English teacher candidates are the two core objectives of this study.For these purposes, the subsequent two research questions guided the study: 1. What are the relationships between English Language proficiency levels (as measured by IELTS scores) of the graduate teacher candidates and their academic performance, as measured by CGPA? 2. How adequate is IELTS in predicting teacher candidates' teaching proficiency?Out of 118 students who had graduated from Rustaq College of Applied Sciences in 2013, 94 teachers were employed.Those teachers have got GPAs ranged from 2.57 to 3.56.The rest were not employed, largely because they have not met the requirement of obtaining band 6.0 in IELTS despite the fact that they have been oriented and prepared to pass the IELTS test through IELTS preparation courses provided by the Department of English language and Literature in the college.From the 94 teachers, the records of 17 teachers' professional competencies were not found although their IELTS and CGPA were reported from the employment section.These nonexistence records of those teachers could be due to changing the job and find another place to work in or getting a scholarship to pursue their higher studies, hence their assessments were not completed.

Instruments
An official letter to the Ministry of Education signed and approved by the College dean of applied Sciences, Rustaq to facilitate collecting the required data.The IELTS and CGPA were collected from the employment section in the ministry and the teachers' professional competencies consist of seven indicators: Knowledge and understanding; Planning and lesson preparation; Classroom management; Teacher reflection and evaluation; CPD and professional characteristics and Competency in English were collected from the department of Educational supervision, English section.Each indicator has been awarded a grade: the highest ( 5) and the lowest (1).Then through the SPSS, the researcher could make the correlation between the needed variables for this research.

Data analysis
To answer the two research questions above, the quantitative and descriptive methods were used.For this study, Spearman Brown test was used to analyze any kind of correlation.As IELTS test scores are not interval scores but simple ratings, it is commonly agreed that the IELTS results are non-parametric data (Ordinal Data), which in correlating with GPA scores; they require a Spearman Brown test of correlation.Thus, Spearman Brown test through the SPSS program was used to investigate a) the relationships between English Language proficiency levels (as measures by IELTS scores) of the graduate teacher candidates and their academic performance, as measured by GPA and b) the relationships between IELTS, required by the Ministry of Education as a reliable criterion to measure teacher candidates' competencies and their teaching proficiencies as evaluated in the schools by supervisors from the Ministry of Education.

Research Findings and discussion
The results are reported for each research questions respectively.1) Is there any relationship between English Language proficiency level (as measured by IELTS scores) of the graduate teacher candidates and their academic performance, as measured by CGPA?The analysis has found a moderate positive correlation between CGPA and IELTS (r= 0.486).In plain English, the analysis suggests that those students who have high achievement during their school years (CGPA) tend to have similarly high scores in IELTS and vice versa.This correlation is statistically significant (p=000), suggesting that the relationship is not a mere random coincidence.This result comes as no surprise, as both exams, notwithstanding the many differences between them; appear to gauge the students' ability to master the language.The findings echo to some extent previous studies by Elder (1993), Feast (2002), Woodrow, (2006) and Wilson & Komba (2012), to name only a few.IELTS can predict the academic performance of Omani graduate students.However, for English teachers, language proficiency is considered an indicator of teacher's academic success as it plays an essential role in enhancing their teaching capabilities and enabling their students to understand the lesson once they graduate.Additionally, language proficiency is part and parcel of the English program in the College.Students joined the foundation program where they are taught the English language skills.During their studies, they took courses in the four skills such as 'Reading and Writing Skills Development', 'Listening and Speaking', 'Grammar and Usage' and 'Advanced Reading and Vocabulary'.Furthermore, it is included as one of the seven indicators of evaluating Omani teachers' proficiency in the schools and, hence, the relationship between Omani teachers' competency in English (CE) and their IELTS scores is statistically significant as shown in Table 2.All the 77 teachers obtained band 6.0, 6.5 and 7.0 in IELTS as an indicator of their English teaching proficiency; therefore they are accepted and employed as English teachers.However, the correlation between IELTS and TPC is 0.194 as shown in Table 3, which means a weak relationship between the two variables.It is evidently shown that IELTS test measures the language proficiency but not the teaching proficiency of the teachers.Yet, it is necessarily needed for evaluating the teachers' language proficiency but alone cannot be a predictor of the Omani teachers' capabilities in teaching.There should be another assessment measuring the professional competencies of Omani graduate teachers who are about to teach.
IELTS is deemed to be a gate-keeping function for employment in Omani context.It was intended to ensure all teachers exiting the English program would have clear and identifiable minimum levels of English Language and to address concerns about levels of language proficiency demonstrated by Omani beginning teachers entering the school system from both in-and outside Oman.Believingly, those who are good at language are good at teaching.However, the result showed that those who are good at language are not necessarily good at teaching a language.
Thus, the ministry should rethink of the IELTS test as if the only assessment to assess the Omani graduate English teachers.The seven indicators of professional competencies namely Knowledge and understanding; Planning and lesson preparation; Classroom management; Teacher reflection and evaluation; CPD and professional characteristics and Competency in English are all taught and measured while Omani graduates are still students in the college through various pedagogical courses such as 'ELT Methods of teaching', 'ELT School Curriculum Analysis', 'Classroom Research and Teacher Development', 'practicum1', 'pracicum2', 'practicum3' and 'practicum4'.Thus, the reports of the students' performance in these indicators could function as graduate attributes measuring student teachers' capabilities of teaching instead of IELTS.

Conclusions
This study is set out to examine if there is a relationship between IELTS and Omani teachers' professional competencies.To this end, the study collected Omani graduate teachers' CGPA, IELTS and their professional competencies scores, which are assessed by supervisors from the Ministry of Education.The results showed a moderate but significant relationship between IELTS and CGPA but a weak relationship between IELTS and teachers' professional competencies.It is evidence to conclude that based on this research IELTS only is not a predictor of teaching proficiency despite its significance as an indicator of English teachers' teaching proficiency.Based on these results, the study puts forwards the following recommendations and limitations which might help in making the teacher assessment more reliable.
Those candidates who have lower than band 6.0, the minimum requirement for accepting English teachers in the ministry of education the requirement, their teaching proficiency is not known as they have not been employed yet.It is only those who passed the required score of IELTS, their CGPA and teaching proficiency is measured.Therefore, it is unreliable to conclude that IELTS can predict the academic performance and teaching performance unless those who got and did not get the minimum requirement are compared and measured.So, it is needed to let those students whose IELTS test less than band 6.0 teach as they have completed five-years of learning and practicing English language in a governmental College.Therefore, the ministry should reconsider its requirement in employing its students as English teachers in order to have highly qualified teachers who could evidently improve the academic performance of their students.In addition, the college should reevaluate the graduate teachers' assessment in the provided courses of their English program by the Department of English Language and Literature.This process will enable them to come up with graduate professional standards or graduate attributes that can serve as a gate-keeping function for employment instead of IELTS.
The key limitation of this study is the sample size.The sample of 77 teachers' teaching proficiency score may not yield reliable results.However, as indicated previously those are the numbers whose information is available by the educational supervision.Additionally, the teachers' proficiency scores may not fully indicate the teachers' competence in teaching; however the descriptive competence reports are confidential and therefore won't be allowed for publishing unless the teachers agreed.Furthermore, the individual modules for the IELTS score are available only with the candidates whose contacts are less accessible as they are employed all over the regions of Oman.Therefore, the individual modules and the academic capabilities of the Omani English teachers were not measured.This study is only small scale research which investigates the issue from a descriptive design only.Therefore, the future research should devote more attention to the graduate teachers' professional competencies' assessment in Rustaq College through investigating the courses' assessment and how effective they are in qualifying Omani English teachers.

Table 2 .
Correlations between IELTS and Competency in English (CE) How adequate is IELTS, required by the Ministry of Education as a reliable criterion to measure teacher candidates' capabilities in teaching, in predicting teacher candidates' teaching proficiency or competencies?The teacher candidates' competencies are labeled in Table3as TPC (Total Professional Competencies) to indicate the total of the seven indicators of professional competencies assessed by the supervisors from the Ministry of Education.