Retrotext-E 1 . 0 : The Beginnings of Computer-based ELT Textbook Evaluation

Background There are two main reasons why textbooks are evaluated; the first which is associated to the task of textbook selection (predictive evaluation) and the second to the need to evaluate it while in use (retrospective evaluation) so that teachers can implement adaptation procedures later. Predictive evaluation is almost nonexistent in most developing countries as textbooks are usually provided free or through a textbook loan scheme. There is however hope for getting teachers involved in while-use/post use or retrospective evaluation of textbooks as they not only help teachers with adaptation but help teachers in their overall professional development. Since textbooks are an essential part of life any form of out-of-


Background
There are two main reasons why textbooks are evaluated; the first which is associated to the task of textbook selection (predictive evaluation) and the second to the need to evaluate it while in use (retrospective evaluation) so that teachers can implement adaptation procedures later.Predictive evaluation is almost nonexistent in most developing countries as textbooks are usually provided free or through a textbook loan scheme.There is however hope for getting teachers involved in while-use/post use or retrospective evaluation of textbooks as they not only help teachers with adaptation but help teachers in their overall professional development.Since textbooks are an essential part of life any form of out-of-The two main ways English Language Teaching (ELT) textbooks are evaluated are through impression, which involves teacher intuitions and by way of use of an instrument, usually a checklist.While the former has been known to be effective especially if done by experienced teachers, the latter is more common because in some teaching situations evaluations are carried out by several teachers who have doi:10.7575/aiac.alls.v.1n.2p.270 to use the same book.In such situations evaluations are best done using a common instrument.
Teachers can easily find checklists for evaluation of ELT textbooks.There are numerous checklists available all of which have been developed by individuals or institutions taking into consideration their own special needs.While there are scores of instruments available, the literature regarding reliability and validity of these instruments is lacking.

The need to redefine textbook evaluation in ELT
There is a need to redefine textbook evaluation in ELT.The development of textbook evaluation instruments from the perspective of predictive evaluation is still relevant but what teachers need to do is to look at evaluation from a wider perspective a perspective which encompasses diagnostic, formative and developmental aspects of textbook use.It is when evaluation is viewed from this perspective that while-use and post-use evaluation of textbooks become more important than evaluation for selection.
If retrospective evaluation of textbooks is to be emphasized then there would be a need to evaluate current tools available which have been predominantly developed for use for selection purposes.The checklists which have dominated textbook evaluation for decades (Mukundan and Ahour, 2010) have been found to be sometimes teacher-unfriendly.Some of the evaluation criteria are also found to be irrelevant.Many of the items which require evaluators to judge on vocabulary loading and distribution, for instance, defy logic as judging cannot be humanly possible under time constraints (Mukundan,2007).

The concept in the invention of software for textbook evaluation
Conceptually the software for textbook evaluation (RETROTEXT-E 1.0) is based on the Composite Framework for ESL Textbook Evaluation (Mukundan, 2004(Mukundan, , 2006)).This framework promotes triangulation of instruments so that evaluation is not overly dependent of data from only one source, which has traditionally been from the checklist.In this new framework, the checklist is supported by two other instruments, the computerized evaluator of vocabulary loading and distribution patterns The functions within RETROTEXT-E 1.0 The software was designed to be as user-friendly as possible.The main functions of this programme are: In Part 1, the textbook evaluation programme assists evaluators by providing key data on the following: 1. Is the loading of vocabulary in the book and across chapters pedagogically sound?
2. Do words used in the textbook reflect those in wordlists like the GSL?
3. Are repetition and recycling of vocabulary emphasized?
Unlike the mono-instrument instrument, the checklist, Part 1 of the software can empirically determine how effective vocabulary loading is as the programme has the ability to compute total number of words across the book and in individual chapters.The full list of words is also provided.In addition to this analysis of words in the book to determine suitability of use from the context of frequency can be carried out.The software does comparison of the words in the textbook with that of words in the GSL.This to some extent will determine if textbook writers have used the most frequently used words in the English language as a starting point, particularly at the lower levels.Another function of Part 1 is to help evaluators determine if words introduced are recycled within chapters and across the book.This is important as many experts have said that words introduced to learners must be repeated at least seven times at intervals so that they can be easily remembered (Thornbury, 2002).The screen shot of the analysis of Part 1 will be as in the figure below: Part 2 Part 2 of the software assists evaluators by way of a checklist to determine the efficiency of the book while in use.The checklist feature was retained as the developer felt that inventions need not replace old but essentially economical aspects of past inventions.The checklist can help teacher evaluators provide a teaching, after a unit or any other time interval.Data from Evaluations at intervals can be saved for study later.

The advantages RETROTEXT-E 1.0 have over other types of textbook evaluation
The main difference between RETROTEXT-E 1.0 and the competition is that when compared to intuitive methods it is empirical and when compared to traditional methods which use instruments like the checklist it is superior in terms of reliability (Mukundan, 2004).Also, traditional checklists are developed for predictive evaluation (mainly for selection purposes).RETROTEXT-E 1.0 while it can still perform as a framework for predictive evaluation is more suited for retrospective evaluation.The other advantages of this software are: i) The software provides record-keeping facilities which are very useful if retrospective evaluation of textbooks is going to be an important feature in the life of teachers.Record keeping is user-friendly and the fact that this is computerized minimizes the threat of loss of notes when it is done using conventional pen and paper sources.
ii) Evaluation can be daily, weekly, monthly, quarterly, yearly the teacher is in-charge of his or her own evaluation.Data are recorded with dates and when printed out this will be useful for monitoring evaluations from end-to-end.
iii) Evaluations can be done individually or in teams.Reports by a team of evaluators can be printed and used as documents for analysis at panel meetings.

iv)
Teacher professional development is enhanced because the evaluations done can be a source for discussion at end of year teacher workshops where teachers teaching the same level and using the same book can deliberate on suggestions on adaptation made by different teachers and come up with activities that can be replacements for those deemed unfit for use.

Conclusion
This innovation is probably not the answer to all the problems that confront teachers with regards textbook evaluation.However, because it does not altogether have a summative stance in evaluation and it supports retrospective evaluation of textbooks, it can become a useful tool for the teacher for not only evaluating textbooks but researching them.
begins it would be necessary to load text files (txt) of the digitized pages of the textbook into the software so that Part 1 of the evaluation can be successfully carried out.How the different parts of RETROTEXT-E 1.0 workPart 1