Evaluation of a Foreign Language Textbook Used in the Greater Boston Region: An Evaluation of a Japanese as a Foreign Language Textbook

Textbook materials continue to play an important role in foreign language teaching and learning classroom. Textbooks are significant because these materials are considered elements in the field of foreign language learning. Teachers are often engaged in the application of textbooks for common teaching goals. Although the significance of the application of textbooks is accepted, ideas on whether textbooks could assist or hinder the teaching–learning process seem polarizing. This study employed a Likert scale survey to collect data on the evaluation of Japanese for busy people I: Romanized version (3 Edition), a Japanese language textbook for new learners. The survey checklist yielded a highly unusual result, that is, all the participants advocated the use of the textbook. This study was conducted in the Greater Boston region in Massachusetts, MA, USA. Upon investigation, over 30 language learning centers, nearly 20 postsecondary institutions, and above 20 secondary schools were determined to be offering Japanese language courses. A total of 49 participants responded to the survey. None of the participants provided negative comments about the textbook. In addition, over 90% of the participants believed that the textbook materials and exercises were organized, contemporary, and applicable.


Introduction
Many people around the world currently speak over one language. English is among the well-known foreign languages that are extensively used in various fields, from commerce to the social sciences. The United States Census Bureau (2015) estimated the population of the city of Boston at 667,137. Between 1990 and 2010, the population of foreign immigrants significantly increased from 114,597 to 167,311. By 2010, immigrants represented over 25% ISSN 2327-5499 2017 of Boston's population, which was an increase from 20% in 1990. In the Greater Boston region, the population of immigrants totaled over eight million.

International Research in Education
The majority of the immigrants came from South America, Asia, and Africa. The city's census report indicates that the top ten countries of origin for migrants are as follows: (1) Dominican Republic, (2) China, (3) Haiti, (4) Vietnam, (5) El Salvador, (6) Columbia, (7) Cape Verde, (8) Jamaica, (9) Brazil, and (10) India. Many of these foreign-born immigrants were well-educated and the report indicated that approximately 30% of these people hold at least an undergraduate degree (Lima & Melnik, 2012). Spanish, French, Chinese, Portuguese and Vietnamese are the most popular foreign languages within the city. The evidence indicates that the residents of Boston are likely to understand over one language for daily communication. Accordingly, a high demand for learning foreign languages may exist within this city.
Boston attracts many foreign-born immigrants, scholars, and students because of this city's rich history. Scholars suggest that the excellent values of learning materials are crucial to assist learners to gain new knowledge. To assess the value of textbook materials, teachers, researchers, and educators must exercise judgment regarding the applicability and appropriateness of these materials with regard to their learners' demands. Therefore, textbook evaluation may determine the quality of textbook materials and assist learners to gain particular skills.
Extensive studies have focused on textbook material evaluation and the assessment of English language textbooks and their supplementary materials. However, no evaluation and assessment have been performed on low-demanding languages. Educators who desire to enhance the effectiveness of the teaching-learning process should conduct a detailed and systematic evaluation to satisfy the needs of students (Krashen, 1982;Richard & Rodgers, 2001;McGrath, 2002;Harwood, 2010;Tomlinson, 2011).

Objective of the Study
The objective of this study is to fill in the gap of low-demanding foreign language textbook materials that are currently being used in language learning centers in the Greater Boston region. Evaluation is a process of matching items to other items; teachers could use a checklist to match the available items to determine the correct materials for their learners. The result of this study enhances the evaluation of textbook materials for a particular foreign language and emphasizes the need to evaluate other learning materials, such as those related to vocational skills and home economics. For this study, the researcher evaluated a Japanese language textbook titled Japanese for busy people I: Romanized version, which was designed for English language users.

The Significant Position of Textbooks in Foreign Language Learning Programs
Teachers commonly use printed textbooks in their classrooms. Without a textbook, teachers and students may be unable to focus on acquiring knowledge and achieving targets for certain International Research in Education ISSN 2327-5499 2017 ages or learning levels. Textbooks effectively guide teachers and students toward the appropriate process of gaining knowledge. In particular, students can use textbook materials as guidelines to learn new knowledge. A textbook typically includes a table of contents, chapter headings, illustrations, statistics, indexes, references, and a glossary of terms. For this study, a low-demanding textbook is considered for use as reference with appropriate materials in an English-speaking environment.

Debate about the Position of Textbook
Textbook materials continue to play an important role in foreign language teaching and learning classroom. Textbooks are significant because these materials are considered elements in the field of foreign language learning (Hutchinson & Torres, 1994). Teachers are often engaged in the application of textbooks for common teaching goals (McDonough & Shaw, 2003). Although the significance of the application of textbooks is accepted, ideas on whether textbooks could assist or hinder the teaching-learning process seem polarizing. Richards (1993) believed that textbooks are resources instead of course materials. McGrath (2006) mentioned that textbooks are beneficial as bases of learning, which could be divided into guidance, support, resource, and constraint. The relationship between teachers and textbooks is based on the effort toward the same goal of addressing the needs of students. Yule (2010) advocated that providing ideas to instructors and students, as well as facilitating their understanding of the subjects in the course, enables textbooks to serve as a map through the directions of cultural knowledge and function as a structured module of linguistics,.

Disadvantage of Using Textbooks
The applications of textbooks are extensively discussed; however, a few researchers have suggested that textbooks could be detrimental (Allwright, 1981;Ur, 1996;Tomlinson, 2011). Allwright (1981) argued that textbooks may negatively influence the learners' engagement in the language acquisition process. Allwright further indicated that textbooks were inadequate to address the comprehensive situation of the language acquisition process. Ur (1993) argued that the organization of textbooks could inhibit creativity and imagination during the teaching-learning process. The notion is that if the text fails to draw the attention of learners, then this situation may result in failure of engagement. Tomlinson (2003) claimed that only a few textbook authors incorporated applied language acquisition principles into their materials.

Viewpoint of Using Textbook
In view of the studies presented in the preceding section, the use of textbooks in classrooms remains essential despite the belief of a few researchers that these materials are detrimental to learning. In addition, the organization and structure of textbooks should be appropriately developed by experienced authors. Accordingly, learners are likely to benefit from using high-quality textbooks. However, teachers should conduct an evaluation to ascertain the suitability of textbooks for their particular classroom. Factors that may influence classroom ISSN 2327-5499 2017 practices include geographical location, medium of instruction, duration of study, background of students, professional experience, learning objectives, age, and social level.

Textbook Materials Selection
McDonough and Shaw (2003) explained that no textbook is perfect for all classrooms. Therefore, before and after the teaching-learning process, teachers should study the materials, provide their comments, and send feedback to publishers about the textbook because authors and publishers may possibly commit errors, as well as neglect critical information and enhancement of ideas. Ur (1996) discussed the evaluation of elements, such as program direction and curriculum on specific ideas. Textbook selection checklists should be applied as instruments to assist school administrators in their decision-making and analysis of which textbooks should be used. The literature suggests that evaluating textbooks is significant to the selection process. Incorrect selection would be detrimental to the development of the language skills and the overall learning development of students (Mukundan, Nimehchisalem, & Hajimohammadi, 2011).

The Differences between Analysis and Evaluation
To evaluate textbooks, teachers should be able to differentiate between analysis and evaluation. Tomlinson (2003) explained that evaluation refers to the calculation of the value of textbooks. Teachers should record their views regarding the learning outcomes of textbooks. The evaluation process pertains to the involvement of students who plan to use textbooks as their primary learning tool, whereas analysis refers to the objective analysis of textbooks in the first stage of evaluation (which is de-contextualized). Evaluation may consider various aspects, such as achievements, materials included, and how the textbooks aim to provide knowledge to students. Thus, evaluation is completed with target students in a learning environment and by assessing the textbook in terms of the needs of teachers and students. However, analysis lacks such elements. Sheldon (1988) explained that the evaluation of foreign language learning textbooks is significantly discussed in the field of education. However, the number of well-established studies is limited. Various researchers believed that evaluation should employ a leveled approach. Such approach has two levels: (1) the first level outlines the impressionistic evaluation and (2) the second level outlines the in-depth evaluation (Cunningsworth, 1995;Ur, 1993;McGrath, 2002).

Textbook Evaluation Process
For the leveled approach to evaluation, teachers using the impressionistic method should study the literal method by carefully reading the entire textbook to identify and understand the advantages and disadvantages of this material (Cunningsworth, 1995). Such understanding offers an overview of the design and organization of the material, including its structure and innovations. After the first level (impressionistic) evaluation, weak materials should be excluded (McGrath, 2002). Thereafter, an in-depth evaluation will be conducted for the particular sections of every material, such as the sections explaining how activities can address the needs of learners. Successful in-depth evaluation should identify one or two units ISSN 2327-5499 2017 and should consider the balance of knowledge, skills, and activities (Cunningsworth, 1995;McDonough & Shaw, 2002).

International Research in Education
The leveled approach has also been applied and developed by many researchers. McGrath (2002) developed the first glance and arm-chair evaluations, McDonough and Shaw (2002) formulated the internal and external evaluations, and Ur (1996) developed the general and specific evaluations. These evaluation tools express the importance of combining the impressionistic and in-depth evaluations for student-oriented materials.
Apart from the leveled approach, McGrath (2002) expressed his idea about stage evaluation. The three stages by which a material could be evaluated are the pre-use, whilst-in-use, and after-use stages. Pre-use evaluation is the first stage in which the textbook may generate the interest of potential textbook users (Cunningsworth, 1995). Whilst-in-use evaluation may involve studying the material while using it or by observing how the textbook is used in teaching (Mukundan, Nimehchisalem, & Hajimohammadi, 2011). Finally, post-use evaluation assists in analyzing the comprehensive short-and long-term goals for the sustainable use of textbooks (McGrath, 2002).
Note that apart from the manner by which people study textbooks, the selection and evaluation processes are completely subjective (McGrath, 2002). Nevertheless, the evaluation process should be conducted. Without deeply understanding the organization of the material, teachers will find it difficult to understand if the textbooks are able to meet the needs of learners (McDonough & Shaw, 2002;Mukundan, Nimehchisalem, & Hajimohammadi, 2011).

Checklists
Harmer (1991) was convinced that using checklists could be beneficial in understanding whether a textbook is able to address the needs of learners. McGrath (2003) explained that a checklist is a good tool to generate different opinions about a textbook. One of the most significant advantages of using a checklist is that it represents a systematic direction to confirm related elements that are considered for evaluation (McGrath, 2002). Checklists and evaluation elements could be customized based on needs to offer sufficient flexibility during the evaluation stage (Mukundan, Nimehchisalem, & Hajimohammadi, 2011). McDonough and Shaw (2003) explained that no standard checklist exists that is suitable for all learning environments. Accordingly, teachers should develop their own checklist based on their own classrooms and the needs of their students. After an impressionistic evaluation, an in-depth evaluation could be conducted to ensure that essential elements are included in the textbooks. McDonough and Shaw (2003) used the development of the leveled approach as basis to suggest an external and internal checklist for a brief overview of the materials from such elements as the cover, introduction, and table of contents; thereafter, a detailed and profound internal assessment is made. Moreover, all checklists should be systematic, organized, and coherent for teachers to evaluate the textbooks. Finally, the previous studies on checklist application were the key supporting evidence for teachers to conduct evaluation before their final decision. Without evaluation, teachers are hardly capable to determine whether textbooks are suitable for their own classroom. Therefore, textbook evaluation is an essential ISSN 2327-5499 2017 step for teachers to select appropriate textbooks for their learners based on their needs.

Methodology
This study employed a Likert scale survey to collect data on the evaluation of Japanese for busy people I: Romanized version (3 rd Edition), a Japanese language textbook for new learners. This textbook was designed for English language speakers and translated into Roman characters with English explanations. To answer the research question regarding the usefulness of foreign language textbooks, 10 questions based on a five-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree) were used. The Likert scale questions are listed in the Findings section. The checklist was adapted from the model of Candlin and Breen (1979). The model for the current study involved two stages: the first stage focuses on usefulness, whereas the second stage involves the selection and application for teaching and learning. To establish the appropriate checklist with questions for a low-demanding foreign language textbooks, the current researcher applied the checklist suggested by Candlin and Breen (1979) but with minor changes.

Site and Participants
This study was conducted in the Greater Boston region in Massachusetts, MA, USA. Upon investigation, over 30 language learning centers, nearly 20 postsecondary institutions, and above 20 secondary schools were determined to be offering Japanese language courses. This researcher sent invitation e-mails to potential institutions and teachers. The message included the title of the investigated textbook along with an image of its cover page, the objective of this study, the study research question, and the Likert scale survey with questions (e.g., on checklists). To be eligible for the study, the institution must use the textbook titled Japanese for busy people I: Romanized version (3 rd Edition). All of the participants agreed to participate in this study. A total of 70% (i.e., 49) of the recipients responded to the e-mail invitation with valid survey data. The rest of the e-mail recipients either claimed that they stopped using Japanese for busy people I: Romanized version (3 rd Edition) or they did not respond.

Steps for Data Collection
First, the researcher sent e-mail invitations to each potential institution. An institution using the selected textbook is eligible to participate in this research. Second, the institutions responded to the invitation whether they wanted to participate. Third, if the institution desired to participate, the school administrators may endorse the survey checklist to their Japanese language teachers. If the institution has over one teacher, they may furnish copies of the survey based on the number of teachers. Fourth, after the Japanese language teachers complete the survey, the institution administrator would send the completed survey via e-mail. Fifth, the researcher analyzes the results.

Findings
The Likert scale question or checklist regarding the textbook for this study was designed to discover the teachers' perspectives on the effectiveness of this material to address the needs International Research in Education ISSN 2327-5499 2017 and goals of their students. The responses provided to the participants were rated from 1 (Strongly Disagree), 2 (Disagree), 3 (Neutral), 4 (Agree), 5 (Strongly Agree).
The results of the survey indicated that a majority of Japanese language teachers believed that Japanese for busy people I: Romanized version (3 rd Edition) was an appropriate teaching and learning material for new Japanese language learners in the Greater Boston region. For checklist questions 1, 2, 3, 5, 8, and 10, the participants rated the items as "totally agree." Checklist questions 4, 6, 7, and 9 had neutral ratings. However, the neutral ratings of these items were only from 8% to 18%. Below 20% of the participants believed these items were somehow undecided. Nevertheless, this textbook was deemed still applicable to contemporary society. The findings of the checklist questions were summarized in Table 1.

Discussion
The Japanese language teachers responded to the survey checklist question by firmly indicating that the current textbook material provides contemporary and applicable structures and exercises for the language learners. Over three different versions of Japanese language textbook materials are available in the market. However, most of the institutions in the Greater Boston region continued to use the studied textbook as their primary teaching and ISSN 2327-5499 2017 learning tool. Although this textbook has been published nearly one decade ago, the vocabulary lists, exercises, and reading materials are still applicable for learners to acquire language skills.

International Research in Education
The survey checklist yielded a highly unusual result, that is, all the participants advocated the use of the textbook. None of the participants also provided negative comments about the textbook. In addition, over 90% of the participants believed that the textbook materials and exercises were organized, contemporary, and applicable. The positive results indicated that this textbook could address the needs of their students, as well as the outcome expectation. Between 4 and 9 participants gave neutral ratings for checklist questions 4, 6, 7, and 9. In particular, 9 participants believed that pronunciation training was inadequate for their learners. Japanese language teachers may encounter difficulties in training language learners from various language backgrounds and accents because Boston is a multi-cultural city where nearly 30% of the residents speak a foreign language. Accordingly, Japanese has a unique pronunciation system; hence, teachers may need to employ special vocal training for students with European language background. However, when teachers need to train learners from international countries, no universal vocal training may address the needs of all learners. For checklist question 9, which is about the four skills training, language teachers may have similar concerns to those they expressed regarding checklist question 4. Therefore, the high scores and result of the survey involving in-service Japanese language teachers indicate that the textbook is considerably appropriate for language learners in the Greater Boston region.

Limitation of the Study
Two limitations were determined in this textbook evaluation study. First, this study was limited to collecting data from the Greater Boston region. Boston is a multi-cultural city where low-demanding foreign languages may remain popular for certain groups of learners. However, this study may be unable to reflect the actual situation in rural and remote regions, such as Alaska and Montana.
Second, the researcher could have added an interview section after the completion of the checklist survey. Interviews may further obtain in-depth comments and feedback on the textbook because the survey was unable to determine the understanding and perspectives of the users.

Future Research Direction
The current study investigated the textbook evaluation of a Japanese language textbook in the Greater Boston region. Currently, most studies on textbook evaluation are concerned with the usage of English language textbook. The current evaluation report was one of a few studies concerned with low-demanding language textbook. Accordingly, this study provides three recommendations as follows.
First, the current checklist was adopted from the model by Candlin and Breen (1979) but with minor modifications. This checklist is applicable for evaluating Japanese language textbooks and those for vocational studies, additional foreign language teaching and learning, adult basic skills, and culinary arts. Moreover, the current checklist questions involve linguistic and verbal elements; hence, evaluators may slightly change the elements into factors that are appropriate in their field.
Second, institution leaders, teachers, and evaluators may apply this methodology to their current textbook and supplementary materials. The result indicates that most of the participants lacked experience in evaluating teaching materials. Evaluation enabled teachers to determine appropriate materials for their learners to achieve their expected outcomes.
Finally, the evaluation indicated that teachers should not rely only on published materials because learners may have different learning objectives and targets. For example, the current textbook may not satisfy the expectations of learners with engineering background. Note that textbooks and supplementary materials constitute only a few of the primary modes of learning. Activities, tailor-made handouts, and practical exercises should also be employed to meet the goals of learners.