Examining the Effectiveness of the Accelerated Reader Program in College Students Enrolled in a Teacher Education Program

The purpose of this study was to explore the perceptions and opinions of college pre-service teachers who experienced the Accelerated Reader program in their K-12 setting. Students were surveyed to elicit responses about the benefits and downfalls of the program. The authors found that, in general, the college students disliked having used AR


Introduction
Motivation to read is a key factor that contributes to a child's success in school and ultimately in life. Learning to read includes many non-ability factors such as self-concept, support and International Research in Education ISSN 2327-5499 2013 motivation. As assistant professors in Teacher Education of a mid-sized university in Indiana, U.S., many of our first discussions with college students are about how they learned to read. In hearing their stories, sometimes humorous and sometimes daunting, the authors decided to delve deeper into one program that many students write and talk about: Accelerated Reader (AR). As former educators in K-12 public education, the authors understand the importance of motivation, especially motivation to read. Unless students are motivated to read, they will not reach their full literacy potential (Gambrell, 2011). In class discussion, the authors became aware that many pre-service teachers' motivation to read was influenced by classroom practices they experienced, namely those activities associated with AR. The purpose of this study was to explore the perceptions and opinions of college pre-service teachers who experienced the AR program during their K-12 education. This study was not funded and resulted in no financial gain.

Literature Review
Engaged readers, as defined by Guthrie & Anderson, are readers who are intrinsically motivated, who read regularly and enthusiastically for a variety of purposes (as cited in Applegate & Applegate, 2010). "Highly motivated students who see reading as a desirable activity will initiate and sustain their engagement in reading and thus become better readers" (Gambrell, 2011, p.177).
According to Wang & Guthrie (2004), children who are intrinsically motivated find reading enjoyable or interesting, and children who are extrinsically motivated read for a designated outcome like getting a reward or getting good grades. "Research suggests that being intrinsically motivated to read is advantageous, as intrinsic reading motivation is associated with higher levels of reading attainment" (Medford & McGeown 2012, p. 786). These authors further assert that the relationship between intrinsic and extrinsic is complex and acknowledge that children are motivated to read in numerous ways. Park (2011) states that extrinsic reading motivation may not be detrimental if there are high levels of intrinsic motivation already established. Efforts to enhance and increase a child's reading ability cannot ignore the role of motivation, and teaching practices should be implemented to support and increase a child's motivation to read. As reading skills are vital to a child's academic success, it is important to engage children in appropriate activities and experiences in the classroom that will get them excited and interested in a variety of reading materials.

The AR Program
AR is a reading program in which students work independently but are closely supervised by the classroom teacher/aide or parent volunteer. Students choose an appropriate book to read from leveled texts, read the book, and then complete a computer-generated quiz to check their comprehension. Quizzes range from 5 to 20 questions and consist only of multiple-choice questions. Classroom teachers use the quiz scores to monitor reading progress, give a grade, adjust reading levels, and identify students who may require other interventions (USDOE, 2008). Reading levels and quizzes for the AR program range from grades one through twelve International Research in Education ISSN 2327-5499 2013 (Scott, 1999). Each book on the AR list has a designated reading level and a suggested number of points to be awarded to students who read the book and pass the quiz. Students are encouraged to earn points that can later be redeemed for prizes. Some schools and teachers use competition between classes and within the classroom as a motivator for students to read and gain points. It should be noted that AR's reach extends far beyond the U.S., according to its website. Although more than half of the 75,000 schools using AR are located with the U.S., the program is in use worldwide, including in some 2000 schools in the U.K. and additional schools in Australia, New Zealand, and parts of Asia.
First introduced in 1988, the AR program has proved to be controversial, with some researchers finding the program raised overall student comprehension, vocabulary, and engagement in reading; other studies have found little or no value in the use of the program. A product of Renaissance Learning, AR has been in use for over 25 years and claims to be a powerful tool for monitoring and managing independent reading practice. According to the AR website, the benefits of using the program are the following: "Monitor students' progress toward College and Career Readiness expectations…, personalize and guide independent reading practice and develop lifelong readers and learners" (http://www.renlearn.com/ar/). The website goes on to suggest that with AR, teachers can Motivate students for learning. Self-selected reading at students' independent reading levels results in success, which spurs enthusiasm, higher attendance, fewer discipline problems, and better attitudes. Students will be motivated to read constantly.
Receive reliable, objective information. The data cover reading comprehension of fiction and nonfiction reading, vocabulary development, literacy skills development, and more.
Prepare students for college and career. Students engage in substantial amounts of reading practice and receive immediate feedback. As a result, scores improve on state tests and more students master standards-research proves it.
Keep all students challenged. Students are guided to books where the level of difficulty is neither too hard nor too easy-the level at which optimal learning takes place.
None of the research we reviewed was neutral on the subject of using AR. Instead, researchers seemed to focus either or positive outcomes or on negative ones. For this reason, the authors have presented previous findings in positive and negative categories.

Positive Findings about the AR Program
Some studies confirm the effectiveness of the AR program. Johnson and Howard (2003) found AR to be effective in increasing the reading scores of 755 urban, inner-city students in grades 3, 4 and 5. As measured by the Gates-MacGinitie Reading Test, "high" participants in AR gained 2.24 years of reading level; "average" participants gained 1.52 years, and students who were labeled "low" participants gained only .75 years over the one-year study. Johnson ISSN 2327-5499 2013 concluded that the more frequently students participated in AR, the greater their growth in reading comprehension and vocabulary. The unanswered question is how much students might have grown in their reading attainment without the AR program, since there was no control group.

International Research in Education
A UK study (Topping & Fischer, 2003) examined AR use among 7-14 year olds, the majority of whom were economically disadvantaged, in 13 different schools throughout the UK -a total of 704 students. These researchers reported reading achievement increases that they termed "abnormally high and statistically significant" (p. 267) when the STAR Reading evaluation was administered. As with the Johnson (2003) study above, student progress was tied to degree of participation in the AR program. Again, the authors wonder how much students might have gained in reading attainment without AR, especially since there was no control group in this study, either. Guastello (2002) conducted a five-year longitudinal study of the impact of using AR with students in grades one through eight at an urban school in Brooklyn, NY, where many students spoke English as a second language. She found that with the use of AR, which she referred to as an "an energizer" (p. 54), students were motivated to read, and parents reported that the children were more engaged in reading. Student attitudes toward reading also improved. Guastello reported increased test scores in reading and writing during the study, and library circulation increased. Again, there was no control group, so other factors may have influenced the increases she reports. Briggs and Clark (1997) completed a report for the Texas Center for Education Research with the express goal of examining effectiveness of various reading programs among lower elementary-grade students. Acknowledging that AR is a supplement to classroom reading instruction, these researchers classified it as an effective program, citing three studies -one of which came from the Institute for Academic Excellence, a subsidiary of the company that also owns AR. None of the three studies reported on the effectiveness of AR with early elementary school readers. Participants in one study (n=39) cited were sixth graders, who did show statistically significant improvement in reading scores (Vollands, Topping, and Evans, 1999). Participants in the second study (Peak & DeWalt, 1994) were ninth graders (n=50), half of whom had participated in AR, starting in third grade; the other half had no AR participation. The researchers reported that the AR students checked out more books from the library and had higher reading scores than non-AR students. The final study by Paul, VanderZee, Rue, & Swanson (1996) analyzed AR and non-AR student median scores on the Texas Assessment of Academic Skills (TAAS) in 2511 Texas schools. They reported higher mean scores for AR students on the TAAS for all tested grade levels -except for 10 th graders. None of the evidence in this report is compelling support for the use of AR in the lower elementary grades, but it does suggest efficacy for some students. Paul, VanderZee, Rue, and Swanson (1996) collected data from 6149 schools, also in Texas, separating them into two groups: AR users and non-AR users. Student achievement was compared, based on scores from the TAAS. These researchers reported the following findings: ISSN 2327-5499 2013 • Schools using AR reported statistically significant higher test scores on virtually every section of the TAAS when compared to socio-economically similar non-AR schools.

International Research in Education
• Schools using AR reported statistically higher attendance rates than non-AR schools.
These findings show strong support for the use of AR as a tool for improving test scores and some support for AR as a motivator. It should be noted, however, that Paul owns the AR software.
McKnight (1992) examined changes in fifth grade students' (n=17) attitudes toward reading and their participation in this activity. Students were introduced to the AR program by the researcher and were encouraged and supported in their use of the program for leisure reading.
McKnight reported modest gains in positive student attitudes and participation once the AR program was implemented. It should be noted that this researcher supplemented the AR program with a number of planned support activities in the school library.
Altogether, a number of studies have reported student gains from participating in the AR program. Even though the gains have varied somewhat, there is evidence that AR can and does serve as a tool in the quest for literacy among young students.

Negative Findings of the AR Program
The What Works Clearinghouse (WWC) of the U.S. Department of Education (2007) examined 100 studies of the effectiveness of AR, but found that only two met its research standards for randomized studies. As a result, WWC reported no measurable effects of AR on reading fluency, mixed results on comprehension, and possible positive effects on general reading achievement for grades K-3.
A similar report from WWC in 2010 reviewed 318 studies of AR effectiveness with older students (grades 4-8) and determined that, again, only two studies met WWC research standards. From these studies, WWC found no significant support for AR in improving reading fluency or comprehension in grades 4-8. However, the WWC proposed its own standards for determining the acceptability of published, peer-reviewed research and failed to list or describe those standards. Krashen (2005) reviewed four studies of the effectiveness of AR used with low reading students in grades 4-6 (ages 9-11). In the studies sited, the AR intervention was used for as little as 3 months or as long as 4-5 years. Krashen concluded that there was no evidence to support the use of AR to improve general reading attainment or interest in reading. On the contrary, he argued that the use of points and prizes amounts to bribery and teaches children the lesson that reading is such an unpleasant task that one should do it only for extrinsic rewards. Pavonetti, Brimmer, Cipielewski (2002-2003 investigated the claim that AR produces life-long readers. They administered the Title Recognition Test to 1534 middle students from three school districts. Of the participants, 834 had used the AR program in elementary school; 700 had not. The researchers found no significant difference in the reading activity of students who did and did not use AR in elementary school. In fact, in two of the districts, ISSN 2327-5499 2013 students who had not participated in AR were reading more than their AR peers. The researchers also found that readers who were reading below grade level --those who might well benefit from more reading practice --tended to have the lowest levels of participation in the program. They suggest: "the more motivated and/or verbally intelligent students may have self-selected themselves into the High Usage Group and, therefore, would have increased their reading comprehension regardless of their participation in the AR program" (p 94).

International Research in Education
In her Master's thesis, Toro (2001) examined the reading scores of second grade students. One class (n=20) participated in the AR program at a private Christian school; the other class (n=16) in the same school did not participate in the AR program. Instead, this second group was asked to read independently for 30 minutes each night. At the end of the six-week experiment, the researcher found no significant difference in the reading comprehension of the two groups. Because the study involved so few subjects, its generalizability is questionable, and most of us recognize that reading is a skill that improves with practice, with or without a program like AR. Melton et al. (2004) studied fifth-grade students (age 10 to 11) in two districts in Mississippi to investigate possible differences in reading growth between AR and non-AR participants. Of the students, 322 had participated in AR; 270 had not. The researchers reported that students in the upper and middle quartiles who did not participate in AR earned higher adjusted mean scores for reading achievement than did students who did participate in AR. There was no significant difference in adjusted mean scores for those in the lowest quartile. Lamme (2003) pointed out in her study of an AR program in an elementary school that not all children's books are in the AR catalog and that very few schools can afford to purchase all the books that AR can offer. She also says that just because a child can read at a certain level does not mean that all the books at that level are appropriate. There are times when children can read beyond their independent reading level if the content is interesting to them. She found children in this study who were so focused on finding a book at their level that they could read and earn points on that they further states, Reading is not a performance activity. Good readers spend time carefully selecting books and care deeply about what they are reading; they don't just grab the next book on the shelf. Reading in school should not be for practice; it should be for pleasure and knowledge (41).
From the studies reviewed above, it would seem there are no clear answers about AR's effectiveness in advancing reading achievement or increasing student motivation to read.

Method
A survey was used to capture the opinions and perceptions of college pre-service teachers about their beliefs concerning the AR program. The survey was administered to college students enrolled in their first literacy methods course and majoring in elementary education. Seventy-four (N=74) student responses were collected. Participants were overwhelmingly ISSN 2327-5499 2013 white and female and were in their sophomore or junior year of college. They were informed of the research and were told their participation was voluntary. The survey was given at the end of a class period at the beginning of the semester so that students would have to draw upon their own opinions and not that of a textbook or the teacher. It should be noted that, presumably, all participants were successful readers because they were attending college. The questions on the survey were designed based on information gleaned from prior class discussions about AR with students in other literacy methods classes and on the stated purposes of the program. Our literacy colleagues at the university reviewed and endorsed the survey questions. The numerical results of the survey are shown in Appendix I.

International Research in Education
In addition to collecting quantitative data, the researchers sought comments from the study participants. The researchers hoped that an analysis of the data could bring new knowledge on the advantages and the disadvantages of the AR program.

Findings
In this study, forty-two of the seventy-four students (57%) agreed that they were avid readers. Whether or not this finding is related to their AR use could not be determined by analyzing the responses to other items in the survey, but it may be that their school success (as evidenced by college admission) separates them from less successful users of the program. Additionally, the survey results showed that 49% of students agreed that AR helped their vocabulary growth as compared to 20% who disagreed. At least in their minds, AR contributed to the vocabulary development of nearly half the participants, a significant portion of the group. Forty percent of students agreed that AR helped them establish goals for reading compared to 32 % who disagreed. These results are too close to provide any reliable conclusions and add to the controversy surrounding AR's use. Forty-two percent enjoyed reading the books in the AR program opposed to 26% who did not, indicating that while about one-quarter of students did not enjoy the books, a much larger percentage did.
However, students reported negative as well as positive responses to the AR program. Forty-one percent of students disagreed with the statement that they were successful readers because of AR, and 62% disagreed or strongly disagreed with the statement that because of AR they had a love for reading. What is not clear from the findings is what factors participants might credit with their reading success and affinity for reading. In this study as in others, a significant percentage (30%) admitted that they sometimes cheated on the AR quiz to earn points. Forty-five percent of students disagreed with the statement that AR challenged them to read more difficult books, and 38% disagreed that AR allowed them to be metacognitive when reading. Most interesting in this study was the fact that only 12% would use AR in their classroom, while 49 % disagreed or strongly disagreed and 38% were neutral on this question. Given that nearly half the participants credited the AR program with helping them develop vocabulary and a sizeable portion indicated that they enjoyed reading the AR books, these responses are surprising.
Forty-three percent of students disagreed with the statement that they enjoyed the AR program, compared to 39% who agreed they did. Forty-two percent agreed that they could pass the quiz without reading all the books while 39 % disagreed. Forty-two percent felt that ISSN 2327-5499 2013 the competition of the program challenged them to read more books while 36 % disagreed. In each of these cases, the numbers indicate a significant split in the way students recollect their AR experiences. Also significant is the percentage of surveyed students who reported their neutrality on the significant issues the survey hoped to uncover. Perhaps more telling than the examination of percentage rankings were the student comments on the survey.

International Research in Education
Because such a large portion of the responses fell into the "neutral" category (12-45%), we would suggest further research to capture the implications of these noncommittal responses. They may indicate that AR had little if any effect on the motivation of these participants. It is also possible that the "neutral responses" represent the inability of the participants to analyze the effects of having used AR early in their academic careers. Being unable to categorize these responses as either positive or negative could certainly skew the interpretation of survey results in either direction.
In analyzing and coding the student comments, the authors wanted to report accurately the range of comments, so the comment data were first coded by relevance to the study Comments deemed irrelevant by the researchers were omitted from further consideration. Irrelevant comments included those about the course in which students were currently enrolled and about other reading programs without reference to AR. These comments constituted only 4% of all comments. Of the remaining 96%, approximately, 65% of student comments were negative, compared to 35% positive comments. The negative comments far outweighed the positive ones. Examples of both positive and negative responses are included below. There were only two mixed responses, both of which have been included later in this paper.

Positive Student Comments
• "AR motivated me to read books out of my comfort zone that challenged me. I could have kept reading easy books, but AR made me try books that were more difficult, and I learned many new words." • "As someone who was a strong reader, I enjoyed AR." • "The [school] library had a great variety of AR books. I feel the AR program worked for me because it forced me to find books I enjoyed. I was, and still am, grade motivated. If I did not do well on the quizzes, it was my own fault." • "My school did a very good job of making it [AR] competitive and rewarded people at every point level so that reading became fun instead of just a chore." These comments suggest that students either were motivated to read before they began the AR program or that AR helped motivate them to read. While 28.4% of survey students agreed or strongly agreed with the statement "I enjoyed AR," 46.3% disagreed or strongly disagreed with that statement. Even more extreme were the responses to the statement, "I love reading because of AR." Of those surveyed, 62.2% either disagreed or strongly disagreed; 31.1% were neutral; 6.8% agreed or strongly agreed. The portion of positive comments (approximately 35%) is consistent with the survey responses. ISSN 2327-5499 2013

Negative Student Comments
• "At first AR made everyone want to read as many easy books as possible to get points.
Once we got older, we had to pick from a list to read, and I hated AR then because I was not interested in the books on the list." • "I did not enjoy AR because I felt as though I was being forced to read. I hated reading when I was younger." • "[AR] discouraged me from reading and put too much pressure on me." • "I hated reading because of AR." • "[AR] defeats its own purpose. It did push me to read the Harry Potter series, but it took me so long to read them, and I would miss questions on the test, so I never reached my point goal." • "AR was a hindrance. It gave me knowledge that reading a more difficult book was a waste of time because I may not pass the test. If I don't pass the test, I wasted all that time I spent reading. So I stuck with the very easy books I could pass." • "It [AR] can cause a lot of students -especially struggling readers -to get frustrated and give up and eventually hate reading." • "When I say it motivated me to read more, I mean more books at a much lower reading level than I needed, simply so I could get more points. No one wanted to be the kid who was losing, so we'd take tests on books we never read or just cheat to get more points. I didn't like it but had to do it so I wouldn't be the one losing." Students who made negative comments about their AR experiences cited perceived pressure to read many (easy) books to earn points and meet goals set for them, while few were willing to read more difficult material and/or longer books. They resented being forced to read books that held little interest for them and were not above cheating to "earn" points, whether or not they deserved them. These participants were well on their way to becoming elementary teachers, yet only 12.2% responded that they would use AR in their classrooms. Because of their success in school and their desire to become teachers, it would be interesting to know to what they attribute their success.
A few students in the Literacy Methods classes gave mixed responses: • "I have always loved to read, and I think I liked AR because I did well, not the other way around. I would love to read, with or without AR." • "AR is only good for some students." Although the researchers initially solicited participant comments to clarify quantitative results, the comments were somewhat at odds with the survey data, leading to more questions than answers. Did avid readers favor AR because of the program's merit, or did AR make them avid readers? Was the competition to earn points for passing AR quizzes a motivator or a deterrent to reading? ISSN 2327-5499 2013

Discussion
College pre-service teachers in past literacy methods classes discussed the AR program and revealed that they had cheated or read easier books to gain points. Some students also stated that the AR program had turned them off to reading. One student, not a part of this study, talked about developing an elaborate plan in second grade (age 7-8) with a friend to win points without reading any books. He and his friend ended up being suspended from school for their efforts. Another student described in detail the cover of the last book he attempted to read in the AR program in third grade (ages 8-9). He stated emphatically that he hated the book, that it was too hard, and that he did not read another book until high school. Lamme (2003) stated that just because students can read at a certain level does not mean that all books at that level will be appropriate or of interest to them.
The Accelerated Reader program may be beneficial for students who are already intrinsically motivated to read. The authors wonder if, perhaps, AR works best with students who are already good readers and see the program as an opportunity to read more books of interest, with or without incentives (points and prizes). It may be that stronger readers are attracted to AR because they are already motivated to read, while struggling readers who may not be intrinsically motivated show few, if any gains, as Melton et al. (2004) found.
Clearly, AR is not the answer for all children and should not be used as the teacher's only "outside" reading strategy. In this study AR seemed to be useful for students who were already motivated to read and enjoyed choosing their own books. Awarding points and prizes seems to frustrate struggling readers and is of questionable value even with fluent readers who may read books below their reading levels only for the prizes and not for the pleasure of reading. AR participants focus almost entirely upon reading level, rather than content, when selecting books, taking virtually any book at their level so that they can rapidly acquire points" (Ishizuka, 2002 as cited in Lamme 2003, p. 40). It may be that the competition aroused by AR implementation discourages thoughtful and pleasurable reading practices among many students. Judging from the responses of college students who had experienced AR in their own K-12 experiences, the program can turn students away from reading if they are forced to participate or encouraged to participate only to earn points and prizes. We question the use of points and prizes when these motivators become the central focus of reading.

Implications
Does the AR program promote higher order thinking? This is a question that educators need to further research concerning the AR program. In relation to Bloom's taxonomy, what level of thinking does it require to pass the AR quizzes? Are students required to use higher cognitive processes like analysis and evaluation, which are not easily written in multiple choice questions, or are they mainly asked to recall and remember what they read? Also, is AR appropriate for all students? Is it appropriate for struggling readers and English language learners? Last but certainly not least, could this program be detrimental to a student's motivation to read? These are important questions to consider for future research.

Conclusion
Motivating students to read is a challenge for teachers. One of the intended outcomes of the AR program is to "motivate students to read constantly." The perceptions of the pre-service teachers did not back this claim even though there were extrinsic rewards. In this study, most participants did not find pleasure and enjoyment in the reading experience. We learn in our literacy courses in college that people read for one of two reasons: either for pleasure or for information. It appears that AR has added a competitive third reason: to earn points and get prizes, in other words, to win. The true rewards of reading are intrinsic like gaining knowledge or losing oneself in a good story. Personal responses about books and sharing them with peers helps students become critical readers. "Literature discussion exposes students to books they have not yet read, completing the reading circle: book selection, book reading, book analysis and response, and book sharing" (Lamme, 2003, p. 43).
With the AR program there is no analysis, responding, or sharing of books. It is more about the number of books read. Offering extrinsic rewards may backfire and turn some students away from the AR program and from reading for personal gain or pleasure. Lamme (2003) comments "too many students leave school without having truly enjoyed a book" (p. 41). AR proposes to develop lifelong readers and learners, motivate students to read constantly and keep all students challenged-statements that this study cannot support. Like many other reading strategies, AR can be a useful tool for teachers, but it cannot be relied upon as a motivator for all students. It needs to be closely monitored and understood that all children may not benefit from it.