Exploring Program Implementation: Demographic, Programmatic and Institutional Predictors of Student Lexile Growth in an Urban School District

The purpose of this research study determined the possible predictors that contributed to student Lexile growth after enrollment in a comprehensive reading program READ 180 for an academic school year. This study explored student demographics, program characteristics, and institutional factors that significantly impacted student Lexile growth. More specifically, this research study determined factors that significantly impacted student Lexile growth after receiving a comprehensive reading intervention for the 2010-2011 school year. The significant findings in the study assists administrators when identifying correct student population, adhering to implementation recommendations, and employing school-level guidelines to ensure fidelity of implementation.


Introduction
The No Child Left Behind educational reform act of 2002 addresses students underperformance on national test data in the following areas: reading, math and science. Although No Child Left Behind has addressed academic performance and produced a modicum of improvement in urban schools settings, the inequalities found in many classrooms in large urban districts remain evident. According to Payne (2010), the school reform movement has been active for two decades and has gone through various phases of implementing policies and practices; and, there appears to be little overall significant improvement in urban school student achievement or school environments during this time. 2009 data indicates the average reading score for twelfth-graders was 2 points higher than in 2005, but 4 points lower than in 1992 (National Assessment Governing Board, NAGB, 2009). There were no significant changes from 1992 to 2009 in the reading score gaps between White and Black students (National Assessment Governing Board, NAGB, 2009).
The National Assessment and Education Progress for Reading (NAGB, 2009) data revealed that on the 2009 fourth grade reading assessment, only 12 percent of black male students nationally and 11 percent of those living in large central cities performed at or above proficient levels compared with 38 percent of white males nationwide. The data indicated that only 9 percent of eighth grade black males across the country and 8 percent black males living in large cities performed at or above the proficient level in reading compared with 33 percent of white males nationwide (NAGB, 2009). Moreover, the average African American fourth and eighth grade male who is not poor fails to perform better in reading on NAEP than white males who are poor; and, black males without disabilities do not perform better than white males with disabilities (NAGB, 2009). The data are replete with evidence of the achievement challenges that plague minority students in large urban districts.
Urban districts are prone to encounter significant challenges when implementing reform efforts, such as the comprehensive literacy program included in this study. National data on the achievement gap in America's public schools continue to reveal that students in urban schools typically score much lower than their suburban and rural counterparts on state-mandated tests (Patterson, Eubank, Rathbun, & Noble, 2010). Thus, a sense of urgency arises when choosing and implementing reform programs in urban districts. School reforms tend to be adopted in a perfunctory manner. Whereby, many executive leadership teams presume that merely the purchase and adoption of new materials, professional development, and equipment associated with the literacy reform will magically transform the district, schools, and classrooms and alleviate the targeted problem (Levin, Catlin, & Elson, 2010).
Due to the data supporting the need for adolescent literacy programs in urban school milieu, many schools and districts implement programs only to discontinue their use after 3 or 4 years. Unfortunately, many urban districts succumb to the habitual adoption practice and accept the turnover of reform ritual as a reality. In lieu of eliminating a reform effort, districts should seek to determine if the absence of effective implementation hinders student performance. Although the research on the importance of implementation is extensive, school organization and operations tend to undervalue the need for strategic implementation International Research in Education ISSN 2327-5499 2015 planning (Levin, Catlin, & Elson, 2010). The district in this study embodies characteristics similar to urban districts across the country. In 2009, the district received scores of 40% on the state assessment in Reading/Language Arts in grades 3-8, falling 10% points behind the state average.
The purpose of this research study determines the possible predictors that contribute to student Lexile growth after enrollment in a comprehensive reading program -READ 180 for an academic school year. This study explores student demographics, program characteristics, and institutional factors that significantly impact student Lexile growth. More specifically, this research study aims to determine the factors that significantly impacted student Lexile growth after receiving a comprehensive reading intervention for the 2010-2011 school year. Furthermore, the primary goal of this study is to identify specific factors to explicate and document the predictors of student Lexile outcomes after being enrolled in an intervention designed to accelerate reading growth for struggling adolescent readers.
There is a plethora of research studies that focus on the reading growth or lack of growth for students enrolled in READ 180 classrooms in urban districts across the country. The vast majority of research on the READ 180 program primarily compares program effects of control groups versus treatment groups. The study is intended to provide insight to student, program and institutional variables that contribute or impede student Lexile growth while enrolled in the READ 180 program.
The implications from this research study will provide educational leaders, policy makers, administrators, and educators with insight on the significance of implementation fidelity in READ 180 classrooms and how this can be a vehicle for obtaining and sustaining reading growth in urban districts. The study can assist administrators when identifying correct student population, adhering to implementation recommendations, and employing school-level guidelines to ensure fidelity of implementation. Additionally, executive district leadership will find the information beneficial when developing an implementation model that requires a tiered approach at the district, school, and classroom level.
The researchers' methodology for this study focused on quantitative methods. Quantitative data were obtained from student usage with the technology component of the program. This study utilized a hierarchal multiple linear regression to discover relationships between three levels of variables and student Lexile growth. The design of this study focused on identifying and exploring multi-level factors that influenced student growth within a district-mandated intervention. The results from this study can guide the implementation of the Response to Intervention Framework within K-12 schools across the country and serve as quantifiable evidence for practitioners regarding the importance of fidelity of implementation as it relates to reading interventions.

Research of READ 180 Implementation Programs
READ 180 has an extensive history of research conducted both internally and externally. This research has allowed policymakers, district administrators, principals, and teachers to become more informed about the effectiveness of the program in various environments. Three studies have been selected to represent a balanced perspective of the available research on the program.

READ 180 Little Rock Study
A mixed method evaluation was conducted in Little Rock, Arkansas was to determine the effectiveness of READ 180 on improving and remediating academic achievement for African American students. Results from this study are timely due the National Assessment of Educational Progress reports on disproportionate number minority students lagging behind their peers in reading. Research has also shown as students move through the grades, the gap between good and poor readers widens (Hasselbring & Groin, 2005). Participants included 1,000 students and 23 teachers. Students' 2006 ITBS and Arkansas State Benchmark scores were the outcome variables in this study. Results indicated that READ 180 student scores from the ITBS and Arkansas Benchmark Literacy exam were not statistically significant for grade 6, 7, 8, or 9. Researchers concluded that students in READ 180 were less likely to obtain proficiency on the Arkansas Benchmark Literacy exam. The qualitative component of the study included classroom observations, focus groups, interviews, and surveys. Although the learning environment was conducive to effective classroom management, active teacher monitoring, and cooperative interactions, researchers observed deviations from the READ 180 instructional model. When fidelity of implementation was evaluated in the classrooms, only 62% of the teacher substantially adhered to program requirements (Mims, Lowther, & Nunnery, 2006). Consequently, the less than mediocre level of implementation fidelity, may explain the lack of student achievement in this study.

READ 180 and the Dropout Rate
The target population for the READ 180 intervention were students enrolled in an urban middle school who experienced reading difficulties (Woods, 2007). The purpose of the study was to investigate the effects of reading interventions on the achievement of students in grades 6-8 and the influence on the dropout rate. Researcher findings consistently show a direct correlation with low literacy skills and the increased probability of dropping out of school. During the first year of implementation, researchers found no significant difference between treatment and comparison groups. The two subsequent years (2005)(2006)(2007) revealed students in READ 180 achieved higher mean gains than students enrolled in a traditional reading program. In 2003-2004 there was a significant difference in the dropout rate for the two cohorts.

Memphis Striving Readers Project
Memphis City Schools received grant from the U.S. Department of Education to improve adolescent literacy skills and the quality of literacy instruction across the curriculum. The ISSN 2327-5499 2015 five-year grant was awarded based on the implementation of the adolescent literacy program READ 180 and Memphis Content Literacy Academy professional development courses to strengthen teachers' instruction in the classroom. The student intervention randomly assigned struggling readers in grades six through eight who meet the eligibility requirements for the program (Schenuk, Coffey, & Feighan, 2010). In Year 1 and 2, program implementation had considerable variability in the eight schools. By Year 3, nine of the thirteen classrooms had improved classroom ratings in Year 3 over Year 2. The statistically analysis for Year 1 and 2 showed lack of significant immediate impact for students enrolled in READ 180. There was a small but significant immediate improvement for sixth grade treatment students on the TCAP Reading/Language Arts measure (Schenuk, Coffey, & Feighan, 2010). Implementation fidelity data for Year 3 significantly improved due to the districts decision to embrace the importance of implementation fidelity. Improved fidelity was the message of the district's executive leadership team. The modest yet significant findings in Year 3 of the Memphis Striving Readers Project conveys the notion that implementation dominates outcomes (Levin, Catlin, & Elson, 2010).

International Research in Education
The aforementioned studies extract typical findings within broader scope of the available literature regarding the effectiveness of the program. To improve the reading skills of students not performing on grade level, an identification and implementation of interventions designed to address the heterogeneity of reading difficulties among older students is paramount.
To expand the study on READ 180 implementation in schools, the following research questions guided this study: 1. After controlling for students' final score on the Scholastic Reading Inventory, which demographic (i.e. gender, special education status, and English language learners) factors contribute to student Lexile growth after being enrolled in a full academic year of READ 180?
2. After controlling for student's final score on the Scholastic Reading Inventory, which READ 180 program specific factors (i.e. student's first SRI score and total sessions) contribute to student Lexile growth after being enrolled in a full academic year of READ 180?
3. After controlling for student's final score on the Scholastic Reading Inventory, which institutional factors (i.e. district location and federal grant status) contribute to student Lexile growth after being enrolled in a full academic year of READ 180?

Methodology
All reading levels are reported in Lexiles within this study. Lexiles are a quantitative measurement of reading growth for students in the READ 180 classrooms. The district implemented specific placement criteria for READ 180 placement for the 2010-2011 school year; however not all schools adhered to this criteria. Data came from students who were enrolled in the program for a full academic year. ISSN 2327-5499 2015 Once the research focus was narrowed and a question was developed, the process for obtaining approval from each institution began. Due to the direct contact with the district and student data, the selection of study participants was obvious. To obtain approval from the district, a form was completed and submitted to the Director of Research requesting data use for research. While the district completed the process, another approval process was initiated at the researchers' education institution. Data for the study was collected, extracted, and analyzed at the conclusion of the 2010-2011 academic school year.

International Research in Education
The Scholastic Achievement Manager housed all student and program data. The institutional data were collected from an internal instrument utilized by Scholastic Inc. to monitor program fidelity. The instrument is known as the Implementation Fidelity Reporting System (IFRS). Data collected on institutions were extracted from this tool which requires one to manually enter data into the online system.
After all data were collected from the district, Scholastic has a process of scrubbing the data to ensure accuracy and reduce anomalies for accurate reporting. The data were provided to the researchers through an online secured VPN access. Data went through several cycles of coding to increase the probability of a meaningful analysis. The research advisor assisted with the analysis of the data with the utilization of SPSS. When the results were entered into the statistical software, researchers went through iterations of the dataset before deciding on the final dataset for this study.

Assumptions
Hinkle, Wiersma, & Jurs (2003) indicate three major assumptions for Multiple Linear Regression. The first assumption is independent variables should be highly correlated with each other but demonstrate low correlations with each other, this is known as multicollinearity. The second assumption states the variance of error is the same across all levels of the independent variable, and this is known as homoscedasticity. The third assumption is variables have normal distributions this is known as normality.

Delimitations and Limitations of the Study
The following delimitations underlie this research project: 1) Students population was extracted from an urban district in the Southeast 2) Student data included only students who were actively enrolled in the intervention in the data file.
The primary limitation of the study is the research was conducted in the researchers' district; consequently, only students enrolled in intervention were included, excluding a quasi-experimental design. Additionally, there were confounding factors the researchers were unable to control for in this study. The following limitations underlie the study: 1) The study only includes data from one school district, thus findings are unable to be generalized to all students in a K-12 setting.
2) Students were assigned to the intervention based on district criteria, thus random

3) Student Lexile growth was determined based on performance on the Scholastic
Reading Inventory which is embedded in the reading intervention, which excludes a comparative analysis of student outcomes employing an additional assessment. 4) Student maturation and differentiation factors were unable to be accounted for between pre-and post test administrations

Setting
The research setting for this study is located within a large urban district serving a predominately African American student population in grades PreK-12. The district represents elementary, middle, high school, and alternative programs. Many schools within the district receive additional Title I funding to serve students who are categorized as low socioeconomic status. The district serves approximately 110,000 students in grades K-12. In a concerted effort to address adolescent literacy, the READ 180 district-wide implementation served 3,000 students in middle and high school. Additionally, the approximately 70 instructors were charged with the responsibility of addressing reading achievement in their classrooms.

Population and Sample
The population for this study includes 2,466 students in grades 6-9 from an urban school district located in the Southeast corridor of the United States. Students in this population can be considered adolescent readers due to their ages ranging from 11-16 years of age. The student population within the district consist of 86% African American, 6% Hispanic, 7% White, and 1% Other. All student data were collected from four internal district databases which contained student related program information. Included in this study were 1,272 female and 1,194 male students enrolled in the intervention. Quantitative methods were satisfied through an analysis of multiple factors related to student, program and district characteristics.
The descriptive statistics of the samples as it relates to gender, special education status, and English Language Learner are referenced in Table 1-3. As shown in Table 1, female students represented a slight majority (52%) over male students (48%). In regard to students identified as receiving Special Education services in the database system, ISSN 2327-5499 2015 the data below is not an exhaustive representation. Since teachers were required to select this information, the data reported in Table 2 reveals that the majority of the students in the database did not have their special education status identified.  Table 3 reports students identified as Language English Proficient in the sample. As noted below, the majority of students did not have their status selected in the database.

Descriptive Statistics
Students included in the study were identified as enrolled in the intervention program when data were extracted from the district databases. Results from this study indicated that on average students participated in the program for a semester or more. Of the 2, 466 students, the total sessions or days (M=83.25 SD=29) indicate the students in this sample consistently participated in at least the instructional software component of the intervention program. The researcher concluded that student participation in the software component of the program increases the likelihood that the READ 180 instructional model was implemented in the classroom. Based on previous studies designed to examine the outcomes of computer software on student performance, media enhanced technologies have been researched to determine the effectiveness of facilitate learning (Hasselbring & Goin, 2004). The READ 180 program is a result of a prototype from the Peabody Learning Lab housed within Vanderbilt University. According to the initial developers of the program, the main purpose for creating these types of environments makes it possible for at-risk students to develop their literacy skills, i.e.: language, mathematics, and thinking skills, in an engaging, supportive environment (Hasselbring & Goin, 2004).
During the implementation of the reading intervention program, the district collaborated with Scholastic to develop placement criteria for students. The initial Lexile score of students (M= ISSN 2327-5499 2015 637.16 SD=219.82) determined student eligibility for the READ 180 program. The researchers concluded that student placement contributed to positive student outcomes. In a qualitative study conducted by Salinger, Toplitz, Jones, Moorthy and Rosenthal (2010), researchers examined factors that contribute successful implementation of READ 180. One of the ten recommendations researchers identified to promote fidelity of implementation was ensuring that appropriate students are targeted by adhering to program recommendations for student placement and exit criteria and creating guidelines for their use was (Salinger et. al, 2010).

Correlation and Regression Analysis
When intervention programs are implemented at the district level, evaluation of the effectiveness is paramount to ensure the district's return on investment. The researchers hypothesized that when students were properly targeted for the intervention and consistently utilized the program student reading growth would ensue. The researchers used Linear Multiple Regression to examine the relationship between student demographic factors, programmatic factors, and institutional factors. The following section provides data regarding the research questions in the study. Each research question is presented with an analysis of the data. The information was extracted from the Scholastic Achievement Manager database in regard to student program use.

Data Analysis
Statistical analysis for the research study employed Hierarchical multiple linear regression. In order to gain greater depth, a follow-up comparing dummy-coded variables utilizing a series of six orthogonal contrasts was conducted. For this statistical model, the dependent variable was students' final Lexile score. Independent variables analyzed employed hierarchical liner regression to determine relationship with the dependent variable.
The purpose for this research method was to investigate the relationship between the factors that increase and hinder student achievement in a comprehensive reading intervention. These factors were analyzed to discover the associations between improvement and hindrances related to student Lexile growth. Data analyzed were student results from the technology component of the software, demographic indicators, and institutional characteristics. Hierarchical multiple linear regression was utilized to gain greater insight about the relationship between several factors in explaining student Lexile growth and the level of significance in the classrooms. Associations from this analysis were linked and identified to make predications on how reading interventions can be implemented to obtain maximum results for adolescent readers.

Research Question 1
After controlling for students' final score on the Scholastic Reading Inventory, which demographic (i.e. gender, special education status, and English Language Proficiency status) factors contribute to student Lexile growth after being enrolled in READ 180?
In order to understand how each independent variable contributed to student Lexile growth (dependent variable) after being enrolled in the program, a multiple regression analysis was performed to examine the relationship between Lexile growth and eight predictor variables: Language English Proficiency, special education status, gender, grade tier, READ 180 total sessions, first Scholastic Reading Inventory assessment, striving reader sites, and district regions). The significance level was set at .05. The overall model is significant, so we reject the null hypothesis, given (F= 162.142), meaning the correlation between the criterion variable and the combined predictor variables is statically significant.
As seen in descriptive statistics Table 6, English Language Proficiency, Special Education status, and grade tier are statistically significant in the regression model. The multiple regression model with all four variables produced an (R 2 = .209). All together, the four categorical variables accounted for almost 20% of the variance in students' final SRI score. Three of the independent variables had a significant relationship with students' final Lexile score. Language English Proficient had a significant relationship with student Lexile growth (t=-5.140, p<.05). As did, student special education status (t= -17.447, p <. 05). Student grade tier also had a significant relationship with the outcome variable (t=18.244, p < .05). As students' grade level increased, final Lexile score increased. ISSN 2327-5499 2015  A second regression model was utilized to examine the relationship between student Lexile growth and READ 180 software session and students' initial Lexile score on the Scholastic Reading Inventory. The researcher hypothesized the two programmatic variables would significantly contribute to student Lexile growth. Again, the ENTER method was utilized with all six independent variables. In a previous study conducted on students enrolled in a READ 180 afterschool program, findings suggested the computer aided reading instruction predicted improvements on measures of sight word reading efficiency and reading vocabulary (Kim et al., 2010).

International Research in Education
As seen in descriptive statistics Table 7, READ 180 total sessions and students' initial SRI score was significant in this model. The significance level was set at .05. Collectively, the six independent variables accounted for approximately 65% of the variance in students' end of the year Lexile growth (R 2 = .209). After adding the two variables to the model, the (R 2 change=.445) added an additional 44% of the variance to the model, thus adding significance to the regression model.

Conclusion
The focus of the research project was to examine the relationship between Lexile growth of students enrolled in the READ 180 intervention program and possible predictors that include student demographics, program characteristics, and institution location. The analysis corroborates with similar qualitative findings as it relates to program implementation. This analysis provides additional evidence regarding the importance of ensuring reading interventions are properly implemented to service the appropriate student populations.
In the past, studies conducted to determine the effectiveness of student performance in a reading intervention typically compare control groups. Albeit significant to the literature, the researchers in this study purport the need for educational leaders and practitioners to begin developing a framework to ensure students enrolled in reading interventions fully benefit from program components.
Fidelity of implementation within the context of reading instruction has been addressed sparingly in the literature. The most recent qualitative study determined and described key components to ensuring READ 180 is implemented with the highest fidelity. Although, READ 180 specific, the recommendations included in the study have potential applicability for other reading interventions. As researchers, educators, leaders, and policy makers determine the trajectory of successful program implementation this study augments the literature regarding the importance of determining what student, program and institutional factors must be considered when predicting possible student outcomes.
If one expects systematic and meaningful change in education, the approach to ensuring fidelity of implementation will need to be proactive. As reading researchers and experts seek to determine how to address this aspect of teaching and learning, specific questions guide their desire to explore the topic more effectively. The results from this study can transform future conversation as it relates to discovering proven frameworks for improving student literacy acquisition and identifying implementation rubrics or continuums to assist teachers with assessing implementation effectiveness (Cooter & Perkins, 2011). This study serves as a prelude to exploring the fidelity of implementation of READ 180 in an effort to produce knowledge directly applicable to implementing reading intervention programs for middle and high school students in America's public schools.