Here we present brief reports of growth measures, models, and polices draws from more comprehensive writings.
Using Effect Size to Evaluate Achievement Gaps (In-Brief)
This research brief provides an overview of methods for estimating the size of achievement gaps. Unfortunately many researchers, analysts,and policy-makers use subjective methods (e.g., visual inspection) to evaluate group differences. Another common method is to take the difference between groups in percent proficient, but there are a number of shortcomings to this approach. A more objective and well tested method to compare groups is effect size (ES). I define ES and demonstrate its computation in several situations including between-group comparisons and within-group change over time. I also discuss newer methods that are appropriate for categorical and ordinal data, methods that examine group differences across the whole score distribution, and tools for visualizing achievement gaps. Stevens, J. J. NCAASE_Research Brief
Growth Models for Students with Significant Cognitive Disabilities (In-Brief 1)
Alternate assessments based on alternate achievement standards (AA-AAS) are designed to measure the academic achievement of students with the most significant cognitive disabilities (SWSCDs). AA-AAS present unique measurement challenges because of the inherent need for individualization in item presentation and response, combined with expectations for rigorous levels of standardization. Additional measurement challenges are presented as states move toward implementation of growth models for accountability. This brief addresses some decisions that states face as growth models with AA-AAS are implemented. Farley, D., Saven, J. L., and Tindal, G. NCAASE_Research Brief_1
Opportunity to Learn: A Key Access and Validity Issue of Academic Assessments for Students with Disabilities (In-Brief 2)
An underlying assumption in our current test-based accountability system is that all participating students have an opportunity to learn the tested academic content. For students with disabilities (SWDs), this assumption is stated clearly in federal legislation intended to ensure their access to the same academic standards that define the general curriculum of students without disabilities (SWODs). To date, few research studies have systemically examined this assumption. Operationalizing the concept of opportunity-to-learn (OTL) and assessing OTL via measures that can account for teachers’ instructional provisions to the overall class and individual students have been a major obstacles to systematic inquiries into OTL. One NCAASE study was designed to (a) describe OTL for students with and without disabilities and (b) examine the relationship between OTL and students’ end-of-year achievement and within-year growth. This research brief provides an overview of the construct of OTL and how it is being measured in the NCAASE study on OTL and student growth. Kurz, A. and Elliott, S. N. NCAASE_Research Brief_2
Constructing Alternate Assessment Cohorts: An Oregon Perspective (In-Brief 3)
Longitudinally modeling the growth of students with significant cognitive disabilities (SWSCDs) on alternate assessments based on alternate achievement standards (AA-AAS) presents many challenges for states. The number of students in Grades 3-8 who remain in a cohort group varies over time, depending on the methods used to construct the longitudinal cohorts. The accuracy of inferences made about student growth from such longitudinal models is further complicated when general assessments are not equated with AA-AAS which is usually the case. Measuring the growth of students switching test types is thus problematic. States mus determine their priorities: measuring the growth of only AA-AAS scores that count towards making Adequate Yearly Progress under the No Child Left Behind Act (2001) or providing a more detailed look at the growth of SWSCDs over time. Saven, J. L., Farley, D., and Tindal, G. NCAASE_Research Brief_3
Learning to Read: Kindergarten Readiness Growth in Reading Skills (In-Brief 4)
Much of the research on curriculum-based measurement (CBM) in reading has focused on oral reading fluency (ORF). However, ORF is only one of five critical reading skill areas in the wider construct of reading that includes foundational skills such as phonological awareness and phonics. In this research brief, we address the construct of readiness in learning to read entering Kindergarten, and then redirect the findings to the results from learning to read over both Kindergarten and Grade 1. Our results suggest that ‘readiness’ may be defined more by social-behavioral indicators than by more strictly academic skills, and that while students enter Kindergarten with low levels of early literacy performance, on average, they appear to learn at dramatic rates. Irvin, P. S., Alonzo, J., Nese, J. F.T., and Tindal, G. NCAASE_Research Brief_4
Measuring Math Growth: Implications for Progress Monitoring (In-Brief 5)
Achievement growth in math is often framed in the context of monitoring student progress within a Response to Intervention (RTI) approach to teaching and learning. In this research brief we report on a study that examined initial status and within-year growth for fourth grade students who received short progress-monitoring assessments in math during the 2011-2012 school year. Our results suggested that while growth in math was statistically observable and linear, given the amount of growth relative to the 16-point scale of the progress-monitoring probes used, it might have limited utility to teachers operating within an RTI framework. Additional research on within- year growth in math is suggested, especially as it relates to the influence of assessment design and instructional practices on such growth. Alonzo, J. and Irvin, P. S. NCAASE_Research Brief_5
Oral Reading Fluency Growth: A Sample of Methodology and Findings (In-Brief 6)
For the past 20 years, the growth of students’ oral reading fluency has been investigated by a number of researchers using curriculum-based measurement. These researchers have used varied methods (student samples, measurement procedures, and analytical techniques) and yet have converged on a relatively consistent finding: General education students grow about a word per week in their fluency during the academic year, and students in special education progress more slowly. In this paper, we chronologically review early and late studies done on growth of fluency and then compare the studies on samples, measures, and conclusions. Tindal, G. and Nese, J. F. T. NCAASE_Research Brief_6
Risk-Category Transition Patterns for Students in Grades 2 to 4 (In-Brief 7)
Early identification of students who are at-risk for reading difficulties (RD) is critical for successful and effective RTI implementation. A large group of students were followed for three years (grades 2 to 4) to investigate transition patterns of risk-categories across three years. Results indicated that although students identified at high-risk for RD in earlier grades make similar growths to their peers who are not at-risk, they are highly likely to stay in the high-risk categories. Also, low-risk students with flat growth from fall to winter are at greater risk for RD in the later grades. Providing additional instruction to students at-risk regardless of their growth in earlier grades may prevent them from lagging behind further from their peers. Park, B. J. NCAASE_Research Brief_7
Statistical Test for Latent Growth Nonlinearity with Three Time Points (In-Brief 8)
Curriculum-based measurement (CBM) is a system of assessment used to screen for students at risk for poor learning. CBM benchmark screening assessments are typically administered to all students in the fall, winter, and spring, and these data are frequently used by researchers to model and perhaps explain within-year growth. Modeling growth with three time points involves distinct choices in the functional form of growth that can be modeled as well as the parameters that can be estimated. Generally, only a linear growth model can be fully specified; however, research suggests that within-year CBM growth is often nonlinear, with decreasing or decelerating growth across the year. The purpose of this brief is to demonstrate how an estimated slope factor loading approach can be used in a structural equation modeling (SEM) framework to obtain a direct statistical test of nonlinearity provided by the output of your SEM program. Nese, J. F. T. NCAASE_Research Brief_8
The Plateau of Oral Reading Fluency Growth: A Preliminary Investigation (In-Brief 9)
The use of oral reading fluency (ORF) as a pre-dominant measurement tool for identifying struggling readers has grown exponentially over the past 30 years. ORF growth using curriculum-based measures (CBM) has also become an important practical and empirical issue influencing the field. Although fluency scores have shown to be reliable predictors of reading performance up through middle school for poor readers, the point at which assessments of ORF fail to provide additional information is currently unknown. The purpose of our paper was to explore a range of fluency scores at which ORF growth plateaus as a potential guide to educators and researchers for ceasing ORF assessments. Results showed that the growth trajectory sharply increased in Grades 3 – 5, whereas the growth trajectory slightly increased from Grade 5 to Grade 6. The plateau range of ORF scores was 37.72 – 170.94 words read correctly per minute (wcpm). Grades 7 and 8 data are needed to further investigation as we found the time point for peak growth was beyond our current data. Patarapichayatham, C., Nese, J. F. T. and Sáez. L NCAASE_Research Brief_9
Cross Sectional and Longitudinal Portrayals of the Special Education Mathematics Achievement Gap (In-Brief 10)
We contrasted portrayals of the achievement gap on the North Carolina state mathematics assessment for students with disabilities (SWD) using two different ways of identifying this group of students. The first method used students’ disability status in third grade as the basis for identifying SWD, and then tracked the achievement gap between SWD and students without disabilities (SWoD) across grades. With our second method, we allowed student disability status to change at each grade, so that only students who were receiving special education services in a particular year were considered members of the SWD subgroup for that year. Schulte, A. C. & Stevens, J. J. NCAASE_Research Brief_10
Mathematics Achievement Gaps for Elementary and Secondary Students: The Influence of Opportunity to Learn and Special Education Status (In-Brief 14)
We examined the relationship among key instructional process variables and the achievement of students with and without disabilities as measured by both interim and end-of-year summative assessments. Examination of the instructional process data indicated that students with and without disabilities receiving mathematics instruction in the same classrooms had virtually equal opportunities to learn (OTL), yet there were significant differences in these two groups of students’ mathematics achievement on both interim and summative tests. Subsequent regression analyses indicated that the collection of five OTL scores, along with Grade Level and Special Education status, accounted for 44% of the variance in student’s end-of-year mathematics scores. Discussion of these results focuses on equality and equity of opportunity to learn and the role these processes may play in interim and end-of-year achievement gaps between students with and without disabilities. Elliott, S. N., Kurz, A., Tindal, G., Stevens, J. J., & Yel, N. NCAASE_Research Brief_14