November 23, 2017

Q&A – Realistic vs. ambitious goals for early literacy skills

ROI6

 

Great question from Beth:

I have seen where there is a chart to show what a realistic vs. ambitious goal for Rate of Improvement is for R-CBM or Oral Reading Fluency but is there a chart like that for Letter Naming Fluency or Letter Sound Fluency?  Or can the ROI charts for R-CBM charts be used for other areas of progress monitoring as well??

Response:

The majority of research studies that I have come across have been related to determining expected growth for oral reading fluency, most likely because it is the most popular form of CBM.  The study below is the study where the growth charts originated from for digits correct and oral reading fluency.

Fuchs, L. S., Fuchs, D., Hamlett, C. L., Walz, L., & Germann, G. (1993). Formative evaluation of academic progress: How much growth can we expect? School Psychology Review, 22, 27-48.

The idea of realistic and ambitious goal setting is also described in more detail in a chapter of Best Practices for School Psychologist, Volume 5.

Shapiro, E. S. (2008). Best practices in setting progress monitoring goals for academic skill improvement. In A. Thomas and J. Grimes (Eds.), Best practices in school psychology V.  (Vol. 2, pp. 141-157). Bethesda, MD: National Association of School Psychologists.

The amount of growth you can expect from students is dependent on a number of factors such as the intensity, duration, and frequency of the intervention; the skill being measured, and the fidelity with which the intervention is delivered.  You may want to see if the assessment system you are using includes norms for rate of improvement.  If none are provided, then you could calculate the typical ROI for students based on the expected benchmark scores and use that as a comparison.  Remember that students scoring below benchmark would require an ROI that is more than the ROI of their typical peers in order to close the achievement gap.  There are many dissertations to be had in this area! It would be great to have more published, peer reviewed studies that review the progress students typically make with specific skills given specific interventions.

FacebookTwitterPrintFriendlyPinterestTumblrGoogle+EmailShare

Q&A – Rate of improvement for EasyCBM

ROI6

Great question from Leasha:

Please help! Our new state policy manual has listed your website as the place to go for calculating our rate of learning for the upcoming school year. Pretty much every county in the state uses Dibels so it is set up and ready to go for them. However, my county uses EasyCBM for reading. Is it possible for me to just edit the benchmark goals, and then the Excel would calculate it properly for students entered, or would I need to do some special tweaking?

My Response:

Absolutely! You can adjusts the benchmarks to whatever you need them to be and the Excel sheet should automatically adjust the calculations for ROI.  Good luck!

FacebookTwitterPrintFriendlyPinterestTumblrGoogle+EmailShare

Q&A – Slope for standard scores

ROI6

Great question from Lavonne:

My question is similar to that of Jen on July 18, 2011. I am wondering if Rate of Improvement can be graphed and slope comparisons used with standard scores such as those generated by STAR Reading. We also have standard scores on our statewide assessments and could easily generate Excel graphs to show the difference between our student’s performance and that reuqired to pass. We can easily generate our own Excel graphs, we just want to make sure the data is valid.Thank you.

P.S. Thanks so much for this site!

My response:

You can calculate slope for scores that have an equal interval between data points.  STAR Reading, STAR Math, and STAR Early Literacy is a good example of non-CBM data and has been validated as a data set for which you can calculate slope.  Joe Kovaleski and colleagues (2013) just published a book that describes using rate of improvement with computer adaptive tests (CATs), specifically with STAR assessments. I highly recommend getting a copy!

Kovaleski, J. F., VanDerHeyden, A. M., & Shapiro, E. S. (2013). The RTI approach to evaluating learning disabilities. New York, NY; Guilford Press.

FacebookTwitterPrintFriendlyPinterestTumblrGoogle+EmailShare

Defining Rate of Improvement

If you were to open your old graduate level stats book, you might not find the phrase “rate of improvement,” in the index, but you would find some text on “slope.” Essentially, they are synonymous terms, one being slightly more angled toward the positive.

In algebraic terms, rate of improvement can be defined as the vertical change (y-axis) over the horizontal change (x-axis). More simply put, slope is the rise over run. Or the steepness of a line. The key word here is line. In order to calculate slope, one must first have a line. Once a line is determined, the formula for calculating slope is:

m = (y2 – y1) / (x2 – x1)

m = slope
(x1, y1) = one point on the line
(x2, y2) = a second point on the line

Typically, when we plot student data, we end up looking at data points on a graph. Some commercially available systems provide a general line as a guide approximating where the student’s data points should fall. With just this information, there is no line from which we could calculate an accurate slope. Therefore, we have to create that line! It is the position of the authors of this site that linear regression is the best method for calculating an accurate line to determine rate of improvement.

FacebookTwitterPrintFriendlyPinterestTumblrGoogle+EmailShare