July 28, 2017

Emerging Best Practice for Determining ROI #1

As part of my dissertation, I’ve compiled a list of what I think are emerging best practices for determining rate of improvement.  In other words, what are the necessary conditions for having confidence in the ROI statistic?  The first emerging best practice I’m considering is the use of technically adequate and psychometrically sound measures.  If we don’t have good data, there’s no point in creating a trend line or calculating an ROI statistic.  One of the best resources is the progress monitoring tools chart.  Note that this chart can still be accessed through the National Center on Response to Intervention but is now housed at the National Center on Intensive Intervention.  Not all assessments are located on this chart as they are submitted voluntarily but it’s good place to start if your school team is looking for ideas.

When school-based teams are considering making high stakes decisions such as special education eligibility, the quality of data needed is much higher.  Teams need to consider if the assessments they are using is measuring what they intends to measure (validity) and if they can measure skills consistently (reliability).  Other aspects of assessments to consider in relation to generating a stable trend line are whether the measure can demonstrate small increments of growth and if the assessments can be repeated through alternate forms.  An example of a technically adequate measure that was designed to produce ROI is the computer adaptive tests from Renaissance Learning called STAR Reading and STAR Math.  After only four data points, the system will generate a stable trend line that can be used to interpret student progress.  A non-example would be teacher-made assessments or unit tests.  The latter measures may be helpful for teachers to know how students are performing with concepts they are learning in class, but have not been validated for the purpose of generating trend lines or ROI from the results.

Questions and comments are welcome! What are your school teams using to document student progress?

FacebookTwitterPrintFriendlyPinterestTumblrGoogle+EmailShare

Q&A – Slope for standard scores

ROI6

Great question from Lavonne:

My question is similar to that of Jen on July 18, 2011. I am wondering if Rate of Improvement can be graphed and slope comparisons used with standard scores such as those generated by STAR Reading. We also have standard scores on our statewide assessments and could easily generate Excel graphs to show the difference between our student’s performance and that reuqired to pass. We can easily generate our own Excel graphs, we just want to make sure the data is valid.Thank you.

P.S. Thanks so much for this site!

My response:

You can calculate slope for scores that have an equal interval between data points.  STAR Reading, STAR Math, and STAR Early Literacy is a good example of non-CBM data and has been validated as a data set for which you can calculate slope.  Joe Kovaleski and colleagues (2013) just published a book that describes using rate of improvement with computer adaptive tests (CATs), specifically with STAR assessments. I highly recommend getting a copy!

Kovaleski, J. F., VanDerHeyden, A. M., & Shapiro, E. S. (2013). The RTI approach to evaluating learning disabilities. New York, NY; Guilford Press.

FacebookTwitterPrintFriendlyPinterestTumblrGoogle+EmailShare

Progress Monitoring… Life?

It’s been a while since I’ve posted anything! Let me explain. A former Tough Mudder teammate and I were eyeing up the same marathon and decided to give training a go back in December! He signed up right away, but I decided to train first and sign up if I could get through the first couple of months. Well after a dozen or so weeks of tracking my running progress, I signed up! The marathon was last Saturday and it was definitely an amazing (and painful!) sense of accomplishment. I realized how much I benefitted from seeing my own progress and will continue to apply those same principals to the students (and teachers!) with whom I work. Have you ever used data “momentum” as encouragement at school or in your own life? Do tell!

FacebookTwitterPrintFriendlyPinterestTumblrGoogle+EmailShare