December 12, 2017

Q&A – Realistic vs. ambitious goals for early literacy skills

ROI6

 

Great question from Beth:

I have seen where there is a chart to show what a realistic vs. ambitious goal for Rate of Improvement is for R-CBM or Oral Reading Fluency but is there a chart like that for Letter Naming Fluency or Letter Sound Fluency?  Or can the ROI charts for R-CBM charts be used for other areas of progress monitoring as well??

Response:

The majority of research studies that I have come across have been related to determining expected growth for oral reading fluency, most likely because it is the most popular form of CBM.  The study below is the study where the growth charts originated from for digits correct and oral reading fluency.

Fuchs, L. S., Fuchs, D., Hamlett, C. L., Walz, L., & Germann, G. (1993). Formative evaluation of academic progress: How much growth can we expect? School Psychology Review, 22, 27-48.

The idea of realistic and ambitious goal setting is also described in more detail in a chapter of Best Practices for School Psychologist, Volume 5.

Shapiro, E. S. (2008). Best practices in setting progress monitoring goals for academic skill improvement. In A. Thomas and J. Grimes (Eds.), Best practices in school psychology V.  (Vol. 2, pp. 141-157). Bethesda, MD: National Association of School Psychologists.

The amount of growth you can expect from students is dependent on a number of factors such as the intensity, duration, and frequency of the intervention; the skill being measured, and the fidelity with which the intervention is delivered.  You may want to see if the assessment system you are using includes norms for rate of improvement.  If none are provided, then you could calculate the typical ROI for students based on the expected benchmark scores and use that as a comparison.  Remember that students scoring below benchmark would require an ROI that is more than the ROI of their typical peers in order to close the achievement gap.  There are many dissertations to be had in this area! It would be great to have more published, peer reviewed studies that review the progress students typically make with specific skills given specific interventions.

FacebookTwitterPrintFriendlyPinterestTumblrGoogle+EmailShare

Q&A – Using ROI for principal evaluations

ROI6

Great question from Kim:

I am a principal in Illinois and am required to set student growth goals as a part of my annual evaluation. I am trying to get a clear understanding of ROI and came across your powerpoint presentations online. I am hoping you can help me understand the rational of the procedure I am being told to use.First, I calculate the growth rates for students using their BOY and EOY DIBELS scores from last year. Then I’m supposed to multiply that by 1.5 (Fuchs’ ambitious goal) to determine each student’s ROI for this year from BOY to MOY using AIMSweb (the district changed measures this year). Finally, I set an overall growth target by which I will be evaluated. For example, “75% of 3rd grade students will reach their ROI targets from BOY to MOY benchmarks.”

I am struggling with the constant, 1.5. I can see setting ambitious goals for students below benchmark, but if a student is already well above benchmark, and will obviously not receive Tier II or Tier III intervention, is it reasonable to expect their rate of improvement to increase 150%? Any advice you can provide will be GREATLY appreciated.

Response:

Andy and I had responded to this email individually, but this a great point that should be mentioned.  There are a couple of studies that illustrated that students performing at the highest and lowest levels on benchmark assessments tend to make the least amount of growth.  That doesn’t mean we can’t expect students to make gains, but we need to be aware that our highest and lowest performing students will require extra effort to improve their achievement gains over time.

Fien, H., Park, Y., Baker, S. K., Smith, J. L. M., Stoolmiller, M., & Kame’enui, E. J. (2010). An examination of the relation of nonsense word fluency initial status and gains to reading outcomes for beginning readers. School Psychology Review, 39, 631-653.

Good, R. H., Wheeler, C. E., Cummings, K. D., Baker, S. K., Fien, H., & Kame’enui, E. J. (2010, March). Rigorous RtI decisions: Normative growth rates for oral reading fluency. Paper presented at the meeting of the National Association of School Psychologists Annual Convention in Chicago, IL.

Silberglitt, B., & Hintze, J. M. (2007). How much growth can we expect? A conditional analysis of R-CBM growth rates by level of performance. Exceptional Children, 74, 71-84.

FacebookTwitterPrintFriendlyPinterestTumblrGoogle+EmailShare

Calculating RoI for non-CBM

We’re starting this Q&A section with a great question submitted by Jen on 2011/07/15 at 2:57 pm as a comment on the following post:

http://rateofimprovement.com/roi/2011/02/22/nasp-convention-2011/#comments

I am in charge of RtI for our district and this year we are going from use of AIMSWeb, which made this simple, to use of DRA and Fountes and Pinnell Benchmark Assessment System. These systems, unlike AIMSWeb, only offer guided reading levels, but will not be helpful in progress monitoring more frequently unless we can determine a productive rate of improvement. We are looking to have our students gain 1.5 years in 1.0 year. Any thoughts on how we can create a formula for this for teachers to use so they aren’t just guessing. DRA and F& P by the way do not go by 1 level at a time but more like 2, 4, 6 and they are not the same. I was thinking to look at 1.5 years ahead of whereever the student is currently at and expect that for the entire year and then break that down by the # of weeks in the year to determine where the student should be at each week.

Your thoughts?

We also wanted to figure out a way to calculate in the strategies expected at those levels, such as this book level, but also this many (2) strategies every 2 weeks.

Again, your thoughts?

Thank SO much!
Glad I found this site!
Jen

FacebookTwitterPrintFriendlyPinterestTumblrGoogle+EmailShare