ALL IN — designing a statewide assessment-literacy workshop series
Designed the activities and materials for a professional-learning program that helped Indiana educators read and use assessment data, revised it three times from educator feedback, and tripled enrollment.
Context
Indiana teachers and administrators receive a steady flow of assessment scores, but few have had training in what those scores can and cannot tell them. Assessment Literacy Leaders in Indiana (ALL IN) was a professional-learning program to close that gap. Its topics were technical: how tests are built, measurement error, and how to interpret a score.
David and Leslie Rutkowski, two assessment faculty at IU, led the program. I ran it day to day: I designed the workshops, ran the research that shaped them, organized outreach, and scheduled the schools and districts we visited.
What I did
Designed the workshop activities and materials. The team first built an alignment document that matched each assessment topic to candidate activities. I then designed nearly all of the activities and materials, including the ones that paired with the faculty’s lecture segments, and led some segments myself. The goal was to make technical ideas hands-on. In one activity, educators got index cards, each naming one step in building a test, and worked together to put the steps in order.
Other activities put educators in a realistic situation. In the one below, a teacher plans to group students by their scores on Indiana’s state test. Educators worked through a sample student score report, the parent letter, the test blueprint, and the item specifications, then decided whether her reading of the scores holds up.
Open the score-report activity (PDF, 7 pages)
Revised the program from evidence. I surveyed more than 1,000 Indiana educators about their assessment practices, and interviewed and observed educators in the workshops. We revised the curriculum three times, with district officials, teachers, and administrators. The first two versions were two-day workshops. Field notes and interviews showed that the technical details confused people more than they helped. The third version was one day, built around a few key takeaways, and focused on the interim assessments teachers use in their classrooms rather than the state test.
Changed how the program was delivered. We started by traveling to individual schools. I led a change to bring educators to IU instead.
Outcome
Enrollment in the program tripled, and more than 100 educators took part. I presented the work at the NCME Special Conference on Classroom Assessment and gave the keynote at the Indiana Department of Education’s 2019 Assessment Literacy Conference.