Data Tip #8: "When we looked at our state criterion-referenced tests (CRT) for sixth grade, life science was our weakest strand. We couldn’t believe that. We thought we had a pretty strong life science program. It wasn’t until we looked at our own local assessments and saw the same weakness that we became convinced that we had to take a closer look at what we were teaching and how."
(Love, Nancy et al., The Data Coach’s Guide to Improving Learning for All Students, 2008, p.7)
All too often state test results may be the only source consulted when targeting specific areas for improvement. However, decisions about instructional changes that reflect only this single data source, might lead to errors in your decision-making.
If you want your data to lead you toward making meaningful changes, an important principle to follow is triangulation. Triangulation means using three independent data sources to examine apparent issues or problems. You might ask, “Why bother with the extra work of triangulating?” Consider this analogy:
A third-grade teacher asks Mary to look through the front panel of the classroom terrarium and list everything she sees. Mary diligently makes a thorough list and begins to return to her seat when the teacher asks her to take a second look through the side panel of the terrarium. She immediately sees several plants and animals obscured in the front panel view by rocks and shrubs. By using this second “window,” Mary now has a more complete picture. Then the teacher asks Mary to peer through the top of the terrarium to see if there is anything else. Mary is able to add to her list before she sits down. Her three-window analysis reveals a far more comprehensive picture than any one window alone.*
The notion of using multiple windows or perspectives also applies to understanding and applying information from student achievement data.
Action Steps
- Since state test data are the most widely publicized and tend to attract the most attention, this is a good starting point. It’s beneficial to thoroughly examine aggregate and disaggregate state data, including digging down into strand and item data, if available.
- Carefully note specific weak achievement areas. Is this weakness across-the-board or for specific demographic groups? Be sure to note achievement gaps between demographics groups.
- When district benchmark or performance assessment data become available, similarly analyze this data and compare results to your state data observations. Do these assessments show similar gaps? For the same populations?
- Now carefully examine student work samples that focus on concepts noted as weak areas in the other tests. What are specific things students can and cannot do/explain? If these samples are common grade-level assessments, they can reveal even more insights.
- Based on your comparisons of findings across data sources, you are ready to consider action. You may realize you need to adjust alignment between your curriculum and assessments, or provide re-teaching of some skills, or address needs for professional development. Or you may find that looking at additional data is required.
Although we have suggested three types of data to consult during the triangulation process, consider all the data sources available to you. End-of-unit tests, informal formative assessments, classroom observations, and teacher and student surveys each offer unique perspectives. A variety of data sources can support or contradict previous data findings and clarify insights about problems and their causes. Use all the rich resources available to you to help understand what changes will offer the most gain.
Triangulation has the following benefits:
• It can compensate for the imperfections of some assessments.
• When multiple measures yield the same results, it can increase your confidence in the results and assure that you know where to focus reteaching or curriculum adjustment.
• When multiple measures fail to yield the same results, it will raise important follow-up questions.
Written by:
Diana Nunnaley, Using Data Director
Mary Anne Mather, Using Data Facilitator
*Segments excerpted from Love, N. Using Date/Getting Results: A Practical Guide for School Improvement in Mathematics and Science. (2002). Christopher-Gordon Publishers, Inc., p.37-38.
|
