Tuesday, October 29, 2013

An Easy Method for Collecting and Analyzing Data

Hi, it's Chris from over on Autism Classroom News here to talk about one of my favorite topics: Data collection.  We know we need to do it but figuring out how to take the data, teach the kids, manage the classroom and analyze the data to use it to make decisions is simple, right? Not in my experience, and probably not in yours?  Data collection is probably one of the biggest issues that I see teachers struggle with.  Over the years I've developed a number of strategies to help teachers take data more easily (I hope) in the classroom and systems for analyzing the data.  I'm a strong believer that you are going to use different types of data collection depending on both the skill being assessed as well as the context you are teaching in.  I reviewed some of these types of data in a post over on my blog.  However, I wanted to share some basic tools that might be helpful in meeting some overall guidelines for data collection.

1.  Take data in real-time if at all possible

Data is best collected at the time that the skill or behavior occurs.  Waiting to take the data means that you are relying on memory and your memory is never quite as good as you think it is--especially as you get older.  When you rely on memory you may only remember the times that fit with what you think is happening. If you think the skill is increasing, you are likely to remember the times it occurred and forget the times that it should have happened but didn't.  The data sheet below can be used to track the behavior in real-time of one student across different activities or multiple students within one activity.  You can check out this post on my blog to download a copy and for directions on how it can be used.

2.  Collect data consistently and systematically.

Data is only as good as the method that is used to collect it.  The old adage, garbage in, garbage out definitely applies.  You don't have to take data on every single instance of a skill or behavior you are tracking.  However, you do need to take data in a way that is planned and consistent and not just haphazardly.  Again, if you take it whenever you think about it, you are back to taking data based on your beliefs about what is happening (i.e., you think to take data when the behavior matches what you expect).  So, you can take data as a sample as long as you sample consistently and in a planful manner.  So you take data two times a week on the playground to track social initiations, on Monday and Thursday every week.  You record each social initiation observed and the length of the playground session (if it varies) so you can figure out how frequently they occur.  This then gives you a consistent sample to track the behavior over time and you will be comparing apples to apples when you analyze the data.

3.  Make a habit of reviewing the data weekly.

If you wait to summarize and analyze the data until it's time for progress reports, and the student is progressing, then you have wasted at least 6-7 weeks of instruction.  Data is important because it tells us when we have to change our instruction.  Making a habit of summarizing and reviewing the data every week also means that you don't have to stay up half the night or spend all weekend doing it before progress reports or IEP meetings.  Think about including others in this effort, like paraprofessionals, if that's possible, so they can see how the data gets used.  Making data functional for everyone means everyone is more committed to taking it correctly than if it is just paperwork to complete.

4.  Graph the data.

The key to this is making sure that you have an easy way to graph it.  If you use the data sheet I referred to above, you can easily use the Excel spreadsheet from this post.  Don't worry, I did a screencast on that post for how to use it, so don't let it intimidate you.  You put in the number of correct / independent responses and the number of opportunities accumulated across the week for up to 3 activities and it will actually graph itself.  Schedule a time during each week when you graph each student's data for IEP skills that can't be tracked with permanent product or tests  This would typically include socialization, communication and behavioral goals and perhaps some academics, so it wouldn't have to be every skill you are working on.  Then you can review the graph regularly.


5.  Analyze the graph.

We actually don't have a ton of research on data collection in the classroom,   However, what we do have is clear on the following items:

  • Teachers tend to make decisions based more on intuition than on data collected
  • Teachers tend to feel uncomfortable analyzing data and using it to make meaningful decisions
  • Teachers generally do not graph their data
  • Teachers who do graph and review their data regularly have students who make more progress than teachers who don't.
So, here are some considerations when you go to analyze the data you have taken.  
  • Use general graphic conventions with meaningful X and Y axes. For instance, you can't have a Y-axis that goes higher than 100% since progress can't go beyond 100%.  If the maximum number of data points is 10, don't make the top of the graph 25 unless your aim is for the skill to improve that high or you are expecting the possibility of it getting that high.  The graphing template I talked about above will result in a percentage graph that graphs to 100% on the Y axis and will graph throughout the year to track progress.  How you organize the graph can skew how you interpret the data, so be careful with this.
  • Analyze at least 5 data points to look for whether the data is improving or not improving.
  • Draw an aim line from where the student starts on the graph to a point depicting the level of mastery (e.g., 80%) on the due date for mastery on the IEP (e.g., May 15, 2014).  Then you can assess if your student's progress is moving up that line.  If there are at least 3/5 points above the line, you are moving in the right direction.  If there are less than 3, you need to change something.
  • Use a set of decision rules, like these from Jimenez, Mims, & Browder (2012), to make your decisions.  Their research, and research before them, showed that teachers using decision rules consistently can make changes in instruction that improve student performance.  You can find the chart with the rules in the article at the link above (I don't want to paste it here because of copyright issues), but the link will take you to the article.
So, it is my hope that this at least gives you some ideas about collecting and analyzing data.  What do you see as the biggest hurdles in data collection?  Please share and perhaps a reader (or I) have some ideas that might be helpful.

Until next time,
Autism Classroom News


3 comments:

  1. Thanks for this post. My biggest challenge in tracking behavior goals is for the students I have that are in the general education classroom setting for the whole day. Finding effective tools to give the classroom teachers (on top of the already big pile they have) has been a little bit of a struggle in the first several weeks. Trying to get that accurate data is challenging when you don't see the individual student during possible data tracking points. But, these are great ideas. Thanks!

    ReplyDelete
    Replies
    1. Jaimie, that's a great point. If you are tracking just frequency of a specific behavior, maybe the teachers could use a counter or a post-it to tally? I talk about a counter in this post if you aren't sure what I mean. http://www.autismclassroomnews.com/2013/09/3-tools-for-data-collection-getting.html If you are looking at more behavioral or social skills, perhaps you could use a rubric that the teacher completes weekly on the quality of the behavior? And of course you can have the teacher just take a sample instead of trying to track the behavior throughout. Self-monitoring data is also a good one to try if the child is capable--and even if the child isn't always accurate, the research shows that self-monitoring itself decreases problem behavior regardless of accuracy. You have raised some good thoughts for future posts...thanks so much for sharing them!
      Chris

      Delete
  2. I know this was posted last October, but I'm just now reading it! I have been trying to organize and keep data accurately all year, and this whole post really spoke to me! Thank you very much, wish I had taken the time to read and try this earlier in the year! better late then never though, Paula @ educating children with disabilities.blogspot

    ReplyDelete

Related Posts Plugin for WordPress, Blogger...