8 Inter rater reliability

Group: Me (of course), Elliot P., Greg

I tried for 20 minutes to get this document to post as a tble and if its not working now, its not going to. I could do PDF, but my ratty old laptop is already slow enough.

As for the link: http://groups.google.com/group/alt.support.stop-smoking/browse_thread/thread/46389dc65d77ae2b/e4bbdb87e60c933c?hl=en#e4bbdb87e60c933c
It is a support post on google for quitting smoking. It had an overall 35 posts, but we did the first 20 after the initial one.



Results:
% inter-rater reliability
0.725
frequency
% of msgs
Information
6
0.3
Tangible assistance
0
0
Esteem support
10
0.5
Network support
0
0
Emotional support
10
0.5
Humor
7
0.35



We found that this was more an exercise in experimental design and less an aspect of social psychology. Although design is important to any scientific field, psychology has a special need for it because it is in such a delicate balance to have empirical evidence to support intangible theories (i.e. why Freud may lose a lot of support). The intercoder reliability for our team was relatively high, but could have been a little better. No one can expect perfection, but I personally would have been happier with closer to a .8. We all always strive for a little better, and the quality of our results, though statistically significant, may come into question in any criticism of our findings.



Our group viewed esteem and emotional support as closely tied and had a hard time distinguishing them since we rated the 9/10 of the same messages as having both, or the rest having none. This subjective aspect is supposed to be corrected by using more than one grader, but I personally am still a bit skeptical (possibly stemming from my own initial ambiguity).


We discovered, similar to Braithwaite, that tangible and network support were least offered (or not at all). This strikes me that in such various forms of support that the results seem to hold true. It also reinforces the findinggs of Braithwaite in a small way. I must also admit that I was surprised that our results concurred because I am always a skeptic of results, but our own group found them reliable and validated them.

We all agreed that Wallace's number theory has too many holes in it. None of us agree with it and have seen no empirical evidence to make us do so. We feel this theory is very weak and un thought out. Wallace overextends a small case (Kitty Genovese) and the subsequent theories and mistakenly applies it to CMC. This may be of value in some contexts, but proves drastically wrong here.

As for Walther's Social Distance, Anonymity, Acccess, and Impression Management: We feel that these are somewhat more in line with our findings than Wallace, though we almost completely agree with Braithwaite. We all liked this theory, and looking at individual posts we could find individual instances of each of these. It reminds us of one of the first studies we did with the Big 5 on breadth and intensity. With further research, possibly Braithwaite and Walther could find similar aspects in their theories and form a more cohesive one.

For now, our findings support Braithwaite mainly. With little else to go on, hi theory is most true based on our observations.