# Kappa Calculation

There are three steps to calculate a Kappa coefficient:

Step one, rater sheets should be filled out for each rater.  In the example rater sheet below, there are three excerpts and four themes.  Enter 1 in the corresponding cell if the rater thought the theme was present in that excerpt; enter 0 if the rater thought the theme was absent.

Rater Sheet: RATER 1

 Excerpt # Theme(1=Present    0=Absent) Theme W Theme X Theme Y Theme Z Excerpt 1 1 1 1 1 Excerpt 2 0 1 0 1 Excerpt 3 0 1 1 1 Excerpt 4 1 0 0 1 Excerpt 5 1 1 1 1

Rater Sheet: RATER 2

 Excerpt # Theme(1=Present    0=Absent) Theme W Theme X Theme Y Theme Z Excerpt 1 1 1 0 0 Excerpt 2 0 0 1 1 Excerpt 3 1 1 1 1 Excerpt 4 1 0 0 1 Excerpt 5 1 1 1 1

Step two, you will need to fill out a matrix for each theme.  For theme W for example, Rater 1 and Rater 2 agreed 3 times that theme W was present in the excerpts, so enter 3 in box A of the matrix.  One time Rater 1 thought theme W was absent and Rater 2 thought theme W was present, so enter 1 in box B.  Rater 1 never thought a theme was present that Rater 2 thought was absent, so enter 0 in box C.  One time Rater 1 and Rater 2 agreed that theme W was absent, so enter 1 in box D.

Key

 Rater 1 Present Absent Rater 2 Present A B Absent C D

Theme W

 Rater 1 Present Absent Rater 2 Present 3 1 Absent 0 1

Theme X

 Rater 1 Present Absent Rater 2 Present 3 0 Absent 1 1

Theme Y

 Rater 1 Present Absent Rater 2 Present 2 1 Absent 1 1

Theme Z

 Rater 1 Present Absent Rater 2 Present 4 0 Absent 1 0

Enter your theme matrix values in the calculator to determine your kappa values.

 Theme A B C D Observed Expected Kappa W 3 1 0 1 0.8 0.56 0.00 X 3 0 1 1 0.8 0.56 0.00 Y 2 1 1 1 0.6 0.52 0.00 Z 4 0 1 0 0.8 0.8 0.00