Skip to content

Scott's pi coefficient

Jump to bottom
Jeffrey M Girard edited this page Feb 20, 2016 · 36 revisions

Index Information

Coming soon...

MATLAB Functions

  • FAST_PI %Calculates pi using simplified formulas
  • FULL_PI %Calculates pi using generalized formulas

Simplified Formulas

pi

pi

pi

pi

pi

Use these formulas with two raters and two categories (dichotomous).

a is the number of items both raters assigned to the first category

d is the number of items both raters assigned to the second category

n is the total number of items

f_1 is the number of items rater A assigned to category 1

f_2 is the number of items rater A assigned to category 2

g_1 is the number of items rater B assigned to category 1

g_2 is the number of items rater B assigned to category 2

Contingency Table

Generalized Formulas

References

  1. Byrt, T., Bishop, J., & Carlin, J. B. (1993). Bias, prevalence and kappa. Journal of Clinical Epidemiology, 46, 423–429.
  2. Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, 76(5), 378–382.
  3. Gwet, K. L. (2014). Handbook of inter-rater reliability: The definitive guide to measuring the extent of agreement among raters (4th ed.). Gaithersburg, MD: Advanced Analytics.
  4. Scott, W. A. (1955). Reliability of content analysis: The case of nominal scaling. Public Opinion Quarterly, 19(3), 321–325.
  5. Siegel, S., & Castellan, N. J. (1988). Nonparametric statistics for the behavioural sciences. New York, NY: McGraw-Hill.
Clone this wiki locally