Observer variation in locomotion scoring

By Steve Mason on

At the 2013 International Conference on Lameness in Bristol UK, a group of European workers reported the results of a trial aimed at estimating observer variation in locomotion scoring (LS). Seven experienced observers were asked to score 58 video recordings of cows walking using a five level LS system (described here) where a cow with score 1 walks normally and a score 5 cow is extremely lame. Among the 58 recordings there were approximately equal numbers of cows representing each of the 5 scores. Each observer scored the same cows on 2 different occasions separated by at least 4 days. Within observer agreement is the percentage of the 58 cows that were given the same LS by an individual observer on both occasions. Between observer agreement is the percentage of cows that were given the same LS by all 7 observers. Results are summarized in the table below.

Within- and Between Observer Agreement in Locomotion Scoring

Locomotion ScoreWithin Observer
Agreement, %
(95% confidence interval)
Between Observer
Agreement, %
(95% confidence interval)
172.5 (64.4 - 80.6)63.9 (60.3 - 67.6)
263.9 (56.9 - 70.9)58.9 (55.9 - 61.6)
360.0 (51.7 - 68.2)53.1 (49.6 - 56.5)
474.5 (67.2 - 81.9)62.1 (58.6 - 65.5)
575.6 (60.2 - 91.9)38.7 (30.5 - 46.9)

source: A Schlageter-Tello, EAM Bokkers, PWG Groot Koerkamp, T Van Hertem, S Viazzi, CEB Romanini, I Halachmi, C Bahr, D Berckmans and C Lokhorst; 17th International Symposium and 9th International Conference on Lameness in Ruminants; Bristol, UK; Aug 11-14, 2013; page 88.

Leave a Reply

Your email address will not be published. Required fields are marked *