Yes. In SPSS it is Analyze > Nonparametric Tests > Legacy Dialogs > K Related Samples > Kendall’s W. There each row is a judge and each column an item: choose "Rows = judges" if you paste your table that way.

Kendall’s W, Aiken’s V or CVC?

Aiken’s V and the CVC tell you whether each item is valid. Kendall’s W tells you whether the judges agree with each other across the instrument. Many theses report one of the first two plus W.

How many judges do I need?

At least 2, though theses usually have 3 to 7.

Does it work if judges answered yes or no?

For yes/no answers the binomial test is a better fit, and it is also in these tools.

All tools

Kendall’s coefficient of concordance (W)

Kendall’s W measures how much several judges agree when rating or ranking the same items. It goes from 0 (no agreement) to 1 (full agreement). In theses it is widely used for expert judgment, alongside Aiken’s V or the CVC. Paste the ratings from Excel: one row per item and one column per judge. If your table has judges in rows, change it in the selector.

Paste the ratings (rows = items, columns = judges)

How your table is laid out

The calculation runs in your browser. We don’t store what you type.

How it’s calculated

Each judge’s ratings are turned into ranks: 1 for the item with the lowest rating. Ties get the average rank.

The ranks of each item are added up (Rᵢ) and S is the sum of squared differences between each Rᵢ and their mean.

W = 12S / (k²(n³ − n) − kΣT), where k is the number of judges, n the number of items and T corrects each judge’s ties.

Significance comes from χ² = k(n − 1)W with n − 1 degrees of freedom, as in SPSS.

How to read the result

p < 0.05: the judges agree more than you would expect by chance.
A common guide (Landis and Koch): up to 0.20 slight, 0.21 to 0.40 fair, 0.41 to 0.60 moderate, 0.61 to 0.80 substantial and above 0.80 almost perfect.
With fewer than 7 items the chi-square is only approximate; report it with care.

Frequently asked questions

Does it match SPSS?: Yes. In SPSS it is Analyze > Nonparametric Tests > Legacy Dialogs > K Related Samples > Kendall’s W. There each row is a judge and each column an item: choose "Rows = judges" if you paste your table that way.
Kendall’s W, Aiken’s V or CVC?: Aiken’s V and the CVC tell you whether each item is valid. Kendall’s W tells you whether the judges agree with each other across the instrument. Many theses report one of the first two plus W.
How many judges do I need?: At least 2, though theses usually have 3 to 7.
Does it work if judges answered yes or no?: For yes/no answers the binomial test is a better fit, and it is also in these tools.

Know what you want to ask?

Build the survey in SurvIQ, share it by link, WhatsApp or QR code, and watch results come in live. The Free plan is free forever.

Create a free survey

More tools

Binomial test for expert judgmentValidate each item with judges who answer yes or no: agreement and exact p-value.Cohen’s and Fleiss’ kappa calculatorMeasures agreement between raters or judges, beyond what chance would produce.Normality test (Shapiro-Wilk and Kolmogorov-Smirnov)Check whether your data is normally distributed before choosing a statistical test.Student’s t-test calculatorCompares the means of two groups or a before and after, with Levene’s test and Cohen’s d.Mann-Whitney U test calculatorCompares two independent groups when the data aren’t normal or are ordinal.Wilcoxon signed-rank test calculatorCompares a before and after of the same people when the data aren’t normal.