Aiken's V, Lawshe, CVC or binomial test: which one to use to validate your instrument

When your advisor asks you to validate your questionnaire with expert judgment, the question usually comes up: Aiken's V, Lawshe, CVC or the binomial test? All four do the same job, which is showing that your judges agree that the items measure what they should. The main difference is how the judges answer. If you pick the method based on the form you'll use, the decision almost makes itself.

The difference in one line

• Aiken's V: judges rate each item on a scale (for example, 1 to 4). It is the most common one in Latin American theses.
• Hernández-Nieto's CVC: also uses a scale (often 1 to 5) and subtracts the part of the result that could be due to chance.
• Lawshe's content validity ratio (CVR): each judge says whether the item is "essential", "useful but not essential" or "not necessary". Only "essential" counts.
• Binomial test: each judge answers yes or no (the item is clear, pertinent, relevant). It gives a p-value for each item.

Aiken's V: for forms with a rating scale

The formula is V = S / (n × (c − 1)), where S is the sum of the judges’ ratings minus the lowest value on the scale, n is the number of judges and c is the number of values on the scale.

Example: 5 judges rate an item’s clarity from 1 to 4 as 4, 4, 3, 4 and 3. Subtracting 1 from each leaves 3, 3, 2, 3 and 2, which add up to 13. The denominator is 5 × 3 = 15, so V = 13 / 15 = 0.87. The most common cut-off is 0.80, and some universities also ask for the lower bound of the confidence interval to pass it.

It works well with few judges (3 to 10) and lets you calculate a V per criterion (clarity, coherence, relevance), per item and for the whole instrument.

Hernández-Nieto's CVC: a scale with a chance correction

For each item, the ratings are averaged (Mx) and divided by the highest value on the scale: CVCi = Mx / Vmax. Then the chance error is subtracted, Pei = (1 / j)^j, where j is the number of judges. The overall CVC is the average of the corrected item CVCs.

Example: 5 judges rate an item from 1 to 5 as 5, 4, 5, 5 and 4. The mean is 4.6 and 4.6 / 5 = 0.92. With 5 judges, Pei = (1/5)^5 = 0.0003, so the corrected CVC is still 0.92. Hernández-Nieto considers 0.71 and above acceptable, 0.81 and above good, and above 0.90 excellent.

It is very close to Aiken's V. Use it if your university or advisor asks for it by name, or if your form goes from 1 to 5.

Lawshe's ratio: when the question is whether the item is essential

Lawshe (1975) asks each judge to classify the item into three options: essential, useful but not essential, or not necessary. The formula is CVR = (ne − N/2) / (N/2), where ne is the number of judges who marked "essential" and N is the total number of judges. It goes from −1 to 1.

Example: 10 judges, 9 mark "essential". CVR = (9 − 5) / 5 = 0.80. The minimum depends on the number of judges. In the table recalculated by Ayre and Scally (2014), with 10 judges the minimum is 0.80, with 8 it is 0.75, and with 7 judges or fewer all of them must mark "essential".

That's why Lawshe needs more judges than the other methods. In Latin America the Tristán-López (2008) modification is also used, which calculates CVR' = ne / N and accepts the item from 0.58. If you use Lawshe, cite which version you applied.

Binomial test: for yes or no forms

If your form asks "is the item clear? yes / no", the binomial test calculates the probability of getting that many "yes" answers if the judges answered at random. If the p-value is below 0.05, the agreement is significant and the item is considered valid.

The key detail is the number of judges. With 5 judges who all say yes, p = 0.5^5 = 0.031, and the item passes. With 4 out of 5, p = 0.19, and it fails. With 4 judges, not even full agreement is significant (p = 0.0625). In practice you need at least 5 judges, and with 5 they all have to agree.

A fifth method: the content validity index (CVI)

Nursing and health research also uses the CVI by Lynn (1986) and Polit and Beck (2006) a lot. Judges rate relevance from 1 to 4, and each item’s CVI is the proportion of judges who gave it a 3 or 4. With 3 to 5 judges an item needs 1.00, and with 6 or more, 0.78. For the instrument you report the mean across items (S-CVI/Ave), which should reach 0.90.

How to choose

1. Look at your validation form. If judges rate on a scale, use Aiken's V (or CVC if it's requested). If they answer yes or no, use the binomial test. If they mark essential, useful or not necessary, use Lawshe.
2. Count your judges. With 3 or 4 judges, Aiken's V is the safest option. The binomial test needs at least 5, and Lawshe works best with 8 or more.
3. Check your university’s guidelines and approved theses from your department. If they all use the same method, your committee will expect it.
4. Pick one main method. Some theses report two (for example, Aiken's V and the binomial test), but it isn't required and it doesn't make a badly worded item valid.

What none of them replaces

A high number doesn't save an item the judges flagged. Always read the comments, fix the wording and, if your university requires it, run a second round with the revised items. After content validity comes reliability, which you measure with a pilot test and Cronbach's alpha.

In SurvIQ's Thesis Mode, judges rate through a link without an account, and the Aiken's V matrix builds itself, with a form per judge for your appendices. SurvIQ doesn't write the thesis: it calculates, lays out the tables and shows which items to review.