Simultaneous, Multiple Proportion Comparisons Using Marascuilo Procedure

Often, there is a need to compare multiple proportions among samples, wherein proportions can be about product's performance or any other characteristic which can be classified among identifiable categories. Examples of binary classifications can be pass/fail or yes/no.

Just the way equality of central tendencies or variance around them cannot be assumed to be equal, equality of proportions also cannot be assumed to be equal.

Chi.SQ. test of equal proportions can be a good starting point.

For this, observations from test samples can be laid out as below, where test results compared against expected outcomes. (In this case, the expected outcome represents equality of proportion among possible test outcome categories such as Pass/Fail).

Chi Sq Metric= SUM{(Observed- Expected)^2/Expected}

This metric is thus calculated and compared against critical threshold from Chi.SQ. table, which documents Chi.SQ. distribution values based on degrees of freedom. 

Degrees of freedom  = (number of proportions being compared for equality-1)

However, validating the null hypothesis of equality is only the first step.

Disproving the null hypothesis merely allows us to conclude that proportions are not equal. It may also be necessary to find which specific proportions stand out in their differences, compared to rest. 

For this purpose, combinations of differences among proportion pairs can be laid out . For instance, to compare among 10 proportions, you will need 45 combinations. For 5 proportions, you will need 10 combinations and so on.

Then, for each such difference pair, critical value for Marascuilo procedure can be computed using the following formula.

criticalVal(i,j)=[sqrt(pi*(1-pi)/ni)+(pj*(1-pj)/nj)]*sqrt(chi.sq.value@d.f.)

Where pi*(1-pi)/ni and pj*(1-pj)/nj are variances for proportions i and j respectively, sum of which is obtained to get total variance and standard deviation, which then is multiplied by Chi.SQ. distribution value chosen to get a desired confidence level as to whether the given proportion (i,j) is indeed significantly different.

Thus, using the Marascuilo procedure, we can pinpoint which specific sample proportions differ significantly.

In summary, we first looked at how to apply Chi.Sq. test to infer whether one or more proportions are equal or not. Here, we need to account for all types of evaluations or scorings ( such as {false, true} or {accept, reject}) to compute the metric score.

Moving beyond proving the existence of significant inequality, we then utilized a statistical procedure to identify the specifics as to among whom such inequality exists.