Conventional wisdom suggests that data derived from relatively small studies which have skewed distributions be transformed (log, square root, etc.) before analysis. Alternatively, one might consider some non-parametric procedure involving a rank transformation to perform a permutation type procedure. To demonstrate the effect of specific alternatives on both the type I error and power of detecting differences between treatment groups, simulation studies were conducted using specific gamma distributions G(alphaphi) to generate the data. A two independent sample study design was assumed. The values of alpha assumed were 1, 2, 4, 8, 16, representing a wide class of distributional profiles from almost symmetric (alpha = 16) to highly skewed (alpha = 1 or 2). The mean disease level mu = alphaphi = 3.0, and test group always had means corresponding to 0%, 10%, 20%, 30%, 40% and 50% reductions (RT) in average disease level. Five thousand pairs of random samples of equal size were selected from these distributions. Group sizes considered were n = 10, 15, 25 and 40. Six statistical test procedures were compared: the UMP test; t-tests based on the original, square root and logarithmic scales, respectively; the Wilcoxon ranksum test; and the randomization test (based on mean differences). Necessarily the UMP test (not a realistic option in practice) produced the greatest power for all combinations of n, alpha and RT. The power losses associated with the randomization test were rather small, ranging from 2% to 4% (absolute differences) over the range of alternative investigated; the power loss associated with the t-test on the original scale as well as the square root scale were slightly larger (3% to 6%) for n = 10 and 15, but these t-tests performed about equally to the randomization test for group sizes of 25 or more. The power loss associated with the t-test on the log scale was surprisingly substantial, ranging from 5% to 10% relative to the t-test on original scale. The wilcoxon test produced similar results of that of the t-test on log scale.
