One-factor-at-a-time method

Advantages

OFAT is favored by non-experts, especially in situations where the data is cheap and abundant.

There exist cases where the mental effort required to conduct a complex multi-factor analysis exceeds the effort required to acquire extra data, in which case OFAT might make sense. Furthermore, some researchers have shown that OFAT can be more effective than fractional factorials under certain conditions (number of runs is limited, primary goal is to attain improvements in the system, and experimental error is not large compared to factor effects, which must be additive and independent of each other). "Friedman, M., and Savage, L. J. (1947), “Planning Experiments Seeking Maxima,” in Techniques of Statistical Analysis, eds. C. Eisenhart, M. W. Hastay, and W. A. Wallis, New York: McGraw-Hill, pp. 365-372.

Disadvantages

In contrast, in situations where data is precious and must be analyzed with care, it is almost always better to change multiple factors at once. A middle-school-level example illustrating this point is the family of balance puzzles, which includes the Twelve Coins puzzle. At the undergraduate level, one could compare Bevington's GRIDLS versus GRADLS. The latter is far from optimal, but the former, which changes only one variable at a time, is worse. See also the factorial experimental design methods pioneered by Sir Ronald A. Fisher. Reasons for disfavoring OFAT include:

1. OFAT requires more runs for the same precision in effect estimation
2. OFAT cannot estimate interactions
3. OFAT can miss optimal settings of factors.

Designed experiments remain nearly always preferred to OFAT with many types and methods available, in addition to fractional factorials which, though usually requiring more runs than OFAT, do address the three concerns above. One modern design over which OFAT has no advantage in number of runs is the Plackett-Burman which, by having all factors vary simultaneously (an important quality in experimental designs), gives generally greater precision in effect estimation.

References

References

1. (2015). "What do we mean by sensitivity analysis? The need for comprehensive characterization of "global" sensitivity in Earth and Environmental systems models". Water Resources Research.
2. Friedman, M., and Savage, L. J. (1947), “Planning Experiments Seeking Maxima,” in Techniques of Statistical Analysis, eds. C. Eisenhart, M. W. Hastay, and W. A. Wallis, New York: McGraw-Hill, pp. 365-372.
3. Daniel, C. (1973),“One-at-a-Time Plans,” Journal of the American Statistical Association 68, 353-360
4. Bevington and Robinson, ''Data Reduction and Error Analysis for the Physical Sciences'', 2nd Ed. McGraw–Hill (1992)
5. Czitrom, Veronica. (1999). "One-Factor-at-a-Time Versus Designed Experiments". American Statistician.