APriot

A computer program for Monte-Carlo simulations of cumulated error probabilities in ANOVA 

When conducting a psychological experiment it often is tempting to intermediately inspect data and then to decide whether to add further participants or to end the study. But we all know that intermediately inspecting statistical data is a bad research practice since there is an accumulation of alpha.

Interstingly, in medical research, intermediately inspecting the data of a running experiment is a widely spread practice. This is comprehensible since ethical reasons often forbid a new drug to be tested for a longer time than necessary. The test procedure is called "group-sequential testing" and is based on the idea that alpha can be adjusted so that the cumulated alpha error probability for multiple inspections does not exceed a predefined value, for example, .05.

In psychology, sequential testing is widely unknown. One reason may be that the sequential tests known from medical research focus on statistical tests based on the normal distribution. This is suitable for testing the effectiveness of a new drug; there is one group of participants receiving the new drug and a control group receiving the older drug. Within classical statistics, a t-test would be conducted. With the number of participants being not too small, the t-distribution approximates the normal distribution and thus, a group-sequential test can be applied to correct alpha for multiple inspections of the data.

In psychological research, the analysis of variance (ANOVA) is very common since it allows for testing more complex experimental designs with multiple variables and interactions between those variables. APriot can conduct Monte-Carlo simulations of the ANOVA and simulate the effects of intermediately inspecting data. APriot can simulate all effects of the ANOVA including repeated measures and interactions of any order.

I have tried to make APriot as easy to use as possible. There is no need for the user to enter complex simulation parameters. Instead, the user enters the data of an earlier study and APriot computes all parameters needed for conducting a simulation.

I hope that APriot will help you to detect effects in a more economical way without the shortcoming of an inflated alpha error probability.