Das von Thomas Stöber, Peter Kotzian, Barbara E. Weißenberger und Florian Hoos (HEC Paris) verfasste Papier “To be or not to be in the sample? On the consequences of using manipulation checks in experimental accounting research” wurde zur Präsentation in einer Parallel Session der Konferenz der European Accounting Association (EAA) in Glasgow angenommen. Die jährliche EAA-Konferenz ist die größte und bedeutsamste internationale wissenschaftliche Accounting-Fachkonferenz in Europa.
Abstract:
Manipulation checks are used to improve the quality of experimental evidence by making sure that only those participants who are tested as having successfully received the experimental treatment are included in the analysis. While standard experimental designs do not call for such a check, they are quasi-obligatory in many disciplines and also in accounting research. In our paper, we discuss the consequences of using manipulation checks resulting in the exclusion of participants. Using an analytical model, we show that this may translate into a bias towards more significant differences between the experimental and control groups, thereby jeopardizing results to be mere artifacts (alpha error). We also provide a simulation indicating that such biases can occur even when only few participants have been excluded. We contribute to the literature by providing an in-depth discussion and analysis of the consequences of using manipulation checks for experimental accounting research. Based on our discussion, we develop recommendations as to how manipulation checks can be used to gain further insights into mechanisms that drove the observed results of an experimental study. In particular, we recommend that data obtained from manipulation checks should be used as an explanatory variable, not as a justification to exclude participants from the final sample.