Non-Experimental Data, Hypothesis Testing, and the Likelihood Principle: A Social Science Perspective

We argue that frequentist hypothesis testing – the dominant statistical evaluation paradigm in empirical research – is fundamentally unsuited for analysis of the non-experimental data prevalent in economics and other social sciences. Frequentist tests comprise incompatible repeated sampling frameworks that do not obey the Likelihood Principle (LP). For probabilistic inference, methods that are guided by the LP, that do not rely on repeated sampling, and that focus on model comparison instead of testing (e.g., subjectivist Bayesian methods) are better suited for passively observed social science data and are better able to accommodate the huge model uncertainty and highly approximative nature of structural models in the social sciences. In addition to formal probabilistic inference, informal model evaluation along relevant substantive and practical dimensions should play a leading role. We sketch the ideas of an alternative paradigm containing these elements.