| Previous: Contents | Up: WHATIF: Software for Evaluating | Next: Installation |
WHATIF implements the methods for evaluating counterfactuals introduced in King & Zeng (2006) and King & Zeng (2007):
Gary King and Langche Zeng. 2006. ``The Dangers of Extreme Counterfactuals,'' Political Analysis 14 (2): 131-159.and
Gary King and Langche Zeng. 2007. ``When Can History Be Our Guide? The Pitfalls of Counterfactual Inference,'' International Studies Quarterly 51 (March): 183-210.The two papers overlap, with the first containing all the proofs and technical material and the second having more pedagogical material and examples.
Inferences about counterfactuals are essential for prediction, answering ``what if'' questions, and estimating causal effects. However, when the counterfactuals posed are too far from the data at hand, conclusions drawn from well-specified statistical analyses become based largely on speculation hidden in convenient modeling assumptions that few would be willing to defend. Unfortunately, standard statistical approaches assume the veracity of the model rather than revealing the degree of model dependence, which makes this problem hard to detect.
WHATIF offers easy-to-apply methods to evaluate counterfactuals that do not require sensitivity testing over specified classes of models. If an analysis fails the tests offered here, then we know that substantive inferences will be sensitive to at least some modeling choices that are not based on empirical evidence, no matter what method of inference one chooses to use. Specifically, WHATIF will indicate whether a given counterfactual is an extrapolation (and therefore risking more model dependence) or a (safer) interpolation. Using an algorithm developed in King & Zeng (2006) to identify whether counterfactual points are within the convex hull of the observed data, this is feasible even for large numbers of explanatory variables. It will also compute either the Gower or Euclidian distances from the counterfactuals to each observed data point. The convex hull test can additionally be used to approximate the common support of the treatment and control groups in causal inference. Numerical and graphic summaries are offered.
WHATIF has been incorporated in MatchIt, and also works easily with Zelig output (Imai, King & Lau, 2006; Ho et al., 2007; Imai, King & Lau, 2008).