Estimates vs. head to head comparisons

Summary: when choosing between two options, it’s not always optimal to estimate the value of each option and then pick the better one.

Suppose I am choosing between two interventions, X and Y. One way to make my decision is to predict what will happen if I do X and predict what will happen if I do Y, and then pick the option which leads to the outcome that I prefer.

My predictions may be both vague and error-prone, and my value judgments might be very hard or nearly arbitrary. But it seems like I ultimately must make some predictions, and must decide how valuable the different outcomes are. So if I have to evaluate N options, I could do it by evaluating the goodness of each option, and then simply picking the option with the highest value. Right?

Read the rest of this entry »