Wednesday, September 21st, 2016
In exploratory analyses or experiments, very many variables may be correlated against an effect just to see which, if any, might possible be related to the effect of interest. It is a sort of fishing expedition. Such “data mining” can lead to many false positives.
Exploratory analyses can suggest some interesting phenomenon, e.g., that two variables are correlated; so, sometimes they are called “hypothesis-generating experiments”. But then you need to do a hypothesis-driven experiment to really convince everybody that the two variables are indeed correlated, and that they weren’t discovered in a fishing expedition. In a hypothesis-driven experiment, you declare at the outset that your hypothesis is that X and Y are correlated, and you design the experiment specifically to test that hypothesis. It’s a surgical strike, rather than a fishing expedition.
The approach of exploring and finding the “best” answer is a dangerous approach. First, it can be time and resource intensive and time and resources are scarce commodities. Second, while you’re looking for the “best” answer, the world is changing and new variables enter the equation so by the time you get to “best” it’s not relevant anymore and you have to find a new answer. This gets you in an endless loop of doing analysis instead of executing ideas. Consider a hypothesis-driven approach. Take an educated guess at the answer. Prove or disprove the hypothesis quickly. If you’re right, execute. If wrong, move to the next idea and repeat the process. You’ll get more done in shorter periods of time.
Summing-up: When solving problems, are you exploratory or hypothesis-driven in your search for answers?