The Fallacy of the Texas Sharpshooter

Posted by M ws On Thursday, March 15, 2012 0 comments
Have you ever heard of the Texas Sharpshooter fallacy?

This link says:

In logic and rhetoric, a fallacy is usually an improper argumentation in reasoning resulting in a misconception or presumption. Literally, "an error in reasoning that renders an argument logically invalid". By accident or design, fallacies may exploit emotional triggers in the listener or participant (appeal to emotion), or take advantage of social relationships between people (e.g. argument from authority). Fallacious arguments are often structured using rhetorical patterns that obscure any logical argument.



The Texas sharpshooter fallacy is a logical fallacy in which pieces of information that have no relationship to one another are called out for their similarities, and that similarity is used for claiming the existence of a pattern. This fallacy is the philosophical/rhetorical application of the multiple comparisons problem in statistics, and apophenia in cognitive psychology. It is related to the clustering illusion, which refers to the tendency in human cognition to interpret patterns in randomness where none actually exist.

The name comes from a joke about a Texan who fires some shots at the side of a barn, then paints a target centered on the biggest cluster of hits and claims to be a sharpshooter.

The Texas sharpshooter fallacy often arises when a person has a large amount of data at their disposal, but only focuses on a small subset of that data. Random chance may give all the elements in that subset some kind of common property (or pair of common properties, when arguing for correlation). If the person fails to account for the likelihood of finding some subset in the large data with some common property strictly by chance alone, that person is likely committing a Texas Sharpshooter fallacy.

To illustrate, suppose that a researcher flips a coin ten times to determine whether or not the coin is balanced. The probability of that researcher seeing the coin come up heads all ten flips is 1 in 1,024 — if it were to happen it would be reasonable to assume that the coin is indeed biased. However, suppose instead that the researcher has 5,000 coins, and performs the same ten-flip test on each of them. Though each coin should only have a ten-head probability of 1 in 1,024, it is overwhelmingly likely (with a probability of 99.24%) that, if this ten-flip test is performed 5,000 times, at least one such time will have a coin coming up heads for all ten flips by pure chance alone. Therefore, the alleged finding of an imbalanced coin is a nearly inevitable outcome of the testing methodology itself. If the researcher indeed finds one such ten-head occurrence in 5,000 trials and claims that the coin involved in the ten-head occurrence is imbalanced, the researcher would be committing a Texas sharpshooter fallacy. (Note that, if the coin is indeed balanced, then subsequent testing will most likely not yield an unusual result).

The fallacy is characterized by a lack of specific hypothesis prior to the gathering of data, or the formulation of a hypothesis only after data has already been gathered and examined. Thus, it typically does not apply if one had an ex ante, or prior, expectation of the particular relationship in question before examining the data. For example one might, prior to examining the information, have in mind a specific physical mechanism implying the particular relationship. One could then use the information to give support or cast doubt on the presence of that mechanism. Alternatively, if additional information can be generated using the same process as the original information, one can use the original information to construct a hypothesis, and then test the hypothesis on the new data. See hypothesis testing. What one cannot do is use the same information to construct and test the same hypothesis (see hypotheses suggested by the data) — to do so would be to commit the Texas sharpshooter fallacy.

Examples

A Swedish study in 1992 tried to determine whether or not power lines caused some kind of poor health effects. The researchers surveyed everyone living within 300 meters of high-voltage power lines over a 25-year period and looked for statistically significant increases in rates of over 800 ailments. The study found that the incidence of childhood leukemia was four times higher among those that lived closest to the power lines, and it spurred calls to action by the Swedish government. The problem with the conclusion, however, was that the number of potential ailments, i.e. over 800, was so large that it created a high probability that at least one ailment would exhibit statistically significant difference just by chance alone. Subsequent studies failed to show any links between power lines and childhood leukemia, neither in causation nor even in correlation.[2]
Attempts to find cryptograms in the works of William Shakespeare, which tended to report results only for those passages of Shakespeare for which the proposed decoding algorithm produced an intelligible result. This could be explained as an example of the fallacy because passages which do not match the algorithm have not been accounted for.

READ MORE HERE.

I always believe that everyone of us must test and verify whatever we read, hear of watch lest we are deceived. Sadly, many students today are not encouraged to think and to question when they are in school. May we always develop a desire to find truth for ourselves. Have a nice day!

0 comments to The Fallacy of the Texas Sharpshooter

Related Posts with Thumbnails
.