4.6 KiB
(U) The SIGINT Philosopher: The Fallacies Behind the Scenes
FROM: (U//FOUO) [REDACTED] Run Date: 03/15/2011
(U) At the end of the day, much of the work done in SID revolves around getting information to people who need to make a decision. Often that decision involves making an estimate as to how probable or plausible a given event is, and what implications such a hypothetical event might have on our national security. It seems straightforward enough. However, at the outset, there are undercurrents flowing through the minds of every analyst, before the fingers even hit the keyboard. The analyst might not even know they're at work, but they are. They're cognitive fallacies.
(U) There's no blame to be handed out -- these fallacies are hard-wired into our brains as a result of thousands of years of evolutionary instinct and decision-making to simply survive. In relative historic terms, we've only very recently shifted our attention from such perfunctory tasks to more complex avenues. So, in the analysis we undertake in our daily duties, here are a few biases to be aware of, or perhaps attempt to change, on an individual and collective level:
The Texas Sharpshooter Fallacy
(U) This logical fallacy is named for a Texan who shot a hole in his barn, drew red and white circles around it, and gathered the neighbors to brag about how great a shot he was. While finding significance and connections between people and events is what intelligence is all about -- the downside is that sometimes we stretch to make the case that relationships or significance are present where they aren't. When we make the case first, the evidence we gather tends only to reinforce it. The reverse should be the rule: go where the evidence leads, don't lead the evidence.
The Gambler's Fallacy
(U) Kenny Rogers knew when to hold 'em. So should we. Since we're hard-wired to look for patterns, we have a tendency to think of numbers and events as being orderly, when they're anything but. Neuroscientists call this phenomenon apophenia. For instance, if you're taking a turn at the roulette wheel, how many hits on black will it take before you think to yourself, "We're due for a red..."? Or perhaps you think because a number on the wheel has come up so often in the past, it's bound to be more absent in the future? Pioneering cognitive scientist Amos Tversky noted "It's natural to be inclined toward patterns when we observe cause and effect. But in nature, events rarely happen with equal frequency." Events are rarely ever "due" to occur, and the frequency of an event in the past is not always an indicator of its future appearance.
The Sunk-Cost Fallacy
(U) How many times have you been watching a terrible movie, only to convince yourself to stick it out to the end and find out what happens, since you've already invested too much time or money to simply walk away? This "gone too far to stop now" mentality is our built-in mechanism to help us allocate and ration resources. However, it can work to our detriment in prioritizing and deciding which projects or efforts are worth further expenditure of resources, regardless of how much has already been "sunk." As has been said before, insanity is doing the same thing over and over and expecting different results.
The Appeal to Probability
(U) Remember Y2K? We fetched our gas masks and stocked up on canned food for the impending doom that would come at the dawn of the millennium. But alas, midnight came and went with nary a global meltdown. That needless commotion might have been attributed to an appello probabilitatem, where an event is thought to be possible, and is hence erroneously determined to be plausible. In other words, it's the false assumption that because something can happen, it likely will happen. While preparation is a noble goal -- bear in mind the actual probabilities.
(U) There are hundreds of other fallacies that I could touch on, but the main point is to engender awareness that our decision-making processes are directly and unconsciously affected by nature's own hard-wiring. Keeping that fact in mind could have positive impacts on the way we choose to execute our mission. Who knows, perhaps there are cognitive fallacies that have developed -- not resulting from many generations of human evolution -- but from decades of SIGINT evolution? It may be worthwhile to consider the unintended effects on our collective processes that such an evolution may have had, and be on guard against cognitive fallacies and logical biases of all stripes, whether they be recent, or as old as humanity itself.
Now -- back to my poker game with Kenny...
(U) Have thoughts on this topic? See a related Tapioca Pebble.