Sunday, July 5, 2009

Days of infamy part 1

A few years ago I got in to an online argument about 9/11 conspiracy theories with someone calling himself RedPill, on the BBC's 606 football forums of all places. He was a hardcore proponent of the Loose Change view in particular. He also linked the events of 9/11 with the assassination of JFK and the Japanese attack on Pearl Harbor - all down to the New World Order of course. He very firmly believed that the events of 7 December 1941 were a precursor and inspiration to those of 11 September 2001. His belief was that both were perpetrated/ provoked by the US government or the mythical New World Order in order to allow the government to declare war. He believed he had all the evidence he needed. He believed he understood the history. He believed he knew how military operations were planned and how intelligence services and intelligence gathering worked.

He was wrong. Very wrong.

He was also not the first or the last person I have run across who believed that Churchill and/or Roosevelt were involved in a conspiracy to allow or provoke the Japanese attack on Pearl Harbor. Just a couple of days ago a co-worker suggested to me there was more going on than a simple surprise attack on December 7, 1941. So, this is the introduction to a series of posts that are (hopefully) going to explode the myth that Pearl Harbor was allowed to happen by the US government as part of some grander conspiracy. Through the lens of failures around Pearl Harbor I hope to then illustrate the problems with assuming the US government knew what was going to happen on 9/11/01, or that a long chain of conspiracy links the two.

The majority of information for these posts is taken from my own personal knowledge and research (World War II has been a long time study area for myself, both as part of my history degree and as a personal hobby) as well as the book Military Intelligence Blunders by Colonel John Hughes-Wilson published by Robinson books, 1999. This book is a must read if you think sophisticated and powerful intelligence agencies can't screw up spectacularly. There is a new edition published in 2004 that includes the events leading up to 9/11 which I haven't got my hands on yet but hope to on or by, say, October 1st.

Problems with intelligence gathering

There were many factors that contributed to the disaster (at least from the US perspective - the Imperial Japanese Navy would hardly call it a disaster) at Pearl Harbor. They can be broken down (albeit roughly) into just a few areas, and these are common features of almost all intelligence gathering and interpretation failures as well as many military disasters (both of which combine for Pearl Harbor):

  1. Intelligence 'noise' and the fog of war.
  2. Competing/rival intelligence agencies and armed services.
  3. No centralised intelligence gathering structure.
  4. Lack of unified command structures.
  5. Personalities - including personal and professional arrogance.
  6. Underestimation of the enemy.
  7. Misinterpretation of available data.
  8. The mistrust inherent in handling secret information.
  9. Availability of information or access to it.
  10. Context of available information.

Combine these things and you have the classic ingredients for an intelligence screw up. Here I'll explain a little about what these things mean, then in later posts we'll look at the specifics of each category as they are reflected in the events surrounding the attack on Pearl Harbor.

Intelligence 'noise' and the fog of war

In intelligence gathering terms 'noise' refers to the fact that important nuggets of information can be lost amongst the vast volumes of data available. When you are gathering intelligence on everything that your enemy is doing or might do regardless of value and significance, you might miss the one thing that they are doing or are going to do that is of vital importance.

This is a particular problem when you have underfunded and understaffed intelligence services, and is also directly related to or influenced by a few of the other items on our list above. If you are collecting volumes of information a day but only have a few people to review it, things will get missed. Bear in mind, the intelligence services of 1941 did not have sophisticated computer programs to work with and analyse data, they did this by hand. If you have competing agencies not even sharing the large volumes of data then the right data might not even get into the right hands, and the hands it does get into might be far from the right ones.

Couple the level of intelligence 'noise' with the fog of war and the situation becomes even worse. Fog of war refers to the fact that you don't always know where your enemy is or what they are doing no matter how hard you try to find out. It refers to the confusion inherent in battle. Intelligence gathering in peace time is hard enough, in war time it is magnitudes of difficulty harder. In a World War, the task is Herculean. We are not talking about satellite flyovers and remote controlled drones in 1941 - this is human intelligence sources on the ground in hostile territory in cultures totally alien and hostile; and intercepted signals intelligence which was heavily encrypted and gathered and cracked with comparatively crude technology in comparison to today.

Competing/rival intelligence agencies and armed services

Intelligence services and the different branches of the military are notoriously territorial. They compete for resources and they compete for influence and prominence. They are fiercely protective of their sources, resources, data and techniques. They frequently compete to out do their rivals. Politicians will favour one or the other and will do what they can to undermine those not favoured. This was true in 1941 as it is now.

Because of this, data flow is restricted. Different agencies have different pieces of the same puzzle, but they never share. The pieces don't always come together. Even worse, when the pieces do, they can be discounted because they come from a rival or competing agency. This was particularly true of the US intelligence gathering community of 1941, as we will eventually see.

Inter service rivalry is not a thing of fiction - it is very real and destructive, and the fact of its existence and effect is often not understood by those who are not familiar with the military and intelligence services. Bear in mind that your average Paranoid Conspiracy Theorist (PCT) thinks that intelligence services are both infallible (they plan these perfect huge conspiracies that go off without a hitch) and flawless (they think that if information is available to them it is only ever understood, disseminated and used correctly).

In the world of the PCT, the conspirators are never incompetent, imperfect or underfunded.

No centralised intelligence gathering structure

With so many competing agencies gathering so much data from so many different sources, perspectives and approaches a central organisation devoted to collating and assessing intelligence information is vital. Where one does not exist, no one group or individual ever has the complete picture. It's like trying to complete a 4,000 piece jigsaw where the pieces are held by 4,000 individuals who never come together, never talk to each other and never share the shape of their piece with any of the others.

The British had one such organisation in the shape of the Joint Intelligence Committee (JIC) founded in 1936. A great proportion of the success of British intelligence operations throughout the Second World War, not discounting undoubted failures obviously, owes much to the existence of the JIC as a coordinating, assessment and guiding body, and it's reports were considered to be very accurate. In comparison, the USA had no such organisation. No one individual or group ever had the complete picture. The United States did not establish anything like the JIC (as far as I can tell) until the Office of the Director of National Intelligence was created by the Intelligence Reform and Terrorism Prevention Act of 2004.

This is one of the central reasons for the intelligence failures behind both the attack on Pearl Harbor and the attacks of 9/11 and can not be underestimated or ignored.

Lack of unified command structures

This is similar to the above point, but refers more directly to the different branches of the military, of which there were three in the USA circa 1941 - Army, Navy, Marines (the US air force was not established until 1946).

Again, the British were a little ahead of the Americans in this respect with the Chiefs of Staff Committee, which during the Second World War was part of the War Cabinet and consisted of the First Sea Lord, Chief of the Imperial General Staff and the Chief of the Air Staff. Whilst not guaranteeing success of action (competence is another matter entirely and Churchill interfered regularly and, it could be said, incompetently with military decisions) it did guarantee coordination and coherent direction. The American Joint Chiefs of Staff did not begin to exist in a form resembling what it is today until July 1942 - eight months after the attack on Pearl Harbor.

Where there is no unified command structure then separate branches of the military tend to act on their own and in their own best interests and in accordance with their proficiencies - there is little to no coordination or cooperation of effort. This, as we will see, is exactly what happened in the years and months before the attack on Pearl Harbor. Both the US Army and the US Navy acted in almost complete isolation from each other for the defence of their bases on O'ahu. They refused to adequately share intelligence with each other. They did not see a connection between their roles and acted in a manner which reflected this.

In order to effectively coordinate the actions of the different branches of the military, unified and joint command of all of them must be established. Without this, actions can be disorganised and often conflicting. Which was exactly the case prior to 7 December 1941.

Personalities - including personal and professional arrogance

Look at any major event in world history and personalities will play a central role. This can be either the personalities of individuals or the collective personalities of nations or groups. In the realms of the military or intelligence gathering personalities can often play a disproportionate part, for good or ill. In the case of the attack on Pearl Harbor, personalities play a generally unfortunate role prior to the attack, and personalities contribute to the tales of individual heroism on the day itself.

Intelligence reports and assessments as well as individual testimony can be discounted because of personal pride, prejudice or incompetence. Professional arrogance leads to discounting and undervaluing the abilities and capabilities of the opponent or enemy. Never underestimate the damaging potential of the Dunning-Kruger effect when applied to national security or military matters - history is filled with generals who vastly over-estimated their abilities and competencies and underrated those of their opposite numbers.

Individuals play a major role in the failures of intelligence and procedure in the US government prior to the attack on Pearl Harbor, and there is ample evidence of this which we will examine. Given the idiots at the top prior to 9/11, where would you place your money - on gross negligence and incompetence or on an ultra sophisticated conspiracy pulled off with spectacular success with precedent all the way back to 7 December 1941?

Underestimation of the enemy

This is related to the previous point but is not exclusive to it. Even without personal and professional arrogance or prejudice individuals and organisations make mistakes. The decision making of your enemy is not predictable. You might not fully understand their capabilities or potential for any number of reasons. You might be just plain old wrong.

Once you underestimate your enemy you start to make plans based on this which will not stand once faced with the enemies true potential. You become complacent in your attitudes - you feel safe when you are not, you downgrade your threat level, you go to play golf when an attack you thought your enemy could not make is already underway, your defences are relaxed or let down altogether.

The US did not think the Japanese capable of attacking Pearl Harbor even when there was information pointing to it as a target, they underestimated the ability of the Imperial Japanese Navy. So, on the morning of 7 December 1941, the US military stationed on Pearl Harbor was woefully unprepared for what was to come.

Misinterpretation of available data

Two people can look at the same data and come to a different conclusion. The scientist looks at a study showing sham acupuncture works as well as 'real' acupuncture and concludes there is probably nothing to acupuncture - the woo concludes it shows there is something to acupuncture.

Furthermore - raw data on its own can point to many different conclusions if the context is not known or not clear. What do images of a carrier force setting sail mean? An exercise? Sea trials? An attack? A show of force to bolster diplomatic actions? Preparations for a dignitaries visit? A training mission? A ceremony? A mutiny? A coup attempt? A ruse intended to distract from something else? A search and rescue mission? A friendly visit to a neighbouring nation?

When you don't believe your enemy capable of reaching your base thousands of miles from the initial sailing point would you conclude it was an attack on your base given all possibilities? What if everything indicated that if an attack were to come it would fall in areas your enemy could reach easily?

It is perfectly feasible to have all the right data in one place and still reach the wrong conclusion. Now spread the data across hundreds of individuals in different and competing organisations who don't communicate effectively, reliably or in a timely fashion and then try reaching the right conclusion.

The mistrust inherent in handling secret information

It's called 'Secret' and 'Top Secret' for a reason. It means you don't want everyone to know about it. That includes people on your own side. That includes the people making many of the decisions. In the case of Pearl Harbor, it has been suggested that information was kept from even President Roosevelt and he was commander in chief of the US armed forces!

It is the nature of the intelligence business to guard and obtain secret information. It is in the nature of the intelligence business to breed distrust. Sharing does not come into it. Unfortunately that also seems to mean, in many cases, guarding secrets from people on the same side who need those secrets to do their jobs properly and competently. Sometimes there are good operational reasons to guard secrets and not share them, other times there is no reason but politics, professional jealousy, personality clashes or plain old incompetence. Sometimes, it is as simple as a mistake. "Oh, I thought I did tell you that."

If you can't understand the culture of secrecy inherent in the intelligence services, you should probably not proclaim "But X organisation knew it so everyone in X's government and bureaucracy must have."

If PCTs have no problem believing that the shadowy New World Order has kept the US involvement in Pearl Harbor and 9/11 secret, why do they have trouble understanding or believing that the CIA and FBI and NSA might not all be sharing what they know with each other all the time?

Availability of information or access to it

Just because the information exists doesn't mean the people who want it can get it. It is important to recognise what many people don't seem to understand - intelligence operatives and services are not omnipotent. If you don't have a human source in place, they can't spy or pass on information. If you haven't broken encryption codes, you can't read your enemies communiques. If you don't have the technology in place, you can't intercept communications.

US Naval Intelligence of 1941 did not have satellite surveillance of Japanese naval yards. They could not track Imperial Japanese Navy (IJN) movements via satellite, drone or high flying surveillance aircraft.

You can't analyse or interpret what you don't have. If your organisation has the information but you don't have access to it, you can't reach a conclusion based on it. Access to intelligence information is very heavily restricted as I pointed out in the previous point.

Context of available information

I touched on this already, but it is worth an entry all of of it's own.

Data on it's own is not enough - without the context there is not a lot you can do with it. If an analyst knows that a fleet has set sail but doesn't know that the controlling nation is preparing for war that will start with naval strikes, they can't reliably reach a conclusion that a naval attack is imminent.

If you are only ever seeing a tenth of the picture, what you see has no context and you don't know what it means. Could you interpret the whole of The Lord of The Rings reliably by reading only page 67 of The Two Towers?

If no one ever sees the whole of the story, can anyone ever set it in context?

Hindsight is 20/20

And here, in a nutshell, is the biggest problem with looking into the past and saying "Well, they must have known."

It is impossible for us looking back into the past now to not want to say this. We want to scream back into history "He's behind you." The problem is that most of us don't realise that in the best traditions of panto the cast of history will shout back "Oh no he isn't." We know that the disparate pieces of information pointed towards an attack because we know the attack happened. The problem is, history's players didn't then. In the context of what happened, everything makes sense and the pieces all fit together. When it is so obvious to us what everything meant, we can't believe it wasn't obvious then. So we make up another story - that they must have known and let it happen or been involved in the first place.

What we forget is that we all have all the information, and we know what each individual piece was pointing at and what that meant as a whole when those involved in the past did not because no one had the complete picture.

The first step in understanding history is to make sure you don't interpret the past through the lens of what you know now. Conspiracy theorists fail at this first step, and go on from there to compound their error.

Next time, we'll start looking at what really happened leading up to that first day of infamy.


  1. First the secret and now conspiracy theories....

    dude, where the F do you work?

  2. REI - tends to attract a lot of tree hugging hippies so I'm told!

  3. "There is a new edition published in 2004 that includes the events leading up to 9/11 which I haven't got my hands on yet but hope to on or by, say, October 1st."

    It's a good job you're not writing a blog about subtlety.

    ^ link to amazon