Spying in WW2: how wartime espionage was just as dramatic as fiction

The sounds of shouting in German and of gunfire spurred them on and all managed to escape, skilfully vanishing into the shadows. Despite the Germans flooding the area with thousands of extra soldiers in the ensuing days, the Norwegian saboteurs were able to escape. Their mission had been a success: the heavy water plant had been seriously damaged, though it would not be the last the Allies would hear of German atomic efforts.

• Was the US justified in dropping atomic bombs on Hiroshima and Nagasaki during the Second World War?

A war of intelligence

The Second World War, unlike any other conflict before it, can be classed as an intelligence war. In every theatre, in every type of operation, and for each major country involved, intelligence became a central facet of war planning. From the breaking of codes, the recruiting of secret agents and the production of detailed assessments, through to the escape of prisoners of war, sabotage and destructive covert missions, the conduct of the war would have been dramatically different had intelligence not played such a vital role.

The Second World War, unlike any other conflict before it, can be classed as an intelligence war

Each of the major powers at the outbreak of war – bar one – had significant intelligence structures in place. Each had a history of espionage and a tradition of cunning in the secret world. Britain’s intelligence history stretched back to 1909, albeit with earlier roots; the Soviet Union had a hugely sophisticated internal and external system; the French had an established process; while the Germans, Italians and Japanese had all spent years focusing on producing an efficient intelligence machine. The exception was the United States, which had little in the way of an intelligence tradition and certainly had no effective intelligence community. By the end of the war, convinced of the value of intelligence, the US would proceed with the creation of the most costly and effective intelligence structure the world has ever seen.

In 1939, there can be little doubt that each of the major powers saw the value of intelligence in the war effort, yet none could have anticipated just how central it would become. One of the first intelligence triumphs occurred before the first shot was fired. It was secured by the Poles, who managed to supply British intelligence with a means of breaking the coding used by the Germans. The Enigma machine, and the Ultra intelligence derived from it, would be the greatest coup of the war. At Bletchley Park in Buckinghamshire, British intelligence was able to develop a means of intercepting, deciphering, translating and assessing the contents of messages within hours of their transmission. The frequency with which the Enigma machine was used meant that the Germans relied upon it as a fast, secure and important means of communication. That its codes were broken therefore gave Allied military commanders an undoubtable advantage but, like any source of intelligence, it was not perfect.

• Alan Turing: The man, the enigma

Ultra intelligence was a secret almost unsurpassed in the war: its existence was very tightly controlled among those with a ‘need to know’. In practice, this ensured a number of difficulties: military commanders fighting in Europe, the Atlantic, Africa and elsewhere could not be told how the intelligence had been obtained, so its provenance was usually concealed. Furthermore, the top levels of the German military and Nazi hierarchy were more reluctant to use it, so although tactical war-related plans could be revealed, little was known about the strategic aspects of what the Germans were up to. There were other difficulties too: having such a fantastic intelligence source was great, but often Allied commanders became over-reliant on it – and it still needed a good military brain to work out how to react. In short, it still required other means of intelligence to complement it.

Much like the military, British intelligence had to fight on all fronts during the war. Back at home, the security service MI5 was responsible for locating and identifying all German agents. Operating out of Wormwood Scrubs, a prewar prison in west London, MI5 officers were able to locate all German spies in the UK. The fact that Ultra could reveal much about them – and that there were around only 120 of them – meant that the task was considerably easier than first feared. Yet the real genius in this was in its application. The German spies were given a simple choice: work for British intelligence or face execution. Unsurprisingly, the majority opted for the first option, but not all did. Josef Jakobs chose not to become a British spy. Instead he was put on trial for committing an “act of treachery” in Huntingdonshire when he “descended by parachute with [an] intent to help the enemy”. Although he pleaded not guilty, the charge was upheld and he was executed by military firing squad, becoming the last person to ever be executed at the Tower of London. Those who did become British spies were used by the mysterious sounding ‘XX Committee’, known as Double Cross, to deceive the Germans. At a tactical level, this involved feeding back inaccurate reports on a variety of issues; at a strategic level, it was used to great effect to confuse the Germans about the location of the D-Day landings and the performance of the V-weapon campaign against London.

From its headquarters in central London, the Secret Intelligence Service (SIS, or MI6 as it is frequently known) also operated a number of operations abroad. It ran a series of successful intelligence networks and individual agents, including a collection of train spotters in Belgium (codenamed ‘Clarence’) and the network masterminded by dashing officers like ‘Biffy’ Dunderdale in occupied Europe. Further aiding the human intelligence operations was the fact that the work at Bletchley Park, undertaken by the Government Code and Cipher School, was part of SIS itself.

On the continent, the most illustrative example of intelligence work in action was the Special Operations Executive, or SOE. Famously created by Winston Churchill to “set Europe ablaze”, SOE had been hived off from SIS at the start of the war and its primary role was sabotage, reconnaissance and planned destruction. Although based in London, its main task was working with local resistance groups to foment opposition to Nazi and Fascist rule, while also hindering enemy activities. SOE worked closely with SIS and, though relations were tense in some parts of Europe, the abilities of both organisations and the expertise of their personnel created an effective force. In Denmark alone, more than 1,000 operations were conducted, ranging from detonating bombs underneath bridges to hinder German transport efforts, to rescuing Jews from certain death.

• The Special Operations Executive: Churchill's secret army

In addition to these organisations, a number of other elements within the British war effort focused on intelligence. The Joint Intelligence Committee was the pre-eminent assessment body, producing a range of papers on political and military subjects. Its assessments would be crucial to the actual timing for the D-Day landings. The Political Warfare Executive focused on propaganda efforts, while smaller organisations concentrated on specific aspects: for instance, MI9 worked on helping prisoners of war escape, while MI10 had a military-scientific focus. The experience and knowledge employed by British intelligence was used to great effect, not only in supporting the war effort, but also in educating other countries in the finer art of intelligence.

One country that undoubtedly benefited was the United States. Until the devastating attack on the naval base at Pearl Harbor in December 1941, which brought the US into the war, there was little in the way of effective intelligence. Specific parts of the US military had intelligence staffs, but there was neither a centralized function nor a specific organization for espionage. The bolt out of the blue that marked the Japanese attack not only signalled the start of the wholesale US military effort, but also its introduction to intelligence.

The result was twofold: an increased effort in the decipherment of the Japanese codes, and the creation of the Office of Strategic Services (OSS). The US had been reading Japanese diplomatic messages since the late 1930s but, with the outbreak of war, this took on an increased purpose, not least because none of the intercepted messages had hinted at the Pearl Harbor attack. This programme, codenamed Magic, was on a par with British successes against the German Enigma. The OSS had a broader remit than any of its British equivalents, encompassing espionage, sabotage and propaganda. Like SIS, it operated in Europe and Asia, but it employed significantly more personnel.

The other major powers also saw an expansion of their intelligence efforts as the war progressed. The Soviet Union was able to employ its vast machinery to great effect, utilising human and technical intelligence sources. Ironically, perhaps, it probably spent as much time spying on its wartime allies as it did the Axis powers. Germany’s intelligence structures were efficient, but were characterized by internal competition, a typical sign of Hitler’s rule. Meanwhile the French – under occupied rule for much of the war – attempted to employ a limited organisation from London.

In every theatre and conflict of the war, intelligence played a role. Sometimes it was significant; at other times, it was readily available but could make little difference to the military outcome. In other instances, it was conspicuously absent. Taken in isolation, there are clear examples of where intelligence did and did not play a role. Taken together, it is far harder to offer a broad conclusion on the importance of espionage to the conflict as a whole. Military historians are often quick to emphasise that one factor helped shorten the war by a certain number of years, but these are attention-grabbing headlines that often bear little resemblance to reality.

Intelligence in the postwar world

Perhaps the clearest sign that the intelligence services had played a truly important part in the war is the fact that the majority of organisations continued into the postwar world. The value of intelligence had certainly been recognised and powerful arguments were made to ensure its preservation. In the UK, in January 1945, the chairman of the influential Joint Intelligence Committee produced a blueprint for the postwar intelligence world. He persuasively argued that, as economic austerity set in, military budgets would be slashed and, accordingly, the value and importance of intelligence would grow. His arguments were met receptively and the postwar British intelligence community became central to military and diplomatic planning.

• The evolution of British code-breaking

Other victorious powers took similar views. French intelligence was effectively recreated, while in the Soviet Union state security expanded out of all proportion. In the United States, the reaction was far slower to take hold. Initial postwar arguments about the US’s place in the world were possibly to blame but, by 1947, the future course had been set on its path with the creation of the Central Intelligence Agency.

• MK Ultra: the secrets of the CIA's mind-control programme

Modern intelligence structures certainly have their roots in the Second World War and that can only be testament to their value in that conflict. Once the war in Europe was over, Winston Churchill – by now deposed as prime minister – wrote to the chief of the SIS, recording how “the Services rendered, the incredible difficulties surmounted, and the advantages gained in the whole course and conduct of the war, cannot be overestimated … Will you, within the secret circle, convey to all possible my compliments and gratitude.” Intelligence was, Churchill concluded, “a rock of safety”.

This article first appeared in BBC History Magazine's 'Spies & Espionage' bookazine in 2015

Michael Goodman is Professor of Intelligence and International Affairs, Head of the Department of War Studies and Dean of Research Impact, King's College London