The Secret World: A History of Intelligence

By Christopher Andrew

“A comprehensive exploration of spying in its myriad forms from the Bible to the present day . . . Easy to dip into, and surprisingly funny.” —Ben Macintyre in The New York Times Book Review

The history of espionage is far older than any of today’s intelligence agencies, yet largely forgotten. The codebreakers at Bletchley Park, the most successful WWII intelligence agency, were completely unaware that their predecessors had broken the codes of Napoleon during the Napoleonic wars and those of Spain before the Spanish Armada.

Those who do not understand past mistakes are likely to repeat them. Intelligence is a prime example. At the outbreak of WWI, the grasp of intelligence shown by US President Woodrow Wilson and British Prime Minister Herbert Asquith was not in the same class as that of George Washington during the Revolutionary War and eighteenth-century British statesmen. In the first global history of espionage ever written, distinguished historian and New York Times–bestselling author Christopher Andrew recovers much of the lost intelligence history of the past three millennia—and shows us its continuing relevance.

“Accurate, comprehensive, digestible and startling . . . a stellar achievement.” —Edward Lucas, The Times

“For anyone with a taste for wide-ranging and shrewdly gossipy history—or, for that matter, for anyone with a taste for spy stories—Andrew’s is one of the most entertaining books of the past few years.” —Adam Gopnik, The New Yorker

“Remarkable for its scope and delightful for its unpredictable comparisons . . . there are important lessons for spymasters everywhere in this breathtaking and brilliant book.” —Richard J. Aldrich, Times Literary Supplement

“Fans of Fleming and Furst will delight in this skillfully related true-fact side of the story.” —Kirkus Reviews

“A crowning triumph of one of the most adventurous scholars of the security world.” —Financial Times

Includes illustrations

• Language: English
• Publisher: Yale University Press
• Release date: Sep 4, 2018
• ISBN: 9780300240528

Book preview

The Secret World

CHRISTOPHER ANDREW

The Secret World

A History of Intelligence

The Henry L. Stimson Lectures at the Whitney and Betty MacMillan Center for International and Area Studies at Yale. First published 2018 in the United States by Yale University Press and in the United Kingdom by Penguin Books Ltd., London.

Copyright © 2018 by Christopher Andrew.

All rights reserved.

This book may not be reproduced, in whole or in part, including illustrations, in any form (beyond that copying permitted by Sections 107 and 108 of the U.S. Copyright Law and except by reviewers for the public press), without written permission from the publishers.

Yale University Press books may be purchased in quantity for educational, business, or promotional use. For information, please e-mail [e-mail] (U.S. office) or [e-mail] (U.K. office).

Set in 10.2/13.5 pt Sabon LT Std

Typeset by Jouve (UK), Milton Keynes

Printed in the United States of America.

Library of Congress Control Number: 2018947154

ISBN 978-0-300-23844-0 (hardcover: alk. paper)

This paper meets the requirements of ANSI/NISO Z39.48-1992

(Permanence of Paper).

10 9 8 7 6 5 4 3 2 1

To Ben, Emily, Joe, Katy, Louis, Sam and Tommy

Contents

List of Illustrations

Introduction: The Lost History of Global Intelligence

1 In the Beginning: Spies of the Bible and Ancient Egypt from Moses to the Last Supper

2 Intelligence Operations in Ancient Greece: Myth and Reality from Odysseus to Alexander the Great

3 Intelligence and Divination in the Roman Republic

4 The Art of War and the Arthashastra: How China and India Took an Early Lead over Greece and Rome

5 The Roman Empire and the Untermenschen

6 Muhammad and the Rise of Islamic Intelligence

7 Inquisitions and Counter-Subversion

8 Renaissance Venice and the Rise of Western Intelligence

9 Ivan the Terrible and the Origins of Russian State Security

10 Elizabeth I, Walsingham and the Rise of English Intelligence

11 The Decline of Early Stuart and Spanish Intelligence, and the Rise of the French Cabinet Noir

12 Intelligence and Regime Change in Britain: From the Civil War to the Popish Plot

13 Intelligence in the Era of the Sun King

14 Codebreakers and Spies in Ancien Régime Europe: From the Hanoverian Succession to the Seven Years War

15 Intelligence and American Independence

16 The French Revolution and the Revolutionary Wars

17 The Napoleonic Wars

18 Intelligence and Counter-Revolution. Part I: From the Congress of Vienna to the 1848 Revolutions

19 Intelligence and Counter-Revolution. Part II: From 1848 to the Death of Karl Marx

20 The Telegraph, Mid-Century Wars and the ‘Great Game’

21 ‘The Golden Age of Assassination’: Anarchists, Revolutionaries and the Black Hand, 1880–1914

22 The Great Powers and Foreign Intelligence, 1890–1909

23 Intelligence and the Coming of the First World War

24 The First World War. Part I: From the Outbreak of War to the Zimmermann Telegram

25 The First World War. Part 2: From American Intervention to Allied Victory

26 SIGINT and HUMINT between the Wars

27 The ‘Big Three’ and Second World War Intelligence

28 Intelligence and the Victory of the Grand Alliance

29 The Cold War and the Intelligence Superpowers

30 ‘Holy Terror’: From the Cold War to 9/11

Conclusion: Twenty-First-Century Intelligence in Long-Term Perspective

Bibliography

Abbreviations Used in the Notes and References

Notes

Acknowledgements

Index

List of Illustrations

Integrated Illustrations

Aldegonde’s solution of Don Juan of Austria’s cipher. (TNA 106/1 document 58 f.144)

The Babington postscript and cipher, 1586. (TNA SP12/193/54)

Forged addition to the Babington postscript and cipher, 1586. (TNA SP12/193/54)

Letter from Pompeo Pellegrini to Jacopo Manucci, 1587. (TNA SP 94/2 part 2 f82)

The Monteagle letter, 1605. (Paul Fearn/Alamy)

Letter from Philip III to the 7th Duke of Medina Sidonia, 1609. (John D Rockefeller Library, Williamsburg, VA)

Mercurius Aulicus, 1645. (Copyright © British Library Board/Bridgeman Images)

Title page of ‘The Kings Cabinet opened’, 1645. (Copyright © British Library Board/Bridgeman Images)

The death warrant of Charles I. (Shelfmark: HL/PO/JO/10/297A)

Washington’s letter to Colonel Elias Dayton, 1777. (Beinecke Rare Book and Manuscript Library, Yale University, CT)

The Great Paris Cipher, c. 1812. (TNA WO 37/9)

A wanted notice for Winston Churchill, South Africa. 1899. (PA Archive)

Frontispiece and title page of William le Queux’s Spies of the Kaiser, Hurst & Blackett, 1909.

Part of the Zimmermann telegram as decrypted by Room 40, 1917.

‘Germany seeks an alliance against us’. Headline from The New York Times, 1917.

German conditions accepted by Lenin and others on boarding a sealed train in Zurich, 1917.

MI5 New Year card by Eric Holt-Wilson and Byam Shaw, 1918. (Service Archives)

The GARBO network, the largest network of bogus agents in intelligence history. (TNA WO 208/4374)

John Walker’s map of a clandestine drop point for the KGB. (Copyright © Naval Institute Heritage Group)

Mossad logo.

Plates

1. Spies sent to scout the land of Canaan. Woodcut from the Cologne Bible, 1478–80. (Granger Collection/Alamy)

2. Clay tablet from the Amarna Letters, inscribed in Babylonian cuneiform. (Copyright © The Trustees of the British Museum)

3. A Chinese bamboo copy of The Art of War. (Special Collections & University Archives, University of California, Riverside)

4. A haruspex observing a liver of a sacred animal, Rome. (Falkensteinfoto/Alamy)

5. Yaqub ibn Ishaq al-Kindi. (Art Directors & TRIP/Alamy)

6. Pages from al-Kindi’s On Deciphering Crytopgraphic Messages containing frequency analysis. (Süleymaniye Ottoman Archive MS 4832)

7. Letterbox on the wall of the Doge’s Palace, Venice, Italy

8. The Doge Leonardo Loredan, by Giovanni Bellini. (Copyright © The National Gallery, London/Scala, Florence)

9. The Return of the Prodigal Son by Robusti Jacopo Tintoretto, Doge’s Palace, Venice, Italy. (Copyright © Cameraphoto/Scala, Florence)

10. Anonymous portrait of the famous Spanish conquistador Hernán Cortés, eighteenth century. (Museum of the Royal Academy of Fine Arts of San Fernando, Madrid)

11. Cortes meets Montezuma, from ‘Homenaje a Cristobal Colon’ by Alfredo Chavero, 1892. (Copyright © British Library Board/Bridgeman)

12. Silver candlestick depicting Ivan the Terrible on horseback, c. seventeenth century. (Paul Fearn/Alamy)

13. Still from Tsar, directed by Pavel Lungin, 2009.

14. Sir Francis Walsingham, attributed to John De Critz the Elder, c. 1589. (Copyright © National Portrait Gallery, London)

15. The Family of Henry VIII: An Allegory of the Tudor Succession, by Lucas de Heere, c. 1572. (National Museum Wales/Bridgeman)

16. Queen Elizabeth I by Isaac Oliver, c. 1600. (Hatfield House/Bridgeman)

17. A Spy from Perugino’s Nova Iconolgia, 1618. (Dr Cornelia Manegold)

18. The Security Services all-seeing eye. (Service Archives)

19. Wooden sculpture of a spy by Francesco Pianta, c. 1657–8. (Copyright © Lessing Archive)

20. Antoine Rossignol des Roches. (Copyright © Château de Versailles, Dist. RMN-Grand Palais)

21. Père Joseph. (Bibliothèque nationale de France, QB-201 (31)-FOL)

22. The visit of Louis XIV to the Château de Juvisy by Pierre-Denis Martin, c. 1700. (Copyright © Victoria and Albert Museum)

23. John Wallis by David Loggan. (Scottish National Portrait Gallery, bequest of William Findlay Watson)

24. Blue plaque commemorating John Thurloe, Secretary of State, 1652.

25. John Thurloe by Thomas Ross, eighteenth century. (Copyright © Image; Crown Copyright: UK Government Art Collection March 2014)

26. Aphra Behn by Peter Lely, c. 1670. (Yale Center for British Art, bequest of Arthur D. Schlechter/Bridgeman)

27. Tomb of Aphra Behn in Westminster Abbey.

28. Attack on the Medway, by Pieter Cornelisz van Soest, c. 1667. (Copyright © National Maritime Museum)

29. King George I, studio of Sir Godfrey Kneller, based on a work of 1714. (Copyright © National Portrait Gallery)

30. Edward Willes by John Faber Jr, after Thomas Hudson, 1750. (Copyright © National Portrait Gallery)

31. Count Wenzel Anton Kaunitz by Jean-Etienne Liotard, 1762. (Private collection. Photograph © Christie’s/Bridgeman)

32. Charles Genevieve Louis Auguste André Timothée d’Éon de Beaumont, called the Chevalier d’Éon, by Thomas Stewart. (Private collection. Photograph © Philip Mould/Bridgeman)

33. George Washington as colonel of the Virginia Militia by Charles Wilson Peale, 1772. (Washington and Lee University)

34. Surrender of Cornwallis at Yorktown by Nathaniel Currier. (Michele and Donald D’Amour Museum of Fine Arts, Springfield, MA. Gift of Lenore B. and Sidney A. Alpert, supplemented with Museum Acquisition Funds. Photography by David Stansbury)

35. Sir George Scovell by William Salter, 1834–40. (Copyright © National Portrait Gallery)

36. Russian postage stamp depicting Barclay de Tolly. (Personal collection of Andrew Krizhanovsky)

37. The Congress of Vienna by Jean Godefroy, 1819. (Copyright © RMN-Grand Palais/G. Blot

38. Paul Pry at the Post Office, Punch, 1844. (Copyright © Punch Ltd)

39. Karl Marx in London, 1864. (AKG/Sputnik)

40. Bomb attack on Scotland Yard, 1884.

41. Thaddeus S. Lowe observing the battle from his balloon ‘Intrepid’, 1862. (Universal History Archive/UIG/Bridgeman)

42. Leadership of the Okhrana at their headquarters in St Petersburg, 1905.

43. Roman Malinovsky. (Paul Fearn/Alamy)

44. Evno Azev, 1910.

45. French President Raymond Poincaré visiting Nicholas II in Russia, 1914. (Copyright © Hulton-Deutsch/CORBIS/Getty)

46. Dragutin Dimitrijevic, c. 1900.

47. Dimitrijevic on trial, 1917.

48. Franz von Papen, Washington D.C., 1914. (Copyright © Hulton-Deutsch/CORBIS/Getty)

49. Franz von Rintelen, Victoria Station, 1933. (AP/Topfoto)

50. Black Tom Depot in Jersey City, NJ, shortly after the munitions explosion in 1916. (Buyenlarge/Getty)

51. Sir William Reginald Hall by Walter Stoneman, 1917. (Copyright © National Portrait Gallery)

52. 1890s advertisement for Pears soap, including the portrait of Commander William ‘Bubbles’ James by his grandfather Sir John Millais. (The Advertising Archives)

53. Georges Painvin, c. 1911–18.

54. General Allenby’s entrance into Jerusalem through Jaffa Gate, 1917. (Frank & Frances Carpenter/Library of Congress)

55. Vladimir Lenin in disguise, 1917. (AKG)

56. Corpse of Sidney Reilly on private display in Lubyanka, 1925.

57. Felix Dzerzhinsky with Joseph Stalin 1924. (ITAR-TASS News Agency/Alamy)

58. Joseph Stalin and Nikolai Yezhov at the shore of the Moscow–Volga Canal, 1937. (ITAR-TASS/Getty)

59. Stalin without Yezhov. (AFP/GettyImages)

60. Lavrenti Beria with Svetlana Stalin.

61. Franklin D. Roosevelt aboard the Nourmahal, 1935. (National Archives and Records Administration, Washington D.C./Franklin D. Roosevelt Library Public Domain Photographs.)

62. Purple diplomatic cipher machine. (Courtesy of the National Cryptologic Museum, Maryland)

63. Enigma I. (Museo Nazionale della Scienza e della Tecnologia)

64. US Army Signals Security Detachment, Brickhill Manor, c. 1942.

65. Codebreaking at Bletchley Park, 1943. (SSPL/Getty)

66. National leaders gather at the Yalta Conference, 1945. (Getty. (Inset) Alger Hiss. (Everett Collection/Alamy)

67. George Abramovich Koval.

68. President Putin toasts Koval at the atom spy’s posthumous ‘Hero of Russia’ award, 2007. (Sovfoto/UIG/Getty)

69. Marshal Tito receives Iosif Grigulevich (alias Teodoro Castro), 1953.

70. Henry Cabot Lodge Jr displays a plaque from the US Moscow embassy bugged by the KGB, 1960. (Bob Gomel/Life/Getty)

71. The headquarters of the KGB (Committee for State Security), Lubyanka Square in Moscow, 1987. (Copyright © Arnold Drapkin/Alamy)

72. St Sofia, Parish Church of the FSB in Central Moscow. (Professor Neil Kent)

73. SVR chief Mikhail Fradkov and Kim Philby’s widow at the unveiling of a memorial plaque to Philby at SVR HQ, 2010. (STR/AFP/Getty Images)

74. US ambassador to the United Nations Adlai Stevenson displays imagery intelligence at a meeting of the UN Security Council, 1962. (Getty)

75. US Secretary of State Colin Powell addressing the UN Security Council, New York City, 2003. (Mario Tama/Getty)

76. Powell’s UN presentation slide showing alleged mobile production facility for biological weapons, Iraq. (US Department of State)

77. Oleg Gordievsky in London, 1982. (Security Service Archives)

78. Reagan congratulates Gordievsky in the Oval office, Washington, 1987. (Able Archer Sourcebook National Security Archives)

79. Vasili Mitrokhin, 1992.

80. Moinul Abedin. (Security Service Archives)

81. Dhiren Barot. (Security Service Archives)

82. President Obama and his National Security team watch the intelligence-led operation to capture Bin Laden in 2011. (White House/Pete Souza)

83. Dangerous Love poster campaign, Beijing, 2016. (Ng Han Guan/AP/REX/Shutterstock)

Introduction: The Lost History of Global Intelligence

Twenty-first-century intelligence suffers from long-term historical amnesia. Early in the Cold War, the historian Sherman Kent, founding father of US intelligence analysis, complained that intelligence was the only profession without a serious literature: ‘From my point of view this is a matter of greatest importance. As long as this discipline lacks a literature, its methods, its vocabulary, its body of doctrine, and even its fundamental theory run the risk of never reaching full maturity.’¹ It was more difficult to learn the historical lessons of intelligence than of any other profession mainly because there was so little record of most of its past experience. Western intelligence services during and after the Second World War knew very little about intelligence before – and sometimes during – the First World War. That was true even of ‘Station X’ at Bletchley Park, which had greater success in breaking enemy ciphers than any codebreaking agency had ever had before. Three times over the previous 500 years, Britain had faced major invasion threats – from the Armada of Philip II of Spain in 1588, from Napoleon at the start of the nineteenth century, and from Hitler in 1940. But the Bletchley Park codebreakers who solved Hitler’s ciphers, though they included some notable historians, had no idea that their predecessors had broken those of Philip II and Napoleon at other times of national crisis.* No other wartime profession was as ignorant of its own past. It is impossible, for example, to imagine an economist who had never heard of the Industrial Revolution.

For centuries before the Second World War, educated British people knew far more about intelligence operations recorded in the Bible than they did about the role of intelligence at any moment in their own history. The Christian Old Testament (the Jewish Tanakh) contains more references to spies than any history of Britain or of most other countries. Most Victorian schoolchildren, as well as adults, knew, for example, how Moses had sent agents to spy out the Promised Land; how Joseph, having become the Egyptian Pharaoh’s Vizier, pretended not to recognize his errant elder brothers and accused them of being spies who had come to identify weak points in Egypt’s defences; and how Judas Iscariot became a paid agent of the high priests on ‘Spy Wednesday’ in Holy Week and betrayed Jesus. Moses (Musa) is also a Muslim prophet. There are 136 references to him in the Quran – far more than to any other human being.* In the Quran, as in the Tanakh, God instructs Moses to send twelve spies to spy out the Promised Land He has given the Israelites. Their mission ended in the first major recorded intelligence failure – due less, as was frequently the case in later centuries, to the quality of the intelligence than to the use made of it. Forty years later, after a better-organized intelligence operation, according to the biblical account, Moses’ successor Joshua led the Israelites into the Promised Land.²

Ever since the missions of Moses’ and Joshua’s spies into Canaan, the first priority of intelligence operations has been to obtain secretly information unavailable from open sources. George Tenet, director of the US Central Intelligence Agency (CIA) at the beginning of the twenty-first century, summed up the Agency’s main mission in three words: ‘We steal secrets.’³ During the Cold War, Allen Dulles, the longest-serving CIA director, wrote that, over the centuries, intelligence organizations had also shown themselves ‘an ideal vehicle for conspiracy’.⁴ From earliest times, intelligence has often involved covert operations intended to influence the course of events by methods ranging from deception to assassination – ‘active measures’, as the twentieth-century KGB called them. Deception involving a bogus defector played a key role in the Athenian victory at the naval Battle of Salamis in 480 BC, at a critical moment during the invasion of Greece by the Persian Empire. For the next two and a half millennia, however, the Salamis deception attracted only a tiny fraction of the interest aroused by the fictional deception of the Trojan Horse, which first featured in Homer’s Odyssey and later, in greater detail, in the Aeneid by the Roman poet Virgil.⁵ Even in the twenty-first century, public understanding of intelligence operations is frequently coloured – if not confused – by spy fiction. No real intelligence practitioner, alive or dead, is remotely as well known as James Bond.

No author in ancient Greece or Rome showed a grasp of strategic intelligence which compared with that of the Jewish priests who, probably during the two centuries before Athens’s golden age, had written the biblical accounts of espionage in Canaan by the spies of Moses and Joshua. Thucydides, the greatest historian of European classical antiquity, famously wrote of the origins of the Peloponnesian War in the fifth century BC, ‘The growth of the power of Athens, and the alarm which this inspired in Sparta, made war inevitable.’⁶ But intelligence on its rival was not a priority for either Athens or Sparta – or for Thucydides, though he was a general as well as a historian.

The first books to argue that intelligence should have a central role in war and peace were written not in Greece or Rome but in ancient China and the Indian subcontinent: The Art of War (Sunzi bingfa), traditionally ascribed to Confucius’s contemporary, the Chinese general Sun Tzu (c. 544–c.496 BC); and the Arthashastra, a manual on statecraft attributed to a senior adviser of the founder of the Mauryan dynasty which dominated India between 322 and 187 BC. Central to the revival of interest in intelligence in twentieth-century China and India was the rediscovery, after centuries of neglect, of these two ancient works. In India today, the Arthashastra has a status similar to that of Aristotle’s Politics and Machiavelli’s The Prince in the West. Sun Tzu has been far more revered in Communist China than in any imperial dynasty since the third century ad. Even in the United States, he is more frequently quoted than any pre-twentieth-century Western writer on intelligence.⁷

During what Europeans later called the ‘Dark Ages’, when the influence of Sun Tzu and the Arthashastra was in decline in Asia, the global leaders in intelligence operations were Muhammad and the Islamic Caliphate established in the Middle East after his death in 632. While uniting the tribes of the Arabian Peninsula under the banner of Islam, Muhammad fought twenty-seven battles and instigated about fifty armed raids. Probably his most widely read Muslim biographer today, Safiur Rahman Al Mubarakpuri, claims that ‘The Prophet was the greatest military leader in the entire world.’⁸ The Hadiths (sacred records of Muhammad’s words and deeds) give many instances of how, during his military campaigns, he paid close attention to intelligence. Al Mubarakpuri and other Muslim biographers of Muhammad, however, barely mention his intelligence operations,* which are more frequently cited by Islamist extremists.⁹

The only intelligence operations in which medieval Europe was probably the world leader were devoted to what was later called counter-subversion. Catholic ‘inquisitions’ to root out religious heresy anticipated the very much larger campaigns by security and intelligence services in modern secular one-party states against what the KGB denounced as ‘ideological subversion’. Probably no pre-twentieth-century security service was capable either of the organized mass interrogation or of the elaborate record keeping involved in, for example, mid-thirteenth-century clerical inquisitions into the Cathar heresy in Languedoc. Twentieth-century secular interrogators were often unaware that some of their methods had been devised by clerical inquisitors centuries earlier. ‘Waterboarding’ (simulated drowning), used by US interrogators to obtain intelligence from senior Al Qaeda personnel after 9/11, was a technique devised half a millennium before by the Spanish Inquisition.¹⁰

Though few historians have noticed it, the European Renaissance was a major turning point in the history of intelligence. For the first time, Europe established a global lead in intelligence which went unchallenged until the American Declaration of Independence. Like twentieth-century interrogators, few if any Renaissance advocates and practitioners of intelligence were aware of how frequently they were reinventing the wheel. It did not occur to Machiavelli that his often-quoted dictum ‘Nothing is more worthy of a good general than the endeavour to penetrate the designs of the enemy’¹¹ merely repeated a maxim attributed to Sun Tzu almost two millennia before. The Renaissance codebreakers who believed they had invented ‘frequency analysis’ (a turning point in the history of cryptanalysis) had no idea that it had been discovered by the Muslim philosopher and cryptanalyst Yaqub ibn Ishaq al-Kindi in the Baghdad ‘House of Wisdom’ over six centuries earlier. The Ottomans, who made Constantinople the capital of their Muslim empire in 1453, seem to have been equally ignorant of al-Kindi’s achievements.¹²

The Renaissance also marked a turning point in the history of diplomacy, which was closely linked to the development of Western intelligence. Hitherto ambassadors had been used only for specific diplomatic missions. The city-states in Renaissance Italy, however, established resident ambassadors in each others’ capitals, a system which became the model for European diplomacy as a whole. Since most resident ambassadors were expected to collect intelligence as well as to represent their governments, the recruitment of spies increased. Early modern diplomacy and intelligence gathering thus overlapped. From 1573 until his death in 1590, Elizabeth’s principal Secretary of State, Sir Francis Walsingham, successfully combined the roles of Foreign Secretary and intelligence chief, with daily access to the Queen and her Chief Minister.¹³

With one important exception, no clear separation between diplomacy and espionage emerged in Western and Central Europe until professional intelligence bureaucracies were founded in the later nineteenth century. The exception was what is now called signals intelligence (SIGINT), derived from the interception and decryption of communications. Beginning in the Renaissance, small, specialized codebreaking agencies were founded which required cryptanalytic expertise unavailable in other branches of government. Half a millennium later, the role of SIGINT remains a major gap in most histories of pre-twentieth-century politics and international relations. Even histories of Elizabethan England which recognize the importance of Walsingham’s role rarely give adequate credit to the achievements of his chief cryptanalyst, who broke the ciphers of both Mary, Queen of Scots, and Philip II of Spain, and was personally rewarded with a royal pension by a grateful Elizabeth I.¹⁴

There was a direct, though incomplete, correlation (little noticed by historians) during the seventeenth and eighteenth centuries between the success of European statesmen in their conduct of foreign policy and their skill in the exploitation of SIGINT. The ablest French statesman of the seventeenth century, Cardinal Richelieu, was also the founder of the first French SIGINT agency, the cabinet noir (black chamber), a term commonly applied over the next few centuries to similar agencies in other countries. Its head, who had his own château, was better recognized and rewarded than any other pre-twentieth-century cryptanalyst. His British counterpart, a founder member of the Royal Society, was rewarded by Oliver Cromwell with a chair at Oxford University, which he kept for half a century. The ablest international statesman of the mid-eighteenth century, Count (later Prince) Wenzel Anton Kaunitz, successively leading Austrian diplomat and Chancellor, was similarly the ablest user of SIGINT.¹⁵ The skill of both Richelieu and Kaunitz in the exploitation of intelligence was largely forgotten by later generations. Just as no French Foreign Minister before the First World War rivalled Richelieu’s grasp of SIGINT, so none of Kaunitz’s successors in imperial Vienna possessed his flair for diplomatic intelligence.

The first country to challenge Europe’s global lead in intelligence was the United States after the Declaration of Independence in 1776. Ironically, the commander-in-chief of the rebel Continental Army, George Washington, had earlier learned the importance of military intelligence as a British army officer fighting the French, who were to become his allies against the British in the Revolutionary War. Washington’s intelligence operations in both HUMINT (human intelligence) and SIGINT outclassed those of his British opponents. As first President of the United States, he was granted a fund by Congress to finance foreign intelligence which rose to about 12 per cent of the federal budget – a higher percentage than the massive US intelligence expenditure of the late twentieth and early twenty-first centuries.¹⁶

The history of intelligence is not linear. In both the United States and the United Kingdom foreign intelligence was a lower priority at the end of the nineteenth century than at the beginning. Save for Washington’s immediate successors, no President attempted to learn from his example. Even the Civil War marked only a temporary halt in the long-term decline of US intelligence. Victorian intelligence too was in decline. As a result of the closure of the Deciphering Branch in 1844, following parliamentary protests, Britain entered the First World War, unlike any of the major wars of the late sixteenth, seventeenth, eighteenth and early nineteenth centuries, without a SIGINT agency.* At the outbreak of war in 1914, the US President, Woodrow Wilson, and the British Prime Minister, Herbert Asquith, had a weaker grasp of intelligence than their eighteenth-century predecessors, George Washington and the two Pitts. These failings passed unnoticed by both contemporaries and most subsequent historians. Also unnoticed has been the fact that, though less intelligent than Wilson and Asquith, Tsar Nicholas II regarded intelligence as a much higher priority. Because of their lack of awareness of the role of SIGINT, none of the numerous twentieth-century histories of the origins of the First World War mentions that, in both St Petersburg and Paris, the first indication in July 1914 that Austria was preparing to deliver the ultimatum to Serbia which was to trigger the outbreak of war came from diplomatic decrypts.¹⁷

The First World War, thanks to its unprecedented scale and intensity in an era of equally unprecedented technological change, marked an even greater turning point in intelligence history than the Renaissance. The vast increase in communications made possible by the nineteenth-century invention of the telegraph and wireless gave SIGINT a greater operational role than in any previous conflict. Particularly in the early stages of the war, however, war leaders’ ignorance of past experience made them ill-equipped to put intelligence to good use. Despite the pre-war successes of Russian foreign intelligence, Russian military intelligence at the outbreak of war was far less effective than it had been a century before on the eve of Napoleon’s invasion of Russia. Within weeks its incompetence had led to Russia’s worst defeat of the war.

The use of intelligence by governments and high commands in the First World War thus owed much to improvisation. Though SIGINT was to prove crucial to British naval operations, the Director of Naval Intelligence in August 1914 had given no thought to its wartime role until intercepted ciphered German naval messages began to pile up on his desk. These unexpected intercepts prompted the foundation in the Admiralty of the first British SIGINT agency since 1844, which, with its military counterpart, MI1b, became the wartime world leader. The creation of the Bolshevik Cheka, forerunner of the KGB, only six weeks after the October Revolution of 1917 was also the result of rapid improvisation. Though it developed between the wars into the world’s largest and most powerful intelligence service, it was originally intended only as a temporary expedient to deal with an impending strike by state employees.¹⁸

More than any previous war, the First World War led to attempts, which varied greatly from country to country, to learn its intelligence lessons. British SIGINT successes in the Second World War owed much to lessons learned in the First – among them the need for better coordination. The failure of the United States to learn the same lesson was a major reason for the SIGINT confusion which preceded, and helped to make possible, the Japanese surprise attack on Pearl Harbor. For over a year before Pearl Harbor, the rival US military and naval cryptanalysts were bizarrely instructed to decrypt Japanese diplomatic telegrams on alternate days.¹⁹

All the Big Three in the Second World War – Joseph Stalin, Winston Churchill and Franklin Roosevelt – were more influenced than has usually been realized by their very different experiences of First World War intelligence. After the Bolshevik Revolution, Stalin spent years studying and annotating almost every page of the huge, multi-volume intelligence file maintained on him by the Tsarist Okhrana. Churchill had longer and more varied experience of intelligence than any previous war leader. One of the main lessons learned by both him and the less experienced Roosevelt from the First World War was the importance of Allied intelligence collaboration in the Second. Roosevelt’s admiration for what he called Britain’s ‘wonderful intelligence service’ in the First World War later led him as President to approve intelligence collaboration with Britain even before Pearl Harbor, while the United States remained officially neutral. The wartime UK/US SIGINT alliance became the cornerstone of a transatlantic Special Relationship which still continues.²⁰ Contrary to the belief of many of its practitioners, this is not the first SIGINT alliance in British history (which was concluded between Elizabethan England and the Dutch),²¹ but it remains the most important.

Since the late twentieth century, far more has been published on intelligence operations in the Second World War than had ever been published on the role of intelligence in any previous conflict. For a generation after the war, however, historians were seriously handicapped by official concealment of the main Western (especially British) wartime intelligence successes against Nazi Germany: chief among them the ULTRA intelligence derived from breaking high-grade enemy ciphers and the Double-Cross System, the most successful strategic deception in the history of warfare. Only when these operations were declassified in the early and mid-1970s did it become possible to write accurate accounts of the war against the U-boats in the North Atlantic and the D-Day landings on the Normandy beaches.²²

Despite the growth of intelligence history since the 1980s, study of the Cold War nowadays suffers from much the same neglect of SIGINT which diminished, and sometimes distorted, understanding of Second World War intelligence until the mid-1970s. None of President Harry S. Truman’s biographers mentions that he was so impressed by his brief experience of the wartime British–American SIGINT alliance that he approved its peacetime continuation – and in so doing profoundly influenced the development of the Special Relationship. Though studies of Cold War US foreign policy invariably mention the Central Intelligence Agency (CIA), there is rarely any reference to the larger National Security [SIGINT] Agency (NSA). The virtual exclusion of SIGINT from the history of post-war international relations helps to explain why so many well-educated Americans mistakenly believed that the atom spies Julius and Ethel Rosenberg, both executed in the electric chair, and other leading Soviet agents detected in the United States were the innocent victims of official paranoia and McCarthyite hysteria. Soviet decrypts declassified by the NSA and GCHQ after the Cold War show that they were guilty as charged.²³

The history of superpower rivalry during the Cold War has also been distorted by the KGB’s ability to keep its operational secrets far more successfully than the CIA. No account of American policy in the Third World omits the role of CIA covert action. By contrast, KGB covert action (‘active measures’) passes almost unmentioned in most histories both of Soviet foreign policy and of developing countries. The result has been a curiously lopsided account of the secret Cold War in the Third World – the intelligence equivalent of the sound of one hand clapping. The admirable history of the Cold War by John Lewis Gaddis, for example, refers to CIA covert action in Chile, Cuba and Iran, but makes no reference to extensive KGB operations in the same countries.²⁴ In reality, as material exfiltrated from KGB archives after the Cold War reveals, from at least the early 1960s onwards the KGB played an even more active global role than the CIA.

In Britain, exaggerated official secrecy about the intelligence services, enforced by both Labour and Conservative governments throughout the Cold War, also inhibited the development of intelligence history. Even the existence of the Secret Intelligence Service (SIS or MI6) was not officially acknowledged until the Queen’s Speech at the opening of Parliament in 1992. In the same year Stella Rimington became the first Director-General of MI5 to be publicly identified. Gender historians who had been unable to research female recruitment by MI5 even as far back as the First World War, because all its records (many since declassified)* were still secret, were surprised to discover that the Security Service had become the first of the world’s major intelligence services to have a female head. The incomprehension of British tabloids at Rimington’s appointment was reflected in headlines such as ‘HOUSEWIFE SUPERSPY’ and ‘MOTHER OF TWO GETS TOUGH WITH TERRORISTS’.

For much of the Cold War, Western intelligence analysts often failed to realize the handicap imposed on them by the shortage of academic research and their own lack of historical perspective. A notable exception was the Yale historian Sherman Kent, who spent a total of seventeen years as an analyst during the Second World War and Cold War. The world’s first peer-reviewed journal on intelligence, Studies in Intelligence, was founded in 1955, not by academics in university departments but by Sherman Kent in the CIA. Though originally classified, a majority of its articles on intelligence methodology and history are now accessible online. Like academic journals, but unlike most intelligence reports, Studies in Intelligence announced at the outset: ‘All statements of fact, opinion, or analysis . . . are those of the Authors. They do not necessarily reflect official positions or views of the Central Intelligence Agency or any other US government entity, past or present.’ The first academic journal, Intelligence and National Security, co-edited by Michael Handel and the author of this volume, did not begin publication until over thirty years after Studies in Intelligence.

Lack of long-term historical perspective was as evident in intelligence assessment at the end of the twentieth century as at the beginning of the Cold War. During the Second World War and Cold War, Western intelligence agencies had been well versed in Nazi and Communist ideology. But the increasingly secularized late-twentieth-century West found it far more difficult to grasp the appeal of Islamic fundamentalism. Its incomprehension of the political power of religious extremism was vividly displayed during the crisis in Iran which led early in 1979 to the fall of the pro-Western Shah and the rise of the 78-year-old Shia Ayatollah Ruhollah Khomeini. ‘Whoever took religion seriously?’ demanded one surprised State Department official after Washington had been taken aback by Khomeini’s popular triumph.²⁵ Over the next decade, many Western intelligence analysts similarly failed to grasp that understanding the appeal of Al Qaeda and the terrorist threat which it posed required serious study of its theology. Those with the best understanding of the Islamist terrorist threat before the 9/11 attacks, among them the leading academic expert on ‘Holy Terror’, Bruce Hoffman, were those with a long-term perspective.²⁶

The dawn of cyberwarfare and the looming threat of terrorist use of weapons of mass destruction confront twenty-first-century intelligence communities with dramatic new challenges. To respond to them as effectively as possible, however, they will need a long-term perspective which has often been forgotten or ignored. Learning from past experience in intelligence, as in most other fields, is, of course, easier said than done. To quote the historian John Bew:

History does not lend itself easily to the PowerPoints or executive summaries on which our policymakers increasingly rely . . . A genuine understanding of history requires a patience that is not easy to reconcile with the urgency of policy. A good starting point is to view the past as a source of wisdom rather than revelation.²⁷

‘Only long-term historical perspective’, writes Quentin Skinner, former Regius Professor of History at Cambridge, can ‘liberate us from the parochialism of our own forms of cultural analysis’.²⁸ Strategic-intelligence analysis which ignores the long term is necessarily parochial.

There are some signs of progress. The academic study of intelligence history, which barely existed little more than a generation ago, is now thriving in a growing minority of universities on both sides of the Atlantic. In the early twenty-first century, all three British intelligence agencies, as well as the Joint Intelligence Committee, commissioned official or ‘authorized’ histories by outside professional historians, who were given unprecedented access to their files.* All followed the pioneering principles set out by Sherman Kent in Studies in Intelligence half a century earlier.†

More frequently than ever before, intelligence in the twenty-first century has become front-page news. Official reports on 9/11 and the Iraq War, as well as unofficial whistleblowers, have published unprecedented amounts of intelligence material.²⁹ The significance of recent intelligence history, however, can only be adequately understood in long-term perspective. Like much else in the early twenty-first century, intelligence studies frequently suffer from what I have called Historical Attention-Span Deficit Disorder (HASDD).

The history of intelligence operations is far older than any of today’s intelligence agencies. The Secret World sets out to recover some of the lost history of global intelligence over the last three millennia, to show how it modifies current historiography, and to demonstrate its continuing relevance to intelligence in the twenty-first century.

1

In the Beginning: Spies of the Bible and Ancient Egypt from Moses to the Last Supper

The first major figure in world literature to emphasize the importance of good intelligence was God. After the Israelites had escaped from captivity in Egypt and crossed the Red Sea, supposedly in about 1300 BC, God told Moses to send spies to reconnoitre ‘the land of Canaan, which I give unto the children of Israel’. Since no trained intelligence personnel were available for the mission to the Promised Land, Moses selected, on God’s instructions, one leading man from each of the twelve tribes of Israel.

And Moses sent them to spy out the land of Canaan, and said unto them, ‘Get you up this [way] southward, and go up into the mountain:

And see the land, what it [is]; and the people that dwelleth therein, whether they [be] strong or weak, few or many;

And what the land [is] that they dwell in, whether it [be] good or bad; and what cities [they be] that they dwell in, whether in tents, or in strongholds;

And what the land [is], whether it [be] fat or lean, whether there be wood therein, or not.

And be ye of good courage, and bring of the fruit of the land. Now the time [was] the time of the first ripe grapes.’¹

Over three millennia later, the Victorian schoolboy George Aston, who went on to become a senior naval intelligence officer with a knighthood, was first attracted to secret service work by a picture in a children’s book of Bible stories which showed two of Moses’ agents on their way back from a forty-day mission to Canaan, staggering under the weight of a huge bunch of grapes hanging from a pole carried on their shoulders.² In Victorian Britain, at a time when Britain had no professional secret intelligence agencies but knowledge of Bible stories was more widespread than at any other time in British history, the use of spies by Moses and his successor Joshua was far better known than any English or British intelligence operation over the previous millennium.

The Old Testament (also known as the Tanakh, the Hebrew Bible) contains more references to spies than any published history of Britain or of most other countries. The first mention comes in the Genesis story of Joseph and his ten elder half-brothers, who were enraged by the favouritism shown to Joseph by their father, Jacob (later renamed Israel), exemplified by his gift to him of a coat of many colours. Some of Joseph’s brothers also felt, with some justification, that he was spying on them. Jacob tactlessly instructed Joseph, then aged only seventeen, to tell him how his brothers were managing the family flock; Joseph gave his father an ‘evil report’ of their behaviour. To rid themselves of Joseph, his brothers sold him for twenty pieces of silver to slave traders, who took him to Egypt. By ripping the coat of many colours and dipping it in goat’s blood, his brothers persuaded their father that Joseph had been eaten by a wild animal.

Unknown to his family, Joseph eventually rose to become the Egyptian Pharaoh’s Vizier. When his elder brothers came to Egypt to buy corn during a famine, they were received by Joseph and failed to recognize him in his now exalted position. Joseph, however, recognized them but pretended not to and accused them of being spies who had come to identify weak points in Egypt’s defences. After he had subjected his brothers to several covert operations (including planting apparently stolen objects in their luggage), the story ended happily. Overcome with emotion, Joseph identified himself as their brother, was reconciled with them, and invited his father and extended family to take up residence in Egypt.³

Whatever the historicity of the Joseph story in Genesis (which is similar to the account given in the longest chapter of the Quran),⁴ it reflects one probably not uncommon reality of life in the era of the Hebrew Bible: that, in a society which was frequently in conflict, strangers and travellers who aroused suspicion were liable to be accused of being spies (which they sometimes were). Even in twentieth-century Britain, spy scares (mostly unfounded) involving supposedly suspicious foreigners were common on the eve, and at the start, of both world wars.⁵

Though the first scriptural reference to spies comes in Genesis, no organized espionage operation is mentioned in the Bible until God commands Moses to ‘spy out the land of Canaan’. Moses’ twelve spies, sent from the desolate Wilderness of Paran (north-west of today’s Aqaba), were amateurs, chosen for their social standing rather than because they had shown any talent for intelligence gathering. The intelligence they were instructed to obtain in preparation for the advance into Canaan was much in line with that sought by modern military commanders. The great nineteenth-century German military theorist, Carl von Clausewitz, defined the intelligence required by commanders on hostile territory as ‘every sort of information about the enemy and his country – the basis, in short, of our own plans and operations’.⁶

All twelve of Moses’ spies reported on their return that Canaan was a land flowing with milk and honey. Two of them, Caleb and Joshua, argued that, since the Israelites had God’s support, they had nothing to fear from invading the Promised Land: ‘Let us go up at once and possess it.’ God was outraged, however, with the pessimistic intelligence assessment of the other ten – an ‘evil report’, according to the Bible, which concluded that the existing inhabitants would prove too strong for the Israelites: ‘We be not able to go up against the people; for they [are] stronger than we.’ The Canaanites, claimed the fearful ten, included giants who had made them feel no bigger than grasshoppers.⁷ The Quran contains a similar account of the spies’ mission, though Joshua and Caleb are identified only as two ‘God-fearing men’ on whom God ‘had bestowed His grace’.⁸

The Bible implies that the mission of the twelve spies was intended as a test of their faith as well as of their aptitude for intelligence gathering. All but Caleb and Joshua failed on both counts. The other ten publicly used their mission report to justify their own belief that, despite the support of the God who had delivered them from bondage in Egypt and parted the waters of the Red Sea to make possible their escape, any attempt to take possession of the Promised Land would prove impossibly dangerous. Unless, improbably, Canaan really was populated by giants, they were guilty of distorting the intelligence they had collected to dissuade the Israelites from going ahead. Both the Bible and the Quran record that the ten won the public debate. The ‘whole congregation’ sided with them, threatened to stone Caleb and Joshua, and ungratefully complained that the Israelites would have been better off remaining in captivity in Egypt.⁹ According to the Quran, the Israelite majority told Moses they would never enter the promised land so long as its current inhabitants remained: ‘So go, you and your Lord, and fight. We are remaining right here.’¹⁰

God’s furious response exceeded any other recorded instance of divine anger at the misuse of intelligence:

And the Lord said unto Moses, How long will this people provoke me? and how long will it be ere they believe me, for all the signs which I have shewed among them? I will smite them with the pestilence, and disinherit them . . .

The ten who had produced the ‘evil’ report of their Canaan espionage mission all died of the plague. After intercession by Moses, God commuted the punishment of the other Israelites from pestilence and disinheritance to forty years’ wandering in the wilderness. However, God decreed that no Israelite over twenty years of age, with the sole exceptions of the righteous spies Caleb and Joshua, was to live long enough to enter the Promised Land.¹¹

Moses’ death on Mount Nebo (at the age of 120, according to the Bible) coincided with the end of the Israelites’ forty years in the wilderness. The visitor to Mount Nebo today, in present-day Jordan, can still see across the Dead Sea much the same view of the Promised Land that the Bible tells us Moses was allowed to glimpse before his death.¹² God told Joshua, Moses’ successor, that the time had come for the Israelites to enter the Promised Land. The Book of Joshua tells the story of a lightning military campaign in which the Israelites, with divine assistance, defeated, one after the other, the kings of Canaan. Conquest was preceded by espionage. Joshua sent two spies (both unidentified but not, apparently, tribal leaders as forty years before) on a secret mission to the fortress city of Jericho, the first major target of the campaign.¹³

The biblical account of their mission provides the first record of a joint operation involving both of, reputedly, the world’s two oldest professions (said to be prostitutes and spies). Once in Jericho, Joshua’s spies found accommodation with Rahab the harlot (prostitute), whose brothel was embedded in the city wall. Though there may have been a shortage of alternatives, there were obvious disadvantages for the spies in lodging in a brothel, whose clients had no pressing reason to keep their presence secret. Indeed, within twenty-four hours the King of Jericho had received reports that Israelites with a mission to ‘explore the whole country’ were staying in the brothel. After hiding the two spies among bales of flax on her roof, Rahab told the King they had left Jericho: ‘Pursue after them quickly; for ye shall overtake them.’¹⁴ Despite the security problems for the spies posed by the brothel, there were some operational advantages in using it as a base. Travellers to Jericho who visited the prostitutes must have been a useful source of information about the surrounding area. Some modern intelligence agencies have made similar use of brothels’ intelligence potential.*

The greatest asset of the Israelite spies staying in Rahab’s Jericho brothel was Rahab herself, who told the spies, ‘I know the Lord hath given you the land [of Canaan].’ She said that the local inhabitants, having heard how God had rescued the Israelites from captivity in Egypt and parted the Red Sea to enable them to cross, lacked the courage to resist their advance into Canaan. In return for a promise to ensure the safety of her family, Rahab secretly sided with the Israelites. Unlike the majority of Moses’ spies, whose ‘evil’ intelligence report forty years earlier had aroused divine indignation, Joshua’s spies told him on their return from Canaan: ‘Truly the Lord hath delivered into our hands all the land; for even all the inhabitants of the country do faint because of us.’¹⁵

Rahab, however, had only a supporting role in the Israelites’ conquest of Jericho. The success of the conquest owed far more to divine intervention than to human espionage. God enabled the Israelite army to cross the Jordan by temporarily drying up the river bed. After their arrival at Jericho, as instructed by God, they walked around the city walls for seven days in succession, blowing ram’s horn trumpets. On the seventh day, after blowing the trumpets, the army ‘shouted with a great shout’ and the city wall fell down. Rahab and her family, who had marked the location of the brothel by hanging a scarlet cord from the window, escaped from Jericho and settled with the Israelites. All the other inhabitants of Jericho were massacred, along with their livestock.¹⁶

In the New Testament both St Paul and St James praise Rahab’s faith and righteousness in their epistles.¹⁷ The beginning of St Matthew’s Gospel adds a new dimension to the Christian understanding of Rahab by including her in the female line of Jesus’ family tree.¹⁸ According to the very first chapter of the New Testament, Jesus was thus descended, on his mother’s side, from the first female spy identified in world literature.

Attempts by twentieth-century intelligence agencies to draw lessons from the biblical account of espionage in the era of Moses and Joshua include a 1978 study in the classified in-house journal of the Central Intelligence Agency (CIA), Studies in Intelligence, entitled ‘A Bible Lesson on Spying’, which has since been declassified.¹⁹ Its author, John M. Cardwell (possibly a pseudonym), saw analogies between the debacle which followed the Israelites’ public debate of the reports of Moses’ spies on Canaan and the tribulations of the CIA in the mid-1970s, when some its recent operations (including assassination plots and other ‘dirty tricks’) received unprecedented publicity and the Agency was subjected for the first time to Congressional oversight:

If there is a lesson to be learned [from the Canaan espionage operation], it would appear that a strong case is made for the conduct of spying activities in secret by professionals, unencumbered by other political or military responsibilities, and that these professionals should report in secret to higher authority who would make policy decisions without debate. Spies should definitely not participate in the policy-decision-making process, nor should they take their cases to the public. When that occurs, although stoning is passé, the people are likely to throw figurative rocks at the wrong people for the wrong reasons.

Joshua, in Cardwell’s view, avoided both the main mistakes made during the first attempt to spy out the land of Canaan, which, he implied, also demonstrated the folly of the excessive publicity given to intelligence operations and Congressional interference that, he believed, undermined the CIA’s effectiveness in the mid-1970s:

Moses’ operation, conducted by amateurs more or less in the public domain, resulted in a weakening of Moses’ position of authority, led to a loss of the people’s confidence in themselves, and precipitated an extended period of severe national punishment. Joshua’s operation, conducted in private by professionals, led to an achievement of national destiny . . . Joshua certainly did not have an oversight problem, nor did he worry about defining a politically acceptable mission scenario.²⁰

The modern intelligence services most concerned to draw lessons from the espionage missions sent by Moses and Joshua into the Promised Land have, predictably, been those of the state of Israel since its foundation in 1948: Mossad, the foreign intelligence agency, and Shin Bet, the domestic security service. In talks to new recruits at the start of the twenty-first century, Efraim Halevy, head of Mossad, used to cite the behaviour of ten of the spies sent by Moses to spy out the land of Canaan as a cautionary example of the importance of sticking to their assigned missions. The ten had incurred divine wrath by allowing their personal views to override their responsibilities for intelligence reporting.²¹

Both Shin Bet and Mossad take their mottoes from the Hebrew Bible. Shin Bet’s comes from Psalm 121: ‘He who watches over Israel shall neither slumber nor sleep.’ The current Mossad motto is: ‘Where no counsel is, the people fall, but in the multitude of counsellors there is safety’ (Proverbs 11:14). This replaced an earlier, more contentious motto based on Proverbs 24:6: ‘By way of deception, thou shalt conduct war.’²² That motto is still sometimes cited by the current Israeli prime minister, Benjamin Netanyahu. When celebrating the Hanukkah festival with President Shimon Peres and Mossad’s chief, Tamir Pardo, in December 2012, Netanyahu declared:

On Hanukkah we traditionally say, ‘Who will sing the praises of Israel’s strength?’, and I add to that ‘Who will carry out Israel’s covert operations?’, as it is written: ‘By way of deception, thou shalt conduct war.’ This is the way the few defeat the many, and we have learned that from the days of our forefathers. We need a body that can operate on the international level using both ancient and modern methods. The Mossad does just that in the most outstanding way.²³

Mossad’s use of deception thus claims biblical origins. According to Exodus, the Israelites’ escape from captivity in Egypt began with a divinely authorized deception. Moses asked Pharaoh to permit the Israelites to go into the wilderness on a three-day religious pilgrimage, not revealing that this was intended to be the beginning of a permanent exodus.²⁴ Among those Jewish commentators who later concluded that Moses was acting ‘by way of deception’ is Rabbi Shlomo Yitzhaki (1040–1104), better known as Rashi, whose commentary on the Talmud has been included in every published edition since it was first printed in the 1520s.

Though the Israelites may have been unique in claiming divine authority for their intelligence operations in Canaan over 3,000 years ago, the ancient Egyptians were also heavily involved in intelligence collection. Despite the fact that the Bible makes no reference to an Egyptian presence in Canaan (much as surviving Egyptian sources make only a solitary reference to an Israelite presence), its city-states were ruled by vassals of the Pharaoh with a strong Egyptian military and administrative presence.²⁵ Canaan was the land bridge between Egypt and the four other ‘Great Kingdoms’ in the ancient Near East with which it had diplomatic relations: Hatti, Mittani, Assyria and Babylonia.²⁶ It was also a vital source of supplies for Egypt’s army and navy, and therefore a priority for Egyptian intelligence collection.

The earliest reliable, though fragmentary, information on Egyptian intelligence collection in Canaan – which is also the earliest major source on intelligence operations anywhere in the world – comes from the ‘Amarna letters’, written on clay tablets in the mid-fourteenth century BC. These were accidentally discovered in 1887 by an Egyptian peasant woman at Tell el-Amarna on the site of the palace of Pharaoh Amenhotep IV (better known as Akhenaten). A century of research by scholars on both sides of the Atlantic culminated in the publication in 1992 of an English translation of the letters by the Harvard Egyptologist Professor W. L. Moran.²⁷ A total of 329 of the Amarna letters in the Moran edition were sent by vassals in Canaan to the Pharaoh about a century earlier than the Israelite conquest of Canaan recorded in the Bible. Thirty-eight of the vassals’ letters contain what would nowadays be regarded as intelligence rather than open-source information, though at the time no clear distinction was made between the two.²⁸ Ports on the Levant coast and caravanserai along trade routes also obtained news from far-flung parts of the Mediterranean and the Fertile Crescent. Through Canaan and its harbours, there passed troops, traders and messengers, both friendly and hostile.²⁹ Egypt’s vassals were well placed to gather information from them and knew that they were expected to do so. One vassal wrote to the Pharaoh, ‘I am your loyal servant, and whatever I hear I write to my Lord.’³⁰ Much (almost certainly more) information was also passed on by word of mouth to local Egyptian officials.³¹

The highest-priority intelligence reported to the Pharaoh concerned threats to the Egyptian Empire from external enemies and internal traitors. On one occasion the Pharaoh threatened the ruler of Amurru (north of Canaan), Aziru, with decapitation by axe after receiving intelligence that he had been consorting with one of his enemies: ‘Now the king has heard as follows: You are at peace with the ruler of Qidša. The two of you take food and strong drink together. And it is true. Why do you act so? Why are you at peace with a ruler whom the king is fighting?’ The source of this unwelcome intelligence must have been a non-Egyptian informant at the court of either Aziru or the ruler of Qidša, since an Egyptian would scarcely have been invited to witness Aziru eating and drinking with an enemy of the Pharaoh.³² On another occasion, Abi-Milku, ruler of Tyre, reported that the ruler of Sidon, who claimed to be an exemplary vassal of the Pharaoh, had transferred his loyalty to Aziru: ‘Zimredda, the king of Sidon, writes daily to the rebel Aziru . . . about every word he has heard from Egypt. I herewith write to my lord [Pharaoh], and it is good that he knows.’³³ The Amarna letters also contain the earliest, very fragmentary historical evidence of obtaining intelligence from message interception,* which, when combined with codebreaking,† later became the most important form of intelligence collection in the world wars of the twentieth century.

One intercepted Amarna letter, when compared with Hittite cuneiform records of the same period, reveals an extraordinary sequel to the death in 1323 BC of the Pharaoh Tutankhamun. Though he died aged only about nineteen, after less than a decade on the throne, the sensational discovery in 1923 in the Valley of the Kings of his almost intact tomb and sarcophagus – the most complete of all ancient Egyptian royal tombs – has turned Tutankhamun into the best known of all the pharaohs. The most celebrated exhibit on display today in the priceless collection of the Cairo Museum of Egyptian Antiquities is Tutankhamun’s death mask, made of eleven kilograms of solid gold. Other treasures from the tomb show Queen Ankhesenamen, who was probably his half-sister, at his side. A carving on a golden throne depicts her in a short Nubian wig anointing her husband. Scenes depicted on a golden shrine show her offering Tutankhamun lotus flowers and standing behind him in a skiff pointing at a duck’s nest in a marsh as he hunts ducks with bow and arrow. Recent research has revealed that, immediately after Tutankhamun’s death, Queen Ankhesenamun wrote to the Hittite King Suppiluliuma I, saying that she had no son and heir, and intended to remarry, but refused to marry an Egyptian without royal blood: ‘They say you have many sons. So give me one of your sons. To me he will be a husband, but in Egypt he will be king.’³⁴

Suppiluliuma, who seems to have been taken aback by Ankhesenamun’s urgent, unexpected request to marry one of his sons, probably suspected some kind of provocation, and sent an envoy to ensure that she was in earnest. Ankhesenamun then despatched a second message reproaching the Hittite king for doubting her sincerity and again appealed to him to allow her to marry one of his sons. This time her message, which revealed her original approach to Suppiluliuma, was intercepted; part of it survives among the Amarna letters as well as in Hittite records.³⁵ Though Suppiluliuma seems to have been reassured that Ankhesenamun was in earnest and sent one of his sons to marry her, he was assassinated during his journey before he reached the Egyptian court. Who was responsible for the death of the Hittite prince will probably never be known with certainty. The most probable culprit, however, was Tutankhamun’s elderly Vizier, Ay, who was plotting to succeed the dead Pharaoh. It was probably to avoid marriage to Ay that Ankhesenamun had tried to marry a Hittite prince. Ay, however, succeeded Tutankhamun as Pharaoh and married his widow – despite having discovered her secret plan to marry a Hittite prince instead. Though there is no proof, he may well have taken revenge on his new wife. Ankhesenamun died from unknown causes about a year after her marriage to Ay.

The Amarna letters, as well as other Egyptian records and recent archaeological discoveries in Canaan, are impossible to reconcile

Reviews for The Secret World

[eyejaybee · Rating: 4 out of 5 stars]

Christopher Andrew’s history of the intelligence world is a huge achievement. In fact, I feel that even to have read his book is quite a feat of endurance, as it weighs in at 941 pages of very small type, densely packed with a wealth of insight into the history of the methodology and application of intelligence work. It was certainly fascinating, although at times I found the depth of detail became slightly overwhelming, and every couple of chapters or so, I broke off to read something lighter instead.Living in the twenty-first century we are familiar with the proliferation of separate bodies charged either with intelligence gathering or with undertaking the prevention of such work by similar agencies from different countries. It is, however, only relatively recently that the British Government openly acknowledged the existence of MI5 and MI6. The work of these organisations, traditionally shrouded in mystery, has become a staple in contemporary fiction, whether through the glamorous, if fatuous, adventures of James Bond or the shady (and often shabby) netherworld populated by the character of John le Carré. The world of intelligence and counter-intelligence has always fascinated me. Among the most intriguing aspects are the extent to which the various campaigns and operations have been documented, and the extent to which such records have been made available. The United Kingdom tends only to release official documentation at a remove of thirty years or so (and often much longer depending upon the sensitivity of the papers in question). Andrew has studied these archives, not just in Britain and America (whose Freedom of Information legislation has always allowed for rather greater, and earlier, access to these documentary treasure troves), but also those that have come to light in Russia and Eastern Europe following the dismantling of the Warsaw Pact. His analysis of the records now laid bare is incisive, and helps the reader draw wholly new perspectives on the East-West struggles of the Cold War.Yet the intelligence and counter-intelligence campaigns of the twentieth and twenty-first centuries take up very little of the book. The opening chapters take us back to ancient Egypt, and the plight of the wandering Israelites, whom the Old Testament describes as sending spies into the land of Canaan, while others detail the intricate intelligence systems devised by rulers in India and China two thousand years ago.The value of robust intelligence might now seem beyond question, but there have been significant fluctuations in the value placed upon it by governing regimes. In England, the intelligence community perhaps reached an apotheosis during the reign of Queen Elizabeth I, when networks managed by Lord Willoughby, and then Robert Cecil, helped foil conspiracies supporting the claims of Mary, Queen of Scots, (culminating in her execution in 1587), and to withstand the threat of invasion from Spain: Francis Drake’s attack on Cadiz, when he ‘singed the King of Spain’s beard’ by destroying much of his naval force, was no chance attack but an intelligence-led operation. Less than twenty years later, however, government spies had no idea of the Gunpowder Plot hatched against Elizabeth’s successor (and son of Mary, Queen of Scots), James (I and VI). That Plot was foiled solely as a consequence of an injudicious communication from one of the plotters, warning his brother-in-law to stay away from Parliament on the day of the State Opening. Even in the run up to the First World War, Britain’s intelligence networks were less extensive and effective than their counterparts from a century earlier, during the Napoleonic Wars.Christopher Andrew documents all this with great clarity, aided by some sardonic footnotes. It is a comprehensive, yet comprehensible, work, and deserves a far wider readership than I fear it will receive.