Our initial attempts on Sunday to examine what Wikileaks is calling "Cablegate" were unsuccessful. -- On Monday evening, however, we did succeed, though the link to "download full site in single archive" (what this means is unclear) was not functional. -- In fact, much appears to be uncertain about the present status and future availability of the documents. -- "The embassy cables will be released in stages over the next few months," Wikileaks says. -- "The subject matter of these cables is of such importance, and the geographical spread so broad, that to do otherwise would not do this material justice." -- However, what is meant by "stages" is not explained. -- Only 278 of the 251,287 documents have been "released," 218 on Sunday and another 60 on Monday. -- At the current rate (139 documents per day), it would take 1808 days, or about five years, to release all the documents. -- Readers are invited to "[p]ick out interesting events and tell others about them. Use twitter, reddit, mail whatever suits your audience best," and to "use the #cablegate or unique reference ID (e.g. #66BUENOSAIRES2481) as hash tags." -- The London Guardian has a page encouraging readers to download "basic details of every cable" (but not the "actual content") from its own servers, in various formats. -- The New York Times appears guilty of disingenuousness in its presentation of selected cables. -- For example, in posting a revealing cable about Libya's Muammar Qadhafi, the Times says it has removed "[a] small number of names and passages in some of the cables, to keep from compromising American intelligence efforts or to protect the privacy of ordinary citizens." -- But comparison with the cable as posted by Wikileaks reveals precisely the same excisions. -- The same is true for the famous Russian wedding cable, and, indeed, for every other cable we have examined. -- This suggests that (1) Wikileaks is being much more careful in its release of data than it is being given credit for (no surprise there), and/or that (2) the New York Times is being dishonest about its own handling of the material. -- Neither possibility reflects well on the Times. -- By way of justifying the leak, Wikileaks said its ongoing action "reveals the contradictions between the U.S.’s public persona and what it says behind closed doors -- and shows that if citizens in a democracy want their governments to reflect their wishes, they should ask to see what’s going on behind the scenes." -- Der Spiegel provided a helpful chart explaining some technical aspects of the cables. -- COMMENT: The documents appear to be massively confirming I.F. Stone's dictum: "All governments lie" (full quote: "All governments lie, but disaster lies in wait for countries whose officials smoke the same hashish they give out" [I.F. Stone's Weekly, Aug. 3, 1967]) -- We can expect American commentators to tell us that this is nothing new. -- But of course there is a difference to "knowing" something to a moral certainty and possessing probative evidence of the same thing. -- Whether it will make much difference politically depends on whether enough Americans still care what is being done in their name (but don't expect to encounter this idea very often in mainstream media in the U.S., or indeed anywhere)....
SECRET U.S. EMBASSY CABLES
November 28, 2010
Wikileaks began on Sunday, November 28th, publishing 251,287 leaked United States embassy cables, the largest set of confidential documents ever to be released into the public domain. The documents will give people around the world an unprecedented insight into US Government foreign activities.
The cables, which date from 1966 up until the end of February this year, contain confidential communications between 274 embassies in countries throughout the world and the State Department in Washington, D.C. 15,652 of the cables are classified Secret.
The embassy cables will be released in stages over the next few months. The subject matter of these cables is of such importance, and the geographical spread so broad, that to do otherwise would not do this material justice.
The cables show the extent of U.S. spying on its allies and the U.N.; turning a blind eye to corruption and human rights abuse in "client states"; backroom deals with supposedly neutral countries; lobbying for U.S. corporations; and the measures U.S. diplomats take to advance those who have access to them.
This document release reveals the contradictions between the U.S.’s public persona and what it says behind closed doors -- and shows that if citizens in a democracy want their governments to reflect their wishes, they should ask to see what’s going on behind the scenes.
Every American schoolchild is taught that George Washington – the country’s first President – could not tell a lie. If the administrations of his successors lived up to the same principle, today’s document flood would be a mere embarrassment. Instead, the US Government has been warning governments -- even the most corrupt -- around the world about the coming leaks and is bracing itself for the exposures.
The full set consists of 251,287 documents, comprising 261,276,536 words (seven times the size of "The Iraq War Logs", the world's previously largest classified information release).
The cables cover from 28th December 1966 to 28th February 2010 and originate from 274 embassies, consulates and diplomatic missions.
Groups to contact for comment
How to explore the data
Search for events that you remember that happened for example in your country. You can browse by date or search for an origin near you.
Pick out interesting events and tell others about them. Use twitter, reddit, mail whatever suits your audience best.
For twitter or other social networking services please use the #cablegate or unique reference ID (e.g. #66BUENOSAIRES2481) as hash tags.
* 15, 652 secret
* 101,748 confidential
* 133,887 unclassified
* Iraq most discussed country – 15,365 (Cables coming from Iraq – 6,677)
* Ankara, Turkey had most cables coming from it – 7,918
* From Secretary of State office - 8,017
According to the US State Departments labeling system, the most frequent subjects discussed are:
* External political relations – 145,451
* Internal government affairs – 122,896
* Human rights – 55,211
* Economic Conditions – 49,044
* Terrorists and terrorism – 28,801
* UN security council – 6,532
Graphics of the cablegate dataset
# Cables by origin and classification
# Cables by Subject
# Cables by Country
# Cables by Organization
# Cables by Program
# Cables by Topic
Data Blog -- Facts are sacred
WIKILEAKS EMBASSY CABLES: DOWNLOAD THE KEY DATA AND SEE HOW IT BREAKS DOWN
By Simon Rogers
** The WikiLeaks embassy cables release has produced a lot of stories but does it produce any useful data? We explain what it includes and how it breaks down - plus you can download the key data for every cable **
November 28, 2010
• 21:35pm, 28.11.2010: We've just added a CSV download -- see below. Remember this is the date, time, sender, and tags for each cable -- NOT the text of the cable itself
WikiLeaks embassy cables revelations cover a huge dataset of official documents: 251,287 dispatches, from more than 250 worldwide U.S. embassies and consulates. It's a unique picture of U.S. diplomatic language -- including over 50,000 documents covering the current Obama administration. But what does the data include?
The cables themselves come via the huge Secret Internet Protocol Router Network, or SIPRNet. SIPRNet is the worldwide U.S. military internet system, kept separate from the ordinary civilian internet and run by the Department of Defense in Washington. Since the attacks of September 2001, there has been a move in the U.S. to link up archives of government information, in the hope that key intelligence no longer gets trapped in information silos or "stovepipes." An increasing number of U.S. embassies have become linked to SIPRNet over the past decade, so that military and diplomatic information can be shared. By 2002, 125 embassies were on SIPRNet: by 2005, the number had risen to 180, and by now the vast majority of U.S. missions worldwide are linked to the system -- which is why the bulk of these cables are from 2008 and 2009.
An embassy dispatch marked SIPDIS is automatically downloaded on to its embassy classified website. From there, it can be accessed not only by anyone in the State Department, but also by anyone in the U.S. military who has a security clearance up to the 'Secret' level, a password, and a computer connected to SIPRNet -- which astonishingly covers over 3m people. There are several layers of data in here -- ranging up to the "SECRET NOFORN" level, which means that they are designed never be shown to non-U.S. citizens. Instead, they are supposed to be read by officials in Washington up to the level of current Secretary of State Hillary Clinton. The cables are normally drafted by the local ambassador or subordinates. The "Top Secret" and above foreign intelligence documents cannot be accessed from SIPRNet.
We've broken down the data for you -- and you can download the basic details of every cable (without the actual content) below. Each cable is essentially very structured data. This is what's included:
• A source, ie the embassy or body which sent it
• There is a list of recipients -- normally cables were sent to a number of other embassies and bodies
• There is a subject field -- basically a summary of the cable
• Tags -- each cable was tagged with a number of keyword abbreviations. We've put together a downloadable Google glossary spreadsheet of most of the important ones here
• Body text -- the cable itself. We have opted not to publish these in full for obvious security reasons
Thanks to *Guardian* developer Daithi Ó Crualaoich we've performed some analysis of the data -- which you can download for yourself below. The key points are:
• 251,287 dispatches
• The State Department sent the most cables in this set, followed by Ankara in Turkey, then Baghdad and Tokyo
• 97,070 of the documents were classified as 'Confidential'
• 28,760 of them were given the tag 'PTER' which stands for prevention of terrorism
• The earliest of the cables is from 1966 -- with most, 56,813, from 2009
What can you do with the data?
Download the data
• DATA: every cable with date, time and tags, EXCLUDING BODY TEXT (via Google fusion tables, subject to heavy traffic)
• DATA: every cable with date, time and tags, EXCLUDING BODY TEXT (Zipped CSV file, 3.1MB)
• DATA: our analysis of the cable by location and tag
• DATA: glossary of keywords and tags
World government data
• Search the world's government with our gateway
Development and aid data
• Search the world's global development data with our gateway
Can you do something with this data?
• Flickr Please post your visualisations and mash-ups on our Flickr group
• Get the A-Z of data
• More at the Datastore directory
• Follow us on Twitter