Skip to content

What is CableWeaver

thedod edited this page Mar 21, 2013 · 9 revisions

Looking for needles in haystacks of cables

The United States diplomatic cables leak (known as CableGate) is a collection of over 250,000 diplomatic cables. That's a lot of text, and even though there are several cable search sites, it would probably take many years before we uncover all worthy information hiding in there.

Graph-theory to the rescue

Since cables often refer to other cables (called reference telgrams or REFTELs), one way to analyze CableGate information is by applying graph theory methods: we can look at the entire cable collection as a large graph where the nodes are cables and the vertices are references between them. This graph can be split into highly-connected sub-graphs using clustering techniques.

A project called Cable2Graph by wlwardiary does exactly that: it treats the entire CableGate collection as a large graphs, and provides various graph theory tools to split it into subgraphs, compute how much each cable contributes to the connectivity of the graph, etc.

Making graphs pretty isn't easy

Cable2Graph can also present the resulting graphs visually, but there are 2 problems there:

  1. Finding an optimal visual layout for a graph is a computationally intensive task (which means it would be hard to turn Cable2Graph into a web service because the load on such a server could be prohibitively high).
  2. Finding a "decent" graph layout is [still] something people do better than machines.

CableWeaver - interactively inspecting Cable2Graph results

CableWeaver tries to solve these 2 problems by

  • Computing all sub-graphs in advance (the project contains more than 5000 sub-graph files).
  • Interactively optimizing the layout on the browser (using the force layout feature of the D3 javascript library).

This way we solve both problems mentioned above:

  1. No server resources are spent on layout (in fact - CableWeaver only contains static files).
  2. The user gets to refine the layout in a way only humans can.

Sharing your findings

CableWeaver not only lets you arrange the layout of the graph in a more readable way, it also lets you select a bunch of cables that you believe are related to the same subject (or "story"). A graph where some of the cables are selected is called a story-line.

You can share a story-line with others by clicking on the share button and copying a link that would open CableWeaver with a graph similar to the one you see in front of you (example).

Sharing a story-line this way doesn't preserve graph layout, so it's mainly useful in order to exchange story-lines with fellow cable researchers. If you want to also preserve the layout, you can use the export button. This would generate HTML for a web page that contains both the graph and textual description of all cables in the story-line (example).

You can then edit such a page (with a text or HTML editor) and add quotations from the cables, remarks, summary, etc. (example).

The CableWeaving story-line repository

CableWeaving is a [so far - small] collection of story-line pages exported from CableWeaver. They can serve as background research for journalistic stories (for example - this story-line was used for a Wikileaks Press article).

Hopefully, journalists will find leads in this repository, and researches will add new story-lines and make existing ones better (by adding remarks and cable quotations to them). Hopefully - you are one of these journalists or researchers :)