first commit
This commit is contained in:
commit
0d90363116
6510 changed files with 679146 additions and 0 deletions
44
data/tint-external-july-2009/metadata.json
Normal file
44
data/tint-external-july-2009/metadata.json
Normal file
|
|
@ -0,0 +1,44 @@
|
|||
{
|
||||
"title": "TINT External July 2009",
|
||||
"type": "REPORTED",
|
||||
"description": "A GCHQ TINT external document from July 2009 describing the BLACKHOLE bulk metadata repository — designed to export unselected metadata in SLR format to enable higher-order analytics — noting anticipated very high volumes expected to exceed MAILORDER transport capacity and proposing a direct connection instead.",
|
||||
"long_description": "This is a fragment from a GCHQ TINT external document dated July 2009. The excerpt is a single entry — item 14 — from what appears to be a larger catalogue of data systems or experimental capabilities, describing BLACKHOLE.\n\nBLACKHOLE is defined by purpose as a repository for exporting bulk unselected metadata in SLR format, enabling higher-order analytics to be carried out on the collected data. The distinction between \"bulk, unselected\" and targeted collection is significant — BLACKHOLE explicitly receives data that has not been filtered against specific selectors, meaning it ingests the full firehose of intercepted internet metadata rather than only records matching known targets.\n\nThe data flow is described as to and from GCHQ, with a direct connection proposed. Anticipated volume is classified as \"very high\" — high enough that the standard MAILORDER transport mechanism is expected to be insufficient to handle it. This is consistent with the Karma Police article's description of BLACKHOLE storing over 1.1 trillion events between August 2007 and March 2009, growing to 10 billion new entries per day and eventually 50 billion per day by 2012. The dependent experiments section references \"SLRs at Scale\" — suggesting the SLR format and its scaling properties were an active research area at time of writing.\n\nThe document connects directly to the TDI Introduction presentation processed immediately before, which describes MUTANT BROTH, KARMA POLICE and the other bulk analytics databases as drawing from the BLACKHOLE repository — confirming BLACKHOLE's role as the central raw data store from which all downstream TDI analytics are derived.",
|
||||
"tags": [
|
||||
"TINT",
|
||||
"SLR",
|
||||
"bulk metadata",
|
||||
"unselected collection",
|
||||
"higher-order analytics",
|
||||
"internet surveillance",
|
||||
"metadata repositories"
|
||||
],
|
||||
"document": {
|
||||
"creator": "GCHQ",
|
||||
"date_created": "2009-01-01",
|
||||
"classification": [],
|
||||
"dissemination": [],
|
||||
"genre": "other",
|
||||
"programs": [
|
||||
"BLACKHOLE",
|
||||
"MAILORDER"
|
||||
],
|
||||
"countries": [],
|
||||
"people": []
|
||||
},
|
||||
"publication": [
|
||||
{
|
||||
"publisher": "The Intercept",
|
||||
"date_published": "2015-09-25",
|
||||
"title": "Profiled: From Radio to Porn, British Spies Track Web Users' Online Identities",
|
||||
"link": "https://theintercept.com/2015/09/25/gchq-radio-porn-spies-track-web-users-online-identities/",
|
||||
"reporters": [
|
||||
"Gallagher, Ryan"
|
||||
],
|
||||
"contributors": [],
|
||||
"doc_source": "https://theintercept.com/document/2015/09/25/tint-external-july-2009/",
|
||||
"doc_url": "https://s3.documentcloud.org/documents/2432302/tint-external-july-2009.pdf"
|
||||
}
|
||||
],
|
||||
"pdf_sha256": "e5ff79eb7ab6b7cfbfbcb9255262b3734adffddc58119872b1597840fa76461b",
|
||||
"pages": 1
|
||||
}
|
||||
Loading…
Add table
Add a link
Reference in a new issue