{ "title": "Data Stored in BLACK HOLE", "type": "REPORTED", "description": "This document describes BLACK HOLE, a TOP SECRET data repository that stores approximately 200GB per day of internet activity \"events\" such as webmail, chat, and browsing. It details the system's various datamarts, including MUTANT BROTH and KARMA POLICE, which collectively hold terabytes of data for intelligence analysis.", "long_description": "BLACK HOLE is a TOP SECRET STRAP1 COMINT data storage system that houses a vast collection of internet communication events. The data, primarily generated by B13B high-rate bearer processing modules, covers a wide range of activities including webmail, chat, FTP, internet browsing, website logins, social networking, and more. The classification of the information stored ranges from SECRET STRAP1 to TOP SECRET STRAP2 UK EYES ONLY, and its sensitivity can be increased by data enrichment processes, such as the addition of target details.\n\nThe system ingests approximately 200GB of compressed data daily. This data feeds into a series of prototype datamarts with codenames like MUTANT BROTH, KARMA POLICE, and MEMORY HOLE. A table illustrates the scale of these databases, showing a current collective size of nearly 24 TB, with projections to grow to over 56 TB to accommodate a six-month retention period. The intended users of this massive data store include internal groups such as Applied Research and SIGINT Development for analysis and exploitation.", "tags": [ "data repository", "surveillance infrastructure", "datamart", "mass surveillance", "internet surveillance", "bearers", "data retention", "metadata repositories", "content database", "data volume", "webmail", "chat", "FTP", "browsing history", "social networks", "data enrichment", "Applied Research", "SIGDEV", "ICTR" ], "document": { "creator": "GCHQ", "date_created": "", "classification": [ "TOP SECRET", "STRAP 1", "COMINT" ], "dissemination": [], "banner": "TOP SECRET STRAP1 COMINT", "genre": "wiki", "programs": [ "BLACKHOLE", "B13B", "MUTANTBROTH", "KARMAPOLICE", "MEMORYHOLE", "AUTOASSOC", "SOCIALANIMAL", "HRMAP", "INFINITEMONKEYS" ], "countries": [], "people": [] }, "publication": [ { "publisher": "The Intercept", "date_published": "2015-09-25", "title": "Profiled: From Radio to Porn, British Spies Track Web Users' Online Identities", "link": "https://theintercept.com/2015/09/25/gchq-radio-porn-spies-track-web-users-online-identities/", "reporters": [ "Gallagher, Ryan" ], "contributors": [], "doc_source": "https://theintercept.com/document/2015/09/25/data-stored-black-hole/", "doc_url": "https://s3.documentcloud.org/documents/2432275/data-stored-in-black-hole.pdf" } ], "pdf_sha256": "1065755fb59ae4e88aa10d724865a9d1d045d085752417dfd78bc092ad45b5a6", "pages": 2 }