first commit

This commit is contained in:
libroot 2026-11-19 04:39:11 +02:00
commit 0d90363116
Signed by: libroot
GPG key ID: 2AC68C029EBE9ACE
6510 changed files with 679146 additions and 0 deletions

View file

@ -0,0 +1,92 @@
TOP SECRET STRAP1 COMINT
The maximum classification allowed on GCWiki is TOP SECRET STRAP1 COMINT. Click to report inappropriate content.
For GCWiki help contact: webteam Support page
Data stored in BLACK HOLE
From GCWiki
Jump to: navigation, search
This page describes the data which is stored in BLACK HOLE, where it comes from and how much of it there is. Some of the information is too highly classified to be contained here, or falls under the Applied Research intellectual property restrictions. Nonetheless, as much as possible is published here.
BLACK HOLE: The Knowledge
* What it is
* Where it came from
* What is in it
* What data it contains
* How much data there is
* How to use it
* How to connect to it
* How to find the file you want
* Who uses it, and why
* Applied Reseach uses
* SIGINT Development uses
* IPT uses
* How it is built
* Logical connectivity
* Physical system
* Security issues
* What could happen to it
* Applied Research enhancements
* SIGINT Development intentions
* Engineering interest
* What we like about it
* Good things
* Things that could be better
* Exploding myths
* Wish list
v · d · e
Contents
* 1 What data BLACK HOLE contains
* 1.1 Types of data
* 1.2 Classification of the data
* 2 Where the data in BLACK HOLE comes from
* 3 How much data there is
* 3.1 Volumes in BLACK HOLE
* 3.2 Volumes in the datamarts
[edit] What data BLACK HOLE contains
[edit] Types of data
The events created cover webmail, email transfers, ftp, chat, internet browsing, website logins, vbulletin web fora, web cams, gaming, social networking -- and the list is growing.
The data is primarily generated by B13B high-rate bearer processing modules, and so the exact types depend on which modules are running at the time.
[edit] Classification of the data
The classification of the data in BLACK HOLE ranges from SECRET STRAP1 to TOP SECRET STRAP2 UK EYES ONLY. Most data should be covered by TOP SECRET STRAP2 and there may, inadvertently, be ECI data (but this is not expected, as we are not actively doing things covered by ECI). The data includes fields which have been produced by enrichment (such as, for example, target details) and this can increase the classification.
[edit] Where the data in BLACK HOLE comes from
Take a look under the covers of BLACK HOLE to find out where the data comes from.
[edit] How much data there is
[edit] Volumes in BLACK HOLE
The total amount of (compressed) data entering BLACK HOLE is approximately 200GB/day.
[edit] Volumes in the datamarts
| Prototype | Current retention time / months | Current database size / TB | Estimated size for six months retention / TB | Current database size / billion rows | Estimated database size for 6 months retention / billion rows |
| :----- | :---------- | :------------ | :------------- | :---------------- | :---------------- |
| MUTANT BROTH | 4 | 7.7 | 11.55 | 21.7 | 32.55 |
| KARMA POLICE | 3 | 6.8 | 13.6 | 17.8 | 35.6 |
| MEMORY HOLE | 0.5 | 0.6 | 7.2 | 2.2 | 26.4 |
| AUTOASSOC | 6 | 0.1 | 0.1 | 11 | 11 |
| SOCIAL ANIMAL | 1 | 1.6 | 9.6 | 1 | 6 |
| HRMAP | 3 | 7 | 14 | 16 | 32 |
| INFINITE MONKEYS | 6 | 0.02 | 0.02 | 0.2 | 0.2 |
| Total | | 23.82 | 56.07 | 69.9 | 143.75 |
Please note - all of these values are approximate.
POC: [REDACTED] (mail Phonebook Find)
Retrieved from [REDACTED] Data stored in BLACK HOLE"
Category: ICTR BLACK HOLE

View file

@ -0,0 +1,68 @@
{
"title": "Data Stored in BLACK HOLE",
"type": "REPORTED",
"description": "This document describes BLACK HOLE, a TOP SECRET data repository that stores approximately 200GB per day of internet activity \"events\" such as webmail, chat, and browsing. It details the system's various datamarts, including MUTANT BROTH and KARMA POLICE, which collectively hold terabytes of data for intelligence analysis.",
"long_description": "BLACK HOLE is a TOP SECRET STRAP1 COMINT data storage system that houses a vast collection of internet communication events. The data, primarily generated by B13B high-rate bearer processing modules, covers a wide range of activities including webmail, chat, FTP, internet browsing, website logins, social networking, and more. The classification of the information stored ranges from SECRET STRAP1 to TOP SECRET STRAP2 UK EYES ONLY, and its sensitivity can be increased by data enrichment processes, such as the addition of target details.\n\nThe system ingests approximately 200GB of compressed data daily. This data feeds into a series of prototype datamarts with codenames like MUTANT BROTH, KARMA POLICE, and MEMORY HOLE. A table illustrates the scale of these databases, showing a current collective size of nearly 24 TB, with projections to grow to over 56 TB to accommodate a six-month retention period. The intended users of this massive data store include internal groups such as Applied Research and SIGINT Development for analysis and exploitation.",
"tags": [
"data repository",
"surveillance infrastructure",
"datamart",
"mass surveillance",
"internet surveillance",
"bearers",
"data retention",
"metadata repositories",
"content database",
"data volume",
"webmail",
"chat",
"FTP",
"browsing history",
"social networks",
"data enrichment",
"Applied Research",
"SIGDEV",
"ICTR"
],
"document": {
"creator": "GCHQ",
"date_created": "",
"classification": [
"TOP SECRET",
"STRAP 1",
"COMINT"
],
"dissemination": [],
"banner": "TOP SECRET STRAP1 COMINT",
"genre": "wiki",
"programs": [
"BLACKHOLE",
"B13B",
"MUTANTBROTH",
"KARMAPOLICE",
"MEMORYHOLE",
"AUTOASSOC",
"SOCIALANIMAL",
"HRMAP",
"INFINITEMONKEYS"
],
"countries": [],
"people": []
},
"publication": [
{
"publisher": "The Intercept",
"date_published": "2015-09-25",
"title": "Profiled: From Radio to Porn, British Spies Track Web Users' Online Identities",
"link": "https://theintercept.com/2015/09/25/gchq-radio-porn-spies-track-web-users-online-identities/",
"reporters": [
"Gallagher, Ryan"
],
"contributors": [],
"doc_source": "https://theintercept.com/document/2015/09/25/data-stored-black-hole/",
"doc_url": "https://s3.documentcloud.org/documents/2432275/data-stored-in-black-hole.pdf"
}
],
"pdf_sha256": "1065755fb59ae4e88aa10d724865a9d1d045d085752417dfd78bc092ad45b5a6",
"pages": 2
}