first commit
This commit is contained in:
commit
0d90363116
6510 changed files with 679146 additions and 0 deletions
52
data/spiegel_-_media-35547/metadata.json
Normal file
52
data/spiegel_-_media-35547/metadata.json
Normal file
|
|
@ -0,0 +1,52 @@
|
|||
{
|
||||
"title": "Crypt Ops Data Store User Guide",
|
||||
"type": "REPORTED",
|
||||
"description": "A SECRET GCHQ user guide for the Crypt Ops Data Store — the internal task management system used by GCHQ's cryptanalytic operations division to receive, triage, assign and track decryption and cryptanalytic requests — notable for its category list which maps task types to operational programmes including BULLRUN, BEARDED PIGGY and VPN exploitation.",
|
||||
"long_description": "This is a SECRET GCHQ user guide for the Crypt Ops Data Store. The document describes the workflow for submitting, processing, assigning and completing cryptanalytic tasks within GCHQ's Crypt Ops division, providing a procedural window into how GCHQ managed its decryption workload operationally.\n\nThe task submission process is straightforward: customers submit requests through a web interface, completing mandatory fields including required by date, team requesting, business case, JIC priority points, HRA justification and expected outcome. Tasks move through a status pipeline from New Task through Assigned, In Progress and Complete to Archive, with a weekly Data Store management review ensuring timelines are maintained.\n\nThe most significant content is the category list, which maps two-letter or three-letter category codes to operational areas. The categories visible in the document are: BR (BULLRUN), CTSC, EA (Electronic Attack), GT (Global Targets), HCA (Hand Crypt and SMS), MENA (Middle East North Africa), MOB (Mobile Infrastructure), PRS (Password Recovery Service), TD (Generic Target Discovery), TR (Technical Research), VPN (Virtual Private Networks) and SIG (Generic or complex signature development) — with the full program names for most categories redacted. Two additional categories were under consideration at time of writing: TTT for Tech Tracking and Trends, and BP for requests involving BEARDED PIGGY accounts. BEARDED PIGGY is identified from the GCHQ codewords list as a database used for discovering Virtual Private Networks.\n\nThe category list is revealing because it shows the operational structure of GCHQ's cryptanalytic tasking in a single table — BULLRUN sitting alongside VPN exploitation, mobile infrastructure, password recovery, electronic attack and Middle East/North Africa targeting as parallel operational tracks within the same task management system. The note that SIG possibly includes XKS fingerprinting and TDI/BEGAL in the future confirms the integration between GCHQ's cryptanalytic operations and XKEYSCORE targeting infrastructure was being planned at time of writing.\n\nThe search fields visible in the Data Store interface add further context: tasks carry JIC priority points and JIC Points fields — Joint Intelligence Committee priority rankings — confirming that GCHQ's cryptanalytic tasking was formally integrated into the UK national intelligence priorities framework, with decryption requests ranked by their relationship to JIC collection requirements.\n\nIn plain terms this document is the instruction manual for an internal GCHQ ticketing system — the equivalent of a helpdesk request portal, but for decryption jobs. When an analyst or intelligence team needed GCHQ's cryptanalysts to break into an encrypted communication, crack a password, exploit a VPN connection or carry out some other cryptanalytic task, they would submit a request through this system. The system would route the request to the appropriate team, track it through to completion and notify the requester when done. What makes the document significant is not the workflow itself but the category list — it provides a rare direct view of the full range of cryptanalytic work GCHQ was managing as routine operational business in a single table, showing that breaking encryption for counter-terrorism, targeting Middle Eastern communications, recovering passwords, exploiting VPNs and executing BULLRUN-related capabilities were all handled through the same task queue, triaged by the same admin team, and prioritised using the same Joint Intelligence Committee ranking system that governs all UK intelligence collection.",
|
||||
"tags": [
|
||||
"cryptanalysis",
|
||||
"operational workflow",
|
||||
"VPN exploitation",
|
||||
"VPN decryption",
|
||||
"target discovery"
|
||||
],
|
||||
"document": {
|
||||
"creator": "GCHQ",
|
||||
"date_created": "",
|
||||
"classification": [
|
||||
"SECRET"
|
||||
],
|
||||
"dissemination": [],
|
||||
"banner": "SECRET",
|
||||
"genre": "guide",
|
||||
"programs": [
|
||||
"BULLRUN",
|
||||
"XKEYSCORE",
|
||||
"BEGAL",
|
||||
"BEARDEDPIGGY"
|
||||
],
|
||||
"countries": [],
|
||||
"people": []
|
||||
},
|
||||
"publication": [
|
||||
{
|
||||
"publisher": "Der Spiegel",
|
||||
"date_published": "2014-12-28",
|
||||
"title": "Prying Eyes: Inside the NSA's War on Internet Security",
|
||||
"link": "https://www.spiegel.de/international/germany/inside-the-nsa-s-war-on-internet-security-a-1010361.html",
|
||||
"reporters": [
|
||||
"Appelbaum, Jacob",
|
||||
"Gibson, Aaron",
|
||||
"Grothoff, Christian",
|
||||
"Müller-Maguhn, Andy",
|
||||
"Poitras, Laura",
|
||||
"Sontheimer, Michael",
|
||||
"Stöcker, Christian"
|
||||
],
|
||||
"contributors": [],
|
||||
"doc_url": "https://cdn.prod.www.spiegel.de/media/c6614c5b-0001-0014-0000-000000035547/media-35547.pdf"
|
||||
}
|
||||
],
|
||||
"pdf_sha256": "bcee86e47723cbeb878398729099cd47609233548dab28b8cdab2d84933fda02",
|
||||
"pages": 4
|
||||
}
|
||||
139
data/spiegel_-_media-35547/spiegel_-_media-35547.md
Normal file
139
data/spiegel_-_media-35547/spiegel_-_media-35547.md
Normal file
|
|
@ -0,0 +1,139 @@
|
|||
SECRET
|
||||
|
||||
Crypt Ops Data Store User Guide
|
||||
Crypt Ops Datastore [REDACTED]
|
||||
|
||||
DataStore Rota:
|
||||
[REDACTED]
|
||||
|
||||
~Data Store to be reviewed weekly at the DataStore management meeting
|
||||
|
||||
The Tasking Process
|
||||
|
||||
Submitting a Task
|
||||
1. The customer follows the links to the Data Store page and clicks 'Add Record'.
|
||||
2. The fields 'Crypt Task POC', 'Updates' and 'Team Plan Ref should all be left blank, to be filled in later by Crypt Ops.
|
||||
3. All fields marked with an * must be completed.
|
||||
4. The Required by Date should be changed.
|
||||
5. The 'Task Status' should be left as 'New Task' and the 'Record Status' should be left set to 'Draft'.
|
||||
6. Once all necessary fields are completed click 'Submit'.
|
||||
|
||||
Once submitted the task will be dealt with as follows:
|
||||
|
||||
Processing a Task
|
||||
1. The DS (Data Store) Admin team is notified of that a Task has been submitted.
|
||||
2. The 'on duty' Admin reviews the task, assessing what category (see below) it falls into and passes it along to the appropriate TL (team leader).
|
||||
|
||||
Categories
|
||||
|
||||
- BR BULLRUN [REDACTED]
|
||||
- CTSC [REDACTED] Counter Terrorism and Serious Crime
|
||||
- EA [REDACTED] Electronic Attack
|
||||
- GT [REDACTED] Global Targets
|
||||
- HCA [REDACTED] Hand Crypt and SMS
|
||||
- MENA [REDACTED] Middle East North Africa
|
||||
- MOB [REDACTED] Mobile Infrastructure
|
||||
- PRS [REDACTED] Password Recovery Service
|
||||
- TD [REDACTED] Generic Target Discovery
|
||||
- TR [REDACTED] Technical Research
|
||||
- VPN [REDACTED] Virtual Private Networks
|
||||
- SIG [REDACTED] Generic or complex signature development,
|
||||
possibly to include XKS fingerprinting (and TDI/BEGAL in the future?)
|
||||
|
||||
And possibly more, including:
|
||||
|
||||
1 of 4
|
||||
|
||||
This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to GCHQ on [REDACTED]
|
||||
|
||||
SECRET
|
||||
SECRET
|
||||
|
||||
- TTT ?? Tech Tracking and Trends – Requests for tech tracking or trends data
|
||||
- BP ?? Requests BEARDED PIGGY accounts
|
||||
|
||||
3. The DS Admin notifies the TL of the task and updates it changing the POC to the TL's sid and adding a note to the 'Updates' field in the following format:
|
||||
|
||||
'Passed to <TL's sid> on <dd/mm/yy> by <DS Admin's sid>'
|
||||
|
||||
4. DS Admin then sends an email to the task originator notifying them which TL is handling the task.
|
||||
|
||||
5. If the task is inappropriate for Crypt Ops or is one that there will be no resources available for in the foreseeable future then the TL changes the Task Status to 'Rejected' and notifies the task originator explaining why the task has been rejected. The task's Record Status is then set to 'Archive'.
|
||||
|
||||
6. If it is an appropriate achievable task the TL takes the following steps:
|
||||
|
||||
- Determine what priority the task should be
|
||||
- Decide if the timescale is achievable given the available resources and personnel
|
||||
- Decide if the necessary resources are available
|
||||
- If an appropriate person is available assign them
|
||||
|
||||
7a. If resources are not currently available but the task is one that Crypt Ops will definitely complete some time in the near future when resources are available the TL should contact the tasks originator and inform them of when, realistically, resources will be available and work started.
|
||||
|
||||
7b. If they still wish the task to be undertaken then a new timescale should be negotiated.
|
||||
|
||||
7c. The Task Status should then be set to 'Suspended' and the Record Status is left as 'Draft'.
|
||||
|
||||
7. These tasks will be checked as part of the Data Store review to ensure that work starts on them as soon as resources become available.
|
||||
|
||||
9. Once this is done the tasks Record Status is set to 'Current', the Task Status is set to 'Assigned' and the update field should be filled in as follows:
|
||||
10.
|
||||
Work to be carried out by <assignees sid> starting on <dd/mm/yy> assigned by <TL's sid>
|
||||
|
||||
2 of 4
|
||||
|
||||
This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to GCHQ on [REDACTED]
|
||||
|
||||
SECRET
|
||||
SECRET
|
||||
|
||||
10. The TL then informs the task originator of the update.
|
||||
|
||||
11. Once the assignee begins work the Task Status should be set to 'In Progress'.
|
||||
|
||||
12. The DS Admin's review all task weekly to ensure they are being acted on and that all timescales and on schedule. If a timescale is deemed unrealistic then the TL should be informed and should negotiate a new deadline with the task originator.
|
||||
|
||||
13. Once a task is completed the Task Status should be set to 'Complete' and the Record Status changed to 'Archive'.
|
||||
|
||||
14. The task originator should be informed that the task is officially complete and where possible feed back obtained.
|
||||
|
||||
15. IMPORTANT: Remember to keep both the 'Record Status' and 'Task Status' fields up to date and make sure that the record is in the correct category. The NEW TASK status should only be used on new tasks still in DRAFT, once
|
||||
ASSIGNED the task should be moved immediately to CURRENT, once work is started the task status should be IN PROGRESS and once COMPLETE it should be moved to ARCHIVE.
|
||||
|
||||
Searching DataStore (Only applicable to those with view privileges)
|
||||
|
||||
There are two search options at the top of the page; Text Search and Date Search.
|
||||
|
||||
Text Search - this allows searching on a number of fields (listed below) both in the page and within the records themselves. This search type is useful for finding all the tasks owned by a specific individual or all the tasks of a specific type (providing the category tag has been applied to the 'Task Title'.
|
||||
|
||||
Task Title
|
||||
Team Requesting
|
||||
Task Status
|
||||
Crypt Task POC
|
||||
Originator
|
||||
Task Classification
|
||||
JIC Points
|
||||
JIC Priority
|
||||
Miranda
|
||||
HRA Justification
|
||||
Expected Outcome
|
||||
Business Case
|
||||
Estimated Resources
|
||||
Updates
|
||||
Team Plan Ref
|
||||
|
||||
3 of 4
|
||||
|
||||
This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to GCHQ on [REDACTED]
|
||||
|
||||
SECRET
|
||||
SECRET
|
||||
|
||||
Date Search - there are three sub categories of this search; 'No date search' (which searches for record with no date set in the chosen field), 'Search for a single date', or 'Search over a date range'. These searches can be conducted on several fields; 'Required by Date', 'Date Submitted', 'Date last updated' and 'Date Task Completed'. This search is useful for finding all the tasks that need to be completed by a specific date.
|
||||
|
||||
The search types can be used in concert e.g. tasks with SIG in the title with a required by date between 01/05/2011 and 06/10/2011, this is achieved by running one search and then the other.
|
||||
|
||||
4 of 4
|
||||
|
||||
This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to GCHQ on [REDACTED]
|
||||
|
||||
SECRET
|
||||
Loading…
Add table
Add a link
Reference in a new issue