first commit
This commit is contained in:
commit
0d90363116
6510 changed files with 679146 additions and 0 deletions
62
data/utt-config-readme/metadata.json
Normal file
62
data/utt-config-readme/metadata.json
Normal file
|
|
@ -0,0 +1,62 @@
|
|||
{
|
||||
"title": "UTT Configuration READ ME",
|
||||
"type": "REPORTED",
|
||||
"description": "A SECRET XKeyscore configuration manual describing how to set up the Unified Targeting Tool — NSA's centralised system for pushing surveillance selectors from analysts to field site collection systems worldwide — revealing the complete data flow from analyst targeting request through MAILORDER transport to XKeyscore ingestion, including a forwarding domain table showing how intercepted data is routed to all Five Eyes partners (NSA, GCHQ, CSE, DSD, GCSB) and third parties.",
|
||||
"long_description": "This is a SECRET//REL TO USA, FVEY XKeyscore configuration read-me document describing the Unified Targeting Tool. UTT is described as a mission-critical component of the TURBULENCE architecture that transitions NSA from a selector-centric mode of operation — where each type of selector is entered into a different targeting tool — to a target-centric system where all targeting requests for intercept from global and private network communications are entered through a single point of service regardless of selector type or access capability.\n\nThe document details the complete UTT data flow architecture. Analysts enter targeting requests into UTT, which pushes selector information to the Site Selection Distribution Manager at each field site. The SSDM performs site-specific processing and forwards the appropriate subset of selectors via MAILORDER to XKeyscore. When XKeyscore processes intercepted traffic matching a tasked selector, the selected data is forwarded through FISHWAY/SCISSORS to PINWALE for storage, and tips are sent to TRAFFICTHIEF for real-time alerting. The SSDM receives acceptance or rejection status from XKeyscore and returns it to UTT, and can request full reloads of all selectors either automatically or manually. The configuration process involves editing xks.config to enable UTT tasking, setting the SSDM routing trigraph, configuring MAILORDER inputs with Producer Designator Digraph and source system digraph parameters, and setting up forwarding domains. The strong_selector_targeting process writes processed UTT tasking files to strong.selectors in the XKeyscore dictionaries directory.\n\nThe forwarding domains table is the most operationally significant element, revealing the complete Five Eyes data routing architecture within XKeyscore. Twelve forwarding domain identifiers are defined: domains 1-2 for NSA (with domain 2 designated NOFORN using routing trigraph SQP rather than the standard KFC), domains 3-4 for CSE (Canada), domains 5-6 for DSD (Australia), domains 7-8 for GCHQ (UK), domains 9-10 for GCSB (New Zealand), domain 11 for third parties, and domain 12 for local support. Each partner has both a standard and a NOFORN variant, confirming that XKeyscore field sites maintain parallel data forwarding channels segregated by releasability. All non-NOFORN domains use routing trigraph KFC while the NSA NOFORN domain uses SQP — meaning intercepted data matching NOFORN-marked selectors is routed through a separate infrastructure path even within NSA.",
|
||||
"tags": [
|
||||
"forwarding domains",
|
||||
"routing trigraph",
|
||||
"KFC",
|
||||
"SQP",
|
||||
"PDDG",
|
||||
"Producer Designator Digraph",
|
||||
"strong selectors",
|
||||
"selector tasking",
|
||||
"field sites",
|
||||
"data flows",
|
||||
"collection architecture"
|
||||
],
|
||||
"document": {
|
||||
"creator": "NSA",
|
||||
"date_created": "",
|
||||
"classification": [
|
||||
"SECRET"
|
||||
],
|
||||
"dissemination": [
|
||||
"REL TO USA, FVEY"
|
||||
],
|
||||
"banner": "SECRET//REL TO USA, FVEY",
|
||||
"genre": "manual",
|
||||
"programs": [
|
||||
"UTT",
|
||||
"TURBULENCE",
|
||||
"XKEYSCORE",
|
||||
"MAILORDER",
|
||||
"SSDM",
|
||||
"FISHWAY",
|
||||
"SCISSORS",
|
||||
"PINWALE",
|
||||
"TRAFFICTHIEF"
|
||||
],
|
||||
"countries": [],
|
||||
"people": []
|
||||
},
|
||||
"publication": [
|
||||
{
|
||||
"publisher": "The Intercept",
|
||||
"date_published": "2015-07-01",
|
||||
"title": "XKEYSCORE: NSA’s Google for the World’s Private Communications",
|
||||
"link": "https://theintercept.com/2015/07/01/nsas-google-worlds-private-communications/",
|
||||
"reporters": [
|
||||
"Marquis-Boire, Morgan",
|
||||
"Greenwald, Glenn",
|
||||
"Lee, Micah"
|
||||
],
|
||||
"contributors": [],
|
||||
"doc_source": "https://theintercept.com/document/2015/07/01/utt-config-readme/",
|
||||
"doc_url": "https://s3.documentcloud.org/documents/2116258/utt-config-readme.pdf"
|
||||
}
|
||||
],
|
||||
"pdf_sha256": "86f6eaeb04e125aec0343a66def787c76c65b3a54f955111e9526b37ad689696",
|
||||
"pages": 5
|
||||
}
|
||||
142
data/utt-config-readme/utt-config-readme.md
Normal file
142
data/utt-config-readme/utt-config-readme.md
Normal file
|
|
@ -0,0 +1,142 @@
|
|||
SECRET//REL TO USA, FVEY
|
||||
|
||||
(U) UTT Configuration
|
||||
READ ME
|
||||
|
||||
(U) Overview
|
||||
|
||||
(S//SI//REL) The Unified Targeting Tool (UTT) makes it possible to transition from today's selector-centric mode of operation, where each type of selector is entered into a different targeting tool, to a target-centric system. In this system, selection management is converged and then streamlined and automated, thereby facilitating analyst operation and collaboration. UTT, a mission critical component of the TURBULENCE architecture, provides a single point-of-service. Analysts enter all of their targeting requests for intercept from global network communications as well as private networks that are resident on or accessible using the global network infrastructure, regardless of the types of selectors or access capabilities to be employed.
|
||||
|
||||
(U) UTT Dataflow
|
||||
|
||||
(U//FOUO) The UTT pushes selector targeting information to the Site Selection Distribution Manager (SSDM). The SSDM manages selector tasking at the field site. The SSDM receives the load/updates that were pushed from the UTT. It is responsible for any site-specific processing that must be performed before forwarding the appropriate subset of information to site-local collection systems.
|
||||
|
||||
(U//FOUO) Once messages are processed, the SSDM sends selector targeting information via MAILORDER to front-end collection systems at a field site (e.g., XKEYSCORE). The SSDM receives status back from these systems indicating whether or not the selector tasking was accepted or rejected and then returns status to the UTT in the form of load and update responses. The SSDM may also request, either automatically or manually by a local user, that the UTT send a complete reload of all selectors targeted for the site. Similarly, the field site's front-end collection systems may also request, either automatically or manually by a local user, that the SSDM send a complete reload of all selectors targeted for the site.
|
||||
|
||||
1 |
|
||||
|
||||
SECRET//REL TO USA, FVEY
|
||||
|
||||
SECRET//REL TO USA, FVEY
|
||||
|
||||
(U//FOUO) UTT Dataflow
|
||||
|
||||
(C)
|
||||
```
|
||||
UTT
|
||||
|
||||
SSDM
|
||||
|
||||
UTT Tasking
|
||||
|
||||
Response Messages back to SSDM
|
||||
|
||||
XKS Master
|
||||
|
||||
MAILORDER
|
||||
|
||||
Selected Data
|
||||
|
||||
FISHWAY/
|
||||
SCISSORS
|
||||
|
||||
PINWALE
|
||||
|
||||
Tips
|
||||
|
||||
TRAFFIC
|
||||
THIEF
|
||||
|
||||
xks.config
|
||||
Forwarding
|
||||
Domains
|
||||
|
||||
NSAW
|
||||
Field Site
|
||||
```
|
||||
(C)
|
||||
|
||||
(U//FOUO) This Readme file is intended as an aid for setting up the appropriate configurations to load, ingest, and scan data against UTT tasking. It does NOT address forwarding selected traffic to PINWALE. Configuring XKEYSCORE for UTT can be accomplished in three phases. First, you will address three pre-configuration requirements. Next you will configure *xks.config* and *xks.advanced.config* to enable UTT tasking and appropriate MAILORDER routing. Last, you will execute a group of set-up processes to ensure all configuration changes are fully implemented.
|
||||
|
||||
(U) Pre-Configuration Requirements
|
||||
|
||||
(U//FOUO) The Unified Targeting Tool (UTT) works hand-in-hand with the Site Selection Distribution Manager (SSDM). Therefore you must have:
|
||||
* an SSDM in place at the field site.
|
||||
* the SSDM's routing trigraph so XKEYSCORE can send response messages back to the SSDM.
|
||||
* a routing trigraph for the XKEYSCORE Master.
|
||||
|
||||
2 |
|
||||
SECRET//REL TO USA, FVEY
|
||||
|
||||
SECRET//REL TO USA, FVEY
|
||||
|
||||
(U/FOUO) Configuring UTT Selection in *xks.config*
|
||||
|
||||
(U//FOUO) Configuration for UTT selection and tasking can be accomplished in just seven steps:
|
||||
|
||||
1. (U//FOUO) Logon as the user `oper`.
|
||||
2. (U//FOUO) At the command line from within any directory, type `viconfig` and then press Enter. The `xks.config` file will open.
|
||||
3. (U//FOUO) In the `[dictionaries]` section of *xks.config*, set the following configurations:
|
||||
a. `utt = yes`: This enables UTT tasking. The default setting for this configuration is `no`.
|
||||
b. `ssdm_trigraph = [field site trigraph]`: This is the SSDM's trigraph for the field site. The default is `XXX`, which will need to be changed.
|
||||
4. (U//FOUO) In the `[processes]` section of *xks.config* confirm that the MAILORDER configuration reads `mailorder = yes`. The default is `no`. This creates the `mailorder_proc` process.
|
||||
5. (U//FOUO) In the `Mailorder inputs` section of *xks.config*, type `mailorder_input[0] = source:OO:AB,dir:ssdm`. This is the input configuration for `check_mailorder_site.php`. This particular configuration indicates that all incoming files in `$XSCORE_DATA_DIR/inputs/mailorder` that have a Producer Designator Digraph (PDDG) of `OO` and a source system digraph of `AB` will be moved to `$XSCORE_DATA_DIR/inputs/ssdm`.
|
||||
6. (U//FOUO) In the `Plugin Config` section of *xks.config* is the following placemark:
|
||||
`forwarding_domain[] = PLACEHOLDER - DO NOT REMOVE - ADD ENTRIES BELOW`
|
||||
Below this placemark, type your own forwarding domains (i.e., routing trigraphs) like those identified in the `Forwarding Domains` chart. Forwarding Domains are in the form:
|
||||
`forwarding_domain[N] = key:value,key2:value2,...`
|
||||
|
||||
3 |
|
||||
|
||||
SECRET//REL TO USA, FVEY
|
||||
|
||||
SECRET//REL TO USA, FVEY
|
||||
|
||||
(U//FOUO) The `N` is an identifier (`forwarding_domain` parameter) between 1 and 12. One type of key is the `fdi` key. It indicates where to send collected data of the domain. The `value` associated with this key is the routing trigraph of the domain. The *Forwarding Domains* table shown below identifies the forwarding locations associated with each identifier (`forwarding_domain` parameter).
|
||||
|
||||
(S)
|
||||
**Forwarding Domains**
|
||||
|
||||
| Routing Trigraph | Forward Location |
|
||||
| :-------------------------- | :-------------------- |
|
||||
| `forwarding_domain[1] = fdi:KFC` | 1-NSA |
|
||||
| `forwarding_domain[2] = fdi:SQP` | 2-NSA (NOFORN Only) |
|
||||
| `forwarding_domain[3] = fdi:KFC` | 3-CSE |
|
||||
| `forwarding_domain[4] = fdi:KFC` | 4-CSE NOFORN |
|
||||
| `forwarding_domain[5] = fdi:KFC` | 5-DSD |
|
||||
| `forwarding_domain[6] = fdi:KFC` | 6-DSD NOFORN |
|
||||
| `forwarding_domain[7] = fdi:KFC` | 7-GCHQ |
|
||||
| `forwarding_domain[8] = fdi:KFC` | 8-GCHQ NOFORN |
|
||||
| `forwarding_domain[9] = fdi:KFC` | 9-GCSB |
|
||||
| `forwarding_domain[10] = fdi:KFC` | 10-GCSB NOFORN |
|
||||
| `forwarding_domain[11] = fdi:KFC` | 11-3rd Party |
|
||||
| `forwarding_domain[12] = fdi:KFC` | 12-Local Support |
|
||||
|
||||
(S)
|
||||
|
||||
7. (U/FOUO) Type: `wq!` and then press Enter to exit *xks.config*.
|
||||
|
||||
4 |
|
||||
|
||||
SECRET//REL TO USA, FVEY
|
||||
|
||||
SECRET//REL TO USA, FVEY
|
||||
|
||||
(U) Additional Processes
|
||||
|
||||
(U//FOUO) In addition to editing UTT/SSDM configurations in *xks.config* and *xks.advanced.config*, it is important to execute several set-up processes. As the `oper` user, execute the following commands only *after* entering the SSDM configurations in *xks.config*:
|
||||
|
||||
1. (U//FOUO) At the command prompt, type `xks setup processes` and press Enter. This creates the `check_mailorder_site.php` and `mailorder_proc` processes on the Master server.
|
||||
2. (U//FOUO) At the command prompt, type `xks proc start` and press Enter. This will ensure all of the running processes pick up any configuration changes.
|
||||
|
||||
As a result of executing these commands, the following processes will begin:
|
||||
|
||||
* (U//FOUO) The `check_mailorder_site.php` process, which polls `$XSCORE_DATA_DIR/inputs/ssdm` and moves the SSDM MAILORDER file(s) to the `$XSCORE_DATA_DIR/inputs/ssdm` directory, according to the `[mailorder_input]` rule in the *xks.config* file.
|
||||
* Note: If a MAILORDER input rule is changed in *xks.config*, then type `xks proc restart cms` to restart the `check_mailorder_site.php` process.
|
||||
* (U//FOUO) The `mailorder_proc` process, which polls `$XSCORE_DATA_DIR/outputs/mailorder_working` directory and properly renames and moves any MAILORDER files to `$XSCORE_DATA_DIR/outputs/mailorder` for pick-up by MAILORDER.
|
||||
* (U//FOUO) The `strong_selector_targeting` process, which polls `$XSCORE_DATA_DIR/inputs/ssdm` for any tasking updates or full loads. When it receives a message from the SSDM that updates are available, a response receipt is generated and sent to `$XSCORE_DATA_DIR/outputs/mailorder_working`. Next, `strong_selector_targeting` processes the UTT updates/tasking files and writes the processed files to `$XSCORE_DIR/config/dictionaries/selectors/strong.selectors`.
|
||||
* (U//FOUO) Another receipt is generated after processing of an update/full load is complete. The message indicates success or failure and is also placed into `$XSCORE_DATA_DIR/outputs/mailorder_working`.
|
||||
* Note: (U//FOUO) The `strong_selector_targeting` process will request a reload if it receives SIGUSR2, if it starts up and sees `utt.selectors` is still there, or if it starts up and has no targeting.
|
||||
|
||||
5 |
|
||||
SECRET//REL TO USA, FVEY
|
||||
Loading…
Add table
Add a link
Reference in a new issue