first commit
This commit is contained in:
commit
0d90363116
6510 changed files with 679146 additions and 0 deletions
47
data/voip-readme/metadata.json
Normal file
47
data/voip-readme/metadata.json
Normal file
|
|
@ -0,0 +1,47 @@
|
|||
{
|
||||
"title": "VoIP Configuration and Forwarding Read Me",
|
||||
"type": "REPORTED",
|
||||
"description": "A SECRET XKeyscore configuration manual describing how to set up the system to intercept internet phone calls, faxes and video calls and forward them to NUCLEON — NSA's voice intercept repository which at time of writing supported over 8,000 users worldwide across 75 servers receiving 700,000 voice, fax, video and tag files per day.",
|
||||
"long_description": "This is a SECRET//REL TO USA, FVEY XKeyscore VoIP configuration read-me document. The document provides step-by-step instructions for configuring XKeyscore field sites to process intercepted Voice over IP traffic and forward captured calls, faxes and video to NUCLEON, NSA's centralised voice intelligence repository.\n\nThe document opens with operational context: VoIP emerged in the late 1990s as an alternative to traditional TDM telephony for international voice traffic and had grown to account for a large percentage of cross-border calls. NUCLEON is quantified as supporting over 8,000 users worldwide, composed of 75 operational servers receiving 700,000 voice, fax, video and tag files per day — providing one of the few published metrics on the scale of NSA's voice intercept infrastructure.\n\nThe configuration section reveals three VoIP processing modes within XKeyscore: \"site\" for normal local processing, \"overlord\" for cross-cluster pairing at the same site (matching call setup signalling with RTP voice data across multiple XKeyscore instances), and \"cross_site\" for central VoIP server cross-site pairing — matching signalling and voice data collected at different facilities worldwide, consistent with the VoIP in XKeyscore presentation's description of F6 HQS coordinating global queries. Targeting is driven by UTT selectors pushed through the SSDM, with fallback options for CSV file-based tasking and a legacy \"royale file\" method explicitly noted as \"not guaranteed to be completely legal for any site\" and recommended only as a last resort when no site-specific tasking is available.\n\nThe forwarding configuration uses MAILORDER file distribution indicators to route captured voice, fax and video to NUCLEON, with a local routing mechanism based on \"zip codes\" — internal routing prefixes beginning with letters L, J and K that direct intercepted media to different processing destinations. The document confirms that XKeyscore's process_data_parent handles dictionary scanning, metadata extraction, database storage and content archival across master and slave nodes in each cluster.",
|
||||
"tags": [],
|
||||
"document": {
|
||||
"creator": "NSA",
|
||||
"date_created": "",
|
||||
"classification": [
|
||||
"SECRET"
|
||||
],
|
||||
"dissemination": [
|
||||
"REL TO USA, FVEY"
|
||||
],
|
||||
"banner": "SECRET//REL TO USA, FVEY",
|
||||
"genre": "manual",
|
||||
"programs": [
|
||||
"XKEYSCORE",
|
||||
"UTT",
|
||||
"NUCLEON",
|
||||
"MAILORDER",
|
||||
"SSDM",
|
||||
"PATHMASTER"
|
||||
],
|
||||
"countries": [],
|
||||
"people": []
|
||||
},
|
||||
"publication": [
|
||||
{
|
||||
"publisher": "The Intercept",
|
||||
"date_published": "2015-07-01",
|
||||
"title": "XKEYSCORE: NSA’s Google for the World’s Private Communications",
|
||||
"link": "https://theintercept.com/2015/07/01/nsas-google-worlds-private-communications/",
|
||||
"reporters": [
|
||||
"Marquis-Boire, Morgan",
|
||||
"Greenwald, Glenn",
|
||||
"Lee, Micah"
|
||||
],
|
||||
"contributors": [],
|
||||
"doc_source": "https://theintercept.com/document/2015/07/01/voip-readme/",
|
||||
"doc_url": "https://s3.documentcloud.org/documents/2116266/voip-readme.pdf"
|
||||
}
|
||||
],
|
||||
"pdf_sha256": "152bb604a5d895d7cc8d7e379c71c7b52d7c7347eb4935a3b4baf3fb3f19deb5",
|
||||
"pages": 3
|
||||
}
|
||||
75
data/voip-readme/voip-readme.md
Normal file
75
data/voip-readme/voip-readme.md
Normal file
|
|
@ -0,0 +1,75 @@
|
|||
SECRET//REL TO USA, FVEY
|
||||
|
||||
(U) VoIP Configuration and Forwarding Read Me
|
||||
|
||||
(U) Overview
|
||||
|
||||
(U//FOUO) In the late 1990s, voice over IP (VoIP) emerged as an alternative to the Time Division Multiplexed (TDM) technology for transporting international voice traffic. Since then, VoIP has developed into a widely used technology that accounts for a large percentage of cross-border voice traffic.
|
||||
|
||||
(S//SI//REL) NUCLEON is NSAW's signals intelligence voice, facsimile, video and pre-released transcription repository. It supports over 8000 users worldwide and is composed of 75 operational servers receiving 700,000 voice, fax, video, and tag files per day. This *Read Me* provides procedures for configuring XKEYSCORE to generate and forward VoIP hits to NUCLEON.
|
||||
|
||||
(U) Pre-Configuration Requirements
|
||||
|
||||
(C//REL) Before configuring `xks.config` for VoIP processing you must:
|
||||
|
||||
1. Confirm UTT tasking is in place for the UTT selectors to be used. Please refer to the *UTT Configuration Read Me* to ensure UTT tasking has been set up.
|
||||
|
||||
2. Confirm MAILORDER is configured to pick up MAILORDER files from the `$XSCORE_DATA_DIR/outputs/mailorder` directory on the Master server. To do this, create a MAILORDER ticket to set up the pick-up directory for your Master XKS server:
|
||||
a. Type `go mailorder` in the URL field of a browser on a high-side computer.
|
||||
b. Click `PATHMASTER Remedy Ticket` near the top right of the screen. The *Remedy Ticket Submission* screen will appear.
|
||||
c. In the bottom half of the *Remedy Ticket Submission* screen is two yellow *Submit Remedy Ticket* buttons. Click the top button of the two. (This ticket pertains to New Data-Flow). The ITSC login screen will appear.
|
||||
d. Enter the SID of your alternate POC and then click `Continue`. A screen containing contact information for you and your POC will appear.
|
||||
e. Add/Edit the contact information as appropriate and then click `Continue` to go to the *Data transport service* screen.
|
||||
f. Click `Dataflow request` and then click `Continue`. The *Request for New Data Flow on an Existing Transport System* screen will appear.
|
||||
|
||||
SECRET//REL TO USA, FVEY
|
||||
|
||||
SECRET// REL TO USA, FVEY
|
||||
|
||||
g. Enter as much information as you can. In the *Data Flow Change Description* field, specifically indicate that the pick-up directory should be:
|
||||
`/$XSCORE_DATA_DIR/outputs/mailorder`
|
||||
h. Click `Submit`. You will receive a confirmation e-mail indicating the ticket has been received and more information may be requested before the ticket can be completed. Overall, the process might take anywhere from a few days to a week to complete.
|
||||
|
||||
(C//REL) Configuring xks.config for VoIP Processing
|
||||
|
||||
(U) Configuration for VoIP can be accomplished in just three steps:
|
||||
|
||||
1. (U//FOUO) Log on as the user `oper`.
|
||||
|
||||
2. (U//FOUO) At the command line from within any directory, type `viconfig` and then press Enter. The `xks.config` file will open.
|
||||
|
||||
3. (U//FOUO) In the VoIP Configuration section of `xks.config`, set the following configurations:
|
||||
a. `voip = yes` : This runs the VoIP workflows.
|
||||
b. `voip mode = site` : This indicates that normal site VoIP workflows will run. Other valid values are:
|
||||
* `overlord`: runs workflows for the overlord server to do cross-cluster pairing at the same site.
|
||||
* `cross_site`: runs workflows for the central VoIP server to do cross-site pairing.
|
||||
c. `voip_tasking_site_utt = {default:yes}` : This uses UTT tasking from the site SSDM. It assumes you have an SSDM at site that is sending UTT tasking to XKEYSCORE and that you have "utt = yes" in the `xks.config` file (see UTT configuration).
|
||||
d. `voip_tasking_site_csv = {default:no}` : This controls where VoIP uses csv files from. By default, it uses the csv files from `$XSCORE_DIR/config/dictionaries/voip_site`.
|
||||
|
||||
**Note:** If you are using target number directory (tnd) tasking, then you must change the setting to `yes`. Provided `tnd=yes` in `xks.config`, the `tnd_proc.py` script will write a csv file to `$XSCORE_DIR/config/dictionaries/voip_site`.
|
||||
e. `voip_tasking_royale_file = {default:no}` : This controls whether or not `config/dictionaries/voip/xkdb_tasking.csv` is used. This file represents the legacy way of tasking VoIP, but it is not guaranteed to be completely legal for any site. By default, it is set not to be used and should only be used as a last option when no site-specific tasking is available.
|
||||
|
||||
2 |
|
||||
|
||||
SECRET// REL TO USA, FVEY
|
||||
|
||||
SECRET// REL TO USA, FVEY
|
||||
|
||||
f. `voip_forward_voice=enable=yes,fdi="FD0",local_fdi="FD1;J:FD2;K:FD3"`
|
||||
This enables forwarding of voice captures. Set the file distribution indicator (fdi) to the MAILORDER trigraph for voice (FD0). Configure `local_fdi` if you want to route local zip codes to a different fdi. Zip codes that begin with 'L' are forwarded to FD1, zip codes that start with 'J' go to FD2, and zip codes that start with 'K' go to FD3.
|
||||
g. `voip_forward_fax=enable=yes,fdi="DEF",local_fdi="XXX"` : This enables forwarding of captured faxes. The file distribution indicator (fdi) should be set to the MAILORDER trigraph for faxes. And, set the `local_fdi` if you want to route local zip codes that start with 'L' to a different fdi.
|
||||
h. `voip_forward_dictionaries="uttdnr"` : This indicates that VoIP forwarding dictionaries will use UTT selectors.
|
||||
i. `csdf_classification = ""` : This overrides the default classification of the system. By default this option is commented out, but it can be uncommented and modified.
|
||||
|
||||
(U) Additional Processes
|
||||
|
||||
(U//FOUO) In addition to editing VoIP configurations in `xks.config`, it is important to execute several set-up processes. As the `oper` user, execute the following commands only after entering the VoIP configurations in `xks.config`:
|
||||
|
||||
1. (U//FOUO) `xks setup voip`: This ensures all VoIP configurations are applied where applicable.
|
||||
|
||||
2. (U//FOUO) `xks rsync push_config`: This pushes any changes to the slaves in the cluster.
|
||||
|
||||
3. (U//FOUO) `xks proc saferestart`: After setup is complete, this restarts `process_data_parent`'s for the new configuration to take effect. This process loads all the dictionaries and fingerprints and then performs dictionary scanning, metadata extraction, databasing of metadata, and archival of content. When the parent is finished reloading, it will do a staggered restart of its children based on which slave the parent is running on.
|
||||
|
||||
3 |
|
||||
SECRET// REL TO USA, FVEY
|
||||
Loading…
Add table
Add a link
Reference in a new issue