3.9 KiB
UNCLASSIFED//FOR OFFICIAL USE ONLY
(U//FOUO) Forwarding Selected MAILORDER Files Read Me
(U//FOUO) This document explains how to configure an XKS cluster to forward selected sessions via MAILORDER. Sessions are still scanned, tagged, and queryable in XKEYSCORE, even if the quota limit (category throttling) has been exceeded for the day. Use this document to learn the procedures for switching all Digital Network Intelligence (DNI) filtering and selection to the XKEYSCORE collection system.
(U//FOUO) Configuring DNI Selection in xks.config
(U//FOUO) Follow these steps to configure xks.config to use the XKS collection system for DNI filtering and selection:
- (U//FOUO) Logon as the user
oper. - (U//FOUO) At the command line from within any directory, type
viconfigand then press Enter. Thexks.configfile will open. - (U//FOUO) In the
inputssection ofxks.configset the following configurations: a.forward sessions = yes: This enables session forwarding on the selected output plugin. You can verify that session forwarding is set toyesin thedefault.xmlfile located in$XSCORE_DIR/config/pluginsNote: Only after running thexks setup pluginsprocess, described on page 2, will you see theselected_outputplugin enabled indefault.xml. b.category_throttle = true(default). This sets the maximum number of sessions per category to 10,000 and the total size forwarded for any given category to be no greater than 1GB. If either of these limits is reached, XKEYSCORE will no longer forward sessions that hit on that particular category for the rest of the day. It will, however, continue to process data for other categories. These limits are reset every 24 hours. c. All MAILORDER files will be stored on the Master server in:/export/data/xkeyscore/outputs/mailorder
1 | UNCLASSIFED//FOR OFFICIAL USE ONLY UNCLASSIFIED//FOR OFFICIAL USE ONLY
You can check the status of the category throttle using one of the following commands:
-
mysqls quota status currentThequota_limitsforwarding status for all categories. -
mysqls quota status yesterdayThequota_limitsforwarding status for the previous day. -
mysqls quota_throttled currentThe top 20 categories for the current day (The phrase "negative remaining" indicates the category was throttled). d.output_location = master: All MAILORDER files will be generated in:/xks_data/<masterHostname>/outputs/mailorder_workingThen the
mailorder_procprocess on the Master will rename and move them to:/xks_data/<masterHostname>/outputs/mailorder
e. mailorder = yes (default) : On the Master, this creates the mailorder_proc process that polls the mailorder_working directory and then moves/renames any MAILORDER files to $XSCORE_DATA_DIR/outputs/mailorder.
4. (U//FOUO) Type: :wq! and then press Enter to save and exit xks.config.
(U//FOUO) Additional Processes
(U//FOUO) In addition to editing the configurations in xks.config, it is important to run several set-up processes.
- (U//FOUO) As the
operuser, at the command prompt, typexks setup pluginsand press Enter. This ensures any applicable changes to plugin configurations will take effect. - (U//FOUO) As the
operuser, at the command prompt, typexks setup processesand press Enter. This will create themailorder_procprocess on the Master. - (U//FOUO) At the command prompt, type
rsync push_configand press Enter. This pushes the latest configuration changes to the slaves. - (U//FOUO) As the
operuser, at the command prompt, typexks proc startand press Enter. This will ensure all of the running processes pick up any configuration changes. - (U//FOUO) At the command prompt, type
xks proc saferestartand press Enter. After setup is complete, this restartsprocess_datasfor the new configurations to take effect.
2 | UNCLASSIFIED//FOR OFFICIAL USE ONLY