first commit
This commit is contained in:
commit
0d90363116
6510 changed files with 679146 additions and 0 deletions
53
data/xks-workflows-2009/metadata.json
Normal file
53
data/xks-workflows-2009/metadata.json
Normal file
|
|
@ -0,0 +1,53 @@
|
|||
{
|
||||
"title": "XKEYSCORE Workflows 2009",
|
||||
"type": "REPORTED",
|
||||
"description": "A 2009 XKeyscore training presentation on Workflows — standing automated queries that run on a schedule against XKeyscore's rolling buffer — walking analysts through the Workflow Central request wizard to build recurring searches with follow-on actions like email alerts, session downloads to Agility, MAILORDER forwarding and SQL reports, with login banners showing the Menwith Hill Station Human Rights Act and Sensitive Targeting Approval compliance regime.",
|
||||
"long_description": "This is a SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL XKeyscore training presentation dated 5 March 2009 on Workflows. It teaches analysts how to create and manage workflows — automated standing queries that run on a schedule — through the Workflow Central request wizard.\n\nWorkflows automate any XKeyscore search type as either one-time or standing queries, offering the same functionality as ad hoc searches but running on a schedule. They can be configured to run once over a set datetime range, or recurringly every 1, 2, 3, 4, 6, 8, 12 or 24 hours with a configurable start offset. The rationale for workflows includes the fact that XKeyscore holds only a rolling buffer of data (so standing queries capture data before it ages off), repetitive SIGDEV queries, fingerprint and AppID testing, and avoiding long query times during peak periods. Anyone can submit a workflow; each has a single owner (though multiple users can be notified), and results appear only in the owner's My Results view. Follow-on actions include email alerts, session downloads, MAILORDER forwarding, and MySQL reports.\n\nThe bulk of the presentation is a step-by-step walkthrough of the request wizard using an example workflow named \"Find_my_appid\" (justified as \"Testing appid signature\"), which searches the Full Log for traffic from IP 1.2.3.4 to port 80 matching the AppID fingerprint search/google*, running every 6 hours starting at 5:00 EST. The walkthrough covers search type selection, basic information entry (with required unique query names and justifications), search field addition (with detailed explanation of Boolean AND/OR logic — values ANDed by default, ORed via the Multiple Field Search tab or \"OR\" between values), the group-by option, database selection, and the various follow-on actions. The follow-on action screens detail email alerts (comma-delimited addresses, sent only if the workflow has results via the ROWR \"Return Only With Results\" flag), SQL reports (CSV or HTML output from a valid SQL statement against the output table, with an example grouping casenotations by SIGAD), and download sessions (including a \"Send To Agility\" option and MAILORDER trigraph routing).\n\nThe approved workflow screen reveals the underlying XML job structure, including — notably — an xks_password field containing a hashed value (18837b706121a0ca), the interval/offset scheduling, the SQL where-clause with fm_ip, to_ap and fingerprint fields, and the routing to the analyst's personal query database (xks-jychan:q0). The common mistakes section provides troubleshooting guidance on Boolean logic pitfalls (From/To IP with the same value, the behaviour of negation where \"!a\" defeats \"a\" but not \"b\", requiring all defeated values to be ANDed), SIGAD/site selection efficiency, and SQL validity.\n\nThe login banners throughout carry the significant compliance apparatus seen in other Menwith Hill-associated XKeyscore documents: \"This system is audited for USSID 18 and Human Rights Act compliance,\" and the notice describing the Sensitive Targeting Approval (STA) requirement before querying terms specific to a UK, British Dependent Territory or Second Party person or location, with legal guidance available from the HRA Compliance Officer at Menwith Hill Station. The database selection screen showing AUS sites, F6 sites and NZ sites confirms the multi-national, Five Eyes-wide reach of the workflow system.\n\nThe document is significant as detailed documentation of how analysts operationalised persistent, automated surveillance through XKeyscore — turning one-off queries into continuous monitoring that automatically captured, alerted on, downloaded and forwarded matching traffic before it aged out of the buffer. The Human Rights Act and STA compliance framework, the reference to Menwith Hill Station, and the AUS/F6/NZ site options together confirm the document reflects a GCHQ-associated deployment operating under UK legal oversight alongside NSA.",
|
||||
"tags": [
|
||||
"workflow",
|
||||
"automated query",
|
||||
"rolling buffer",
|
||||
"Menwith Hill",
|
||||
"MHS",
|
||||
"query database"
|
||||
],
|
||||
"document": {
|
||||
"creator": "NSA",
|
||||
"date_created": "2009-03-05",
|
||||
"classification": [
|
||||
"SECRET",
|
||||
"COMINT"
|
||||
],
|
||||
"dissemination": [
|
||||
"REL TO USA, AUS, CAN, GBR, NZL"
|
||||
],
|
||||
"banner": "SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL",
|
||||
"genre": "slides",
|
||||
"programs": [
|
||||
"XKEYSCORE",
|
||||
"MAILORDER",
|
||||
"AGILITY"
|
||||
],
|
||||
"countries": [],
|
||||
"people": []
|
||||
},
|
||||
"publication": [
|
||||
{
|
||||
"publisher": "The Intercept",
|
||||
"date_published": "2015-07-01",
|
||||
"title": "XKEYSCORE: NSA’s Google for the World’s Private Communications",
|
||||
"link": "https://theintercept.com/2015/07/01/nsas-google-worlds-private-communications/",
|
||||
"reporters": [
|
||||
"Marquis-Boire, Morgan",
|
||||
"Greenwald, Glenn",
|
||||
"Lee, Micah"
|
||||
],
|
||||
"contributors": [],
|
||||
"doc_source": "https://theintercept.com/document/2015/07/01/xks-workflows-2009/",
|
||||
"doc_url": "https://s3.documentcloud.org/documents/2157515/xks-workflows-2009.pdf"
|
||||
}
|
||||
],
|
||||
"aka_id": [],
|
||||
"pdf_sha256": "4289f684941779eb3d4275bab8c81548cf3342c32e14738f8c5d615de2f33041",
|
||||
"pages": 24
|
||||
}
|
||||
881
data/xks-workflows-2009/xks-workflows-2009.md
Normal file
881
data/xks-workflows-2009/xks-workflows-2009.md
Normal file
|
|
@ -0,0 +1,881 @@
|
|||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
# XKEYSCORE Workflows
|
||||
|
||||
05 March 2009
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
DERIVED FROM: NSA/CSSM 1-52
|
||||
DATED: 20070108
|
||||
DECLASSIFY ON: 20320108
|
||||
|
||||
---
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
## What is a workflow?
|
||||
|
||||
* Workflows automate queries.
|
||||
* One-time
|
||||
* Standing
|
||||
* Every search type can be a workflow.
|
||||
* Same functionality and capability
|
||||
* Follow on actions
|
||||
* Email alert
|
||||
* Download actions
|
||||
* Metadata summary
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
---
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
## Who can submit a workflow?
|
||||
|
||||
* Anyone!
|
||||
* One owner per workflow
|
||||
* Multiple-users can be notified
|
||||
* If ownership needs to be changed, a ticket can be submitted to the team.
|
||||
* Future: sharing workflows
|
||||
* Right now, only the owner has the results in their "My Results" view.
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
---
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
## What can I do with a workflow?
|
||||
|
||||
* Workflows can be configured to run once
|
||||
* Workflows can be configured to run daily
|
||||
* Every 1, 2, 3, 4, 6, 8, 12 or 24 hours
|
||||
* You can set an offset to start running at a certain hour
|
||||
* Download results
|
||||
* Email results and email alerts
|
||||
* MAILORDER results
|
||||
* MySQL report
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
---
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
## Why do I want a workflow?
|
||||
|
||||
* XKEYSCORE has a rolling buffer of data
|
||||
* Repetitive queries
|
||||
* Sigdev purpose
|
||||
* Fingerprint and appid testing
|
||||
* Queries take a long time during high times
|
||||
* Follow on actions
|
||||
* Google Earth data
|
||||
* Statistics
|
||||
* Customizable – write a script!
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
---
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
## How do I setup a workflow?
|
||||
|
||||
```
|
||||
This system is audited for USSID 18 and Human Rights Act compliance
|
||||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, and NZL//20320108
|
||||
XKEYSCORE Welcome: switch users
|
||||
|
||||
Navigation Menu
|
||||
Explorer
|
||||
Home
|
||||
Workflow Central
|
||||
Request
|
||||
My Workflows
|
||||
Search
|
||||
Classic
|
||||
MultiSearch
|
||||
Classic A-M
|
||||
Classic N-Z
|
||||
Common
|
||||
Category DNI
|
||||
Document Metadata
|
||||
Email Addresses
|
||||
UserActivity
|
||||
VoIP
|
||||
Wireless
|
||||
Results
|
||||
My Recent Results
|
||||
My Previous Results
|
||||
My Ongoing Results
|
||||
My Downloads
|
||||
Statistics
|
||||
Link Summarization
|
||||
Tagging
|
||||
Local Tagging
|
||||
Tech Extractor Tagging
|
||||
|
||||
Preferences Help
|
||||
|
||||
Welcome to the New XKEYSCORE Home Page!
|
||||
If you have questions or bug reports go to XKEYSCORE New GUI Forum
|
||||
To use the old GUI, click here
|
||||
|
||||
HUMAN RIGHTS ACT,
|
||||
USSID 18 AND USSID 9
|
||||
I (SYSTEM) queries require a justification to ensure Human Rights Act (HRA), USSID 18 and
|
||||
SSID 9 compliance. Please enter information as prompted by the query interface. An audit
|
||||
all has been established and will be searched as part of Menwith Hill Station's response to
|
||||
y complaint brought under HRA and as part of the USSID 18 and USSID 9 process.
|
||||
ease note that SENSITIVE TARGETING APPROVAL (STA) is required for HRA before submitting
|
||||
y query which includes terms specific to a person or company (eg name, address, identity
|
||||
etails such as communications address, passport/bank account number) who EITHER (a) is
|
||||
efined as a UK, British Dependent Territory (BDT) or Second Party "person" or (b) is located in
|
||||
e UK, or a BDT or Second Party country. STA is also required for wildcard pulls that are
|
||||
evitably going to retrive a substantial proportion of such enties (e.g. wildcarding on a UK city
|
||||
de). Full legal guidance is available from the HRA Compliance Officer at Menwith Hill Station.
|
||||
|
||||
This system is audited for USSID 18 and Human Rights Act compliance
|
||||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, and NZL//20320108
|
||||
```
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
---
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
## How do I setup a workflow?
|
||||
|
||||
First, s a
|
||||
workflo
|
||||
|
||||
```
|
||||
Workflow Central Request Wizard
|
||||
Please select a Search Type.
|
||||
|
||||
Full Log
|
||||
|
||||
Every session collected, indexed by "standard" DNI meta-data
|
||||
(to/from IP, port, casenotation, application id, sigad, etc).
|
||||
|
||||
Search Type Help
|
||||
|
||||
Cancel Prev Next Submit
|
||||
```
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
---
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
## How do I setup a workflow?
|
||||
|
||||
```
|
||||
Workflow Central Request Wizard
|
||||
Basic Information
|
||||
Query Name: Find_my_appid
|
||||
Query Justification: Testing appid signature
|
||||
Additional Justification:
|
||||
Miranda Number:
|
||||
Datetime: 1 Day Start: 2009-03-04 00:00 Stop: 2009-03-05 23:59
|
||||
Reccurring Search | One Time Search
|
||||
Basic Features Help
|
||||
|
||||
Runs once over a set datetime range
|
||||
|
||||
Cancel Prev Next Submit
|
||||
```
|
||||
|
||||
ring or one-
|
||||
st be unique per user
|
||||
must have a justification
|
||||
justifications
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
---
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
## How do I setup a workflow?
|
||||
|
||||
Selec nt to
|
||||
searc
|
||||
|
||||
Select a field to search
|
||||
|
||||
or every field,
|
||||
ou must select
|
||||
e PLUS key
|
||||
|
||||
```
|
||||
Workflow Central Request Wizard
|
||||
|
||||
Add Search Fields
|
||||
|
||||
Search Values are ANDed by default.
|
||||
|
||||
To OR Search Fields:
|
||||
* Use the Multiple Field Search tab (below the input fields).
|
||||
* Select all the fields you wish to search.
|
||||
To OR Search Values:
|
||||
* Type 'OR' between each value (no quotes).
|
||||
|
||||
See Search Value Help below for more details or
|
||||
for a description of boolean logic go to here.
|
||||
|
||||
Search Field Search Value Remove
|
||||
From IP Address OR To IP Address 1.2.3.4
|
||||
Attribute Info
|
||||
From IP Address
|
||||
To IP Address
|
||||
From Port
|
||||
To Port
|
||||
Single Field Search | Multiple Field Search
|
||||
Search Value Help
|
||||
|
||||
Cancel Prev Next Submit
|
||||
```
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
---
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
## Group by option
|
||||
|
||||
* Group b ta results.
|
||||
* Red
|
||||
* Retu
|
||||
|
||||
```
|
||||
Workflow Central Request Wizard
|
||||
Group Search Fields
|
||||
Would you like to group any fields?
|
||||
No
|
||||
Yes
|
||||
Group By Type
|
||||
Table Unique Values:
|
||||
Global Unique Values:
|
||||
Group By Type Help
|
||||
Columns to Group By
|
||||
Datetime:
|
||||
Client IP (X-Fowarded-For):
|
||||
Username:
|
||||
Attribute Info:
|
||||
From IP Address:
|
||||
To IP Address:
|
||||
From Port:
|
||||
To Port:
|
||||
From Country (IP):
|
||||
To Country (IP):
|
||||
From City (IP):
|
||||
To City (IP):
|
||||
From Latitude (IP):
|
||||
|
||||
This option groups each and concatenated.
|
||||
|
||||
Select the fields you want to group by.
|
||||
|
||||
Cancel Prev Next Submit
|
||||
```
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
---
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
## Select databases
|
||||
|
||||
```
|
||||
Workflow Central Request Wizard
|
||||
Select the Database(s) to query
|
||||
|
||||
xks- :q0(xks :qo)
|
||||
xks- :qsummary (xks- qsummary)
|
||||
|
||||
Content must exist
|
||||
Check All
|
||||
Uncheck All
|
||||
|
||||
Basic Features Help
|
||||
|
||||
If this is selected, results
|
||||
are only returned if the
|
||||
content still exists at site.
|
||||
|
||||
Cancel Prev Next Submit
|
||||
```
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
---
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
## Follow on Actions
|
||||
|
||||
* All
|
||||
* All ntent) to another loca
|
||||
* All
|
||||
|
||||
```
|
||||
Workflow Central Request Wizard
|
||||
Follow-on Actions
|
||||
Would you like to add any follow on actions
|
||||
No
|
||||
Yes
|
||||
Script Script Arguments Add
|
||||
Email Alert
|
||||
Email Alert
|
||||
SQL Report
|
||||
Download Sessions
|
||||
|
||||
Script Arguments
|
||||
Email To:
|
||||
ROWR: Return Only With Results
|
||||
|
||||
mpleted.
|
||||
```
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
---
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
## Email alert
|
||||
|
||||
Comma delimited email addresses.
|
||||
This option only sends an email if you workflow has results.
|
||||
|
||||
```
|
||||
Workflow Central Request Wizard
|
||||
Follow-on Actions
|
||||
Would you like to add any follow on actions
|
||||
No
|
||||
Yes
|
||||
Script
|
||||
Email Alert
|
||||
Script Arguments
|
||||
Email To:
|
||||
ROWR: Return Only With Results
|
||||
Add
|
||||
```
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
---
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
## SQL report
|
||||
|
||||
```
|
||||
Workflow Central Request Wizard
|
||||
Follow-on Actions
|
||||
Would you like to add any follow on actions
|
||||
No
|
||||
Yes
|
||||
Script
|
||||
SQL Report
|
||||
Script Arguments
|
||||
Type:
|
||||
Email To:
|
||||
Email Subject:
|
||||
Email Content:
|
||||
Email Attachment: Email Attachment
|
||||
ROWR: Return Only With Results
|
||||
Filename:
|
||||
Mail Order Trigraph:
|
||||
SQL: SELECT
|
||||
FROM%{OUTPUT_TABLE}
|
||||
WHERE
|
||||
GROUP BY
|
||||
GZIP: Compress Contents
|
||||
Add
|
||||
Cancel Prev Next Submit
|
||||
|
||||
CSV or HTML
|
||||
This must be a VALID SQL statement.
|
||||
|
||||
Email metadata that a user can set.
|
||||
|
||||
Example.
|
||||
|
||||
SELECT casenotation, sigad
|
||||
FROM %{OUTPUT_TABLE}
|
||||
WHERE sigad!=''
|
||||
GROUP BY casenotation
|
||||
```
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
---
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
## Download Results
|
||||
|
||||
```
|
||||
Workflow Central Request Wizard
|
||||
Follow-on Actions
|
||||
Would you like to add any follow on actions
|
||||
No
|
||||
Yes
|
||||
Script
|
||||
Download Sessions
|
||||
Script Arguments
|
||||
User ID:
|
||||
Email To:
|
||||
Email Subject:
|
||||
Email Content:
|
||||
ROWR: Return Only With Results
|
||||
Filename:
|
||||
Mail Order Trigraph:
|
||||
GZIP: Compress Contents
|
||||
Send To Agility: Send To Agility
|
||||
Add
|
||||
|
||||
Cancel Prev Next Submit
|
||||
```
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
---
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
## You're almost done!
|
||||
|
||||
```
|
||||
Workflow Central Request Wizard
|
||||
Workflow Review
|
||||
This query (Find_my_appid) will search the Full Log table in database(s):
|
||||
xks-jychan:q0
|
||||
|
||||
The query will run CONTINUOUSLY executing every 6 hours beginning at 5:00 EST
|
||||
|
||||
The query will execute the following search criteria:
|
||||
|
||||
<and>
|
||||
<field>From IP Address</field>
|
||||
<value>1.2.3.4</value>
|
||||
</and>
|
||||
|
||||
<and>
|
||||
<field>To Port</field>
|
||||
<value>80</value>
|
||||
</and>
|
||||
|
||||
<and>
|
||||
<field>AppID (+Fingerprints)*</field>
|
||||
<value>search/google*</value>
|
||||
</and>
|
||||
|
||||
Workflow Values Workflow XML
|
||||
|
||||
Cancel Prev Next Submit
|
||||
```
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
---
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
## Workflow Pending
|
||||
|
||||
```
|
||||
This system is audited for USSID 19 and Human Rights Act compliance
|
||||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108
|
||||
XKEYSCORE Welcome: jychan switch users
|
||||
|
||||
Home Workflow Central Seach Results Statistics Tagging Preferences Help
|
||||
|
||||
Navigation Menu
|
||||
Explorer
|
||||
Home
|
||||
Workflow Central
|
||||
Request
|
||||
My Workflows
|
||||
Search
|
||||
Classic
|
||||
MultiSearch
|
||||
Classic A-M
|
||||
Classic N-Z
|
||||
Common
|
||||
Category DNI
|
||||
Document Metadata
|
||||
Email Addresses
|
||||
Extracted Files
|
||||
Full Log DNI
|
||||
HTTP Activity
|
||||
Phone Number Extractor
|
||||
User Activity
|
||||
Dictionary Hits
|
||||
File Transfer
|
||||
MultiSearch
|
||||
IP Addresses
|
||||
Mac Addresses
|
||||
Username
|
||||
Network Management
|
||||
Search Wizard
|
||||
User Activity
|
||||
VoIP
|
||||
Wireless
|
||||
Results
|
||||
My Recent Results
|
||||
My Previous Results
|
||||
My Ongoing Results
|
||||
My Downloads
|
||||
Statistics
|
||||
Link Summarization
|
||||
Tagging
|
||||
Local Tagging
|
||||
Tech Extractor Tagging
|
||||
|
||||
My Workflows
|
||||
Help Actions
|
||||
|
||||
Query Type Query Name Last Modified State Actions
|
||||
full_log Find_my_appid 2009-03-05 14:14: 5 Pending
|
||||
|
||||
State Actions
|
||||
pending
|
||||
```
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
---
|
||||
|
||||
## Workflow Approved
|
||||
|
||||
|
||||
```
|
||||
This system is audited for USSID 18 and Human Rights Act compliance
|
||||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, and NZL//20320108
|
||||
XKEYSCORE Welcome: jychan switch users
|
||||
|
||||
Home Workflow Central Seach Results Statistics Tagging Preferences Help
|
||||
|
||||
Navigation Menu
|
||||
Explorer
|
||||
Home
|
||||
Workflow Central
|
||||
Request
|
||||
My Workflows
|
||||
Search
|
||||
Classic
|
||||
MultiSearch
|
||||
Classic A-M
|
||||
Classic N-Z
|
||||
Common
|
||||
Category DNI
|
||||
Document Metadata
|
||||
Email Addresses
|
||||
Extracted Files
|
||||
Full Log DNI
|
||||
HTTP Activity
|
||||
Phone Number Extractor
|
||||
User Activity
|
||||
Dictionary Hits
|
||||
File Transfer
|
||||
MultiSearch
|
||||
IP Addresses
|
||||
Mac Address
|
||||
Username
|
||||
Network Management
|
||||
Search Wizard
|
||||
User Activity
|
||||
VoIP
|
||||
Wireless
|
||||
Results
|
||||
My Recent Results
|
||||
My Previous Results
|
||||
My Ongoing Results
|
||||
My Downloads
|
||||
Statistics
|
||||
Link Summarization
|
||||
Tagging
|
||||
Local Tagging
|
||||
Tech Extractor Tagging
|
||||
|
||||
My Workflows
|
||||
Help Actions
|
||||
|
||||
Query Type
|
||||
full_log
|
||||
|
||||
Workflow: Find_my_appid
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<query_jobs>
|
||||
<internal_gui>1</internal_gui>
|
||||
<datetime_created>1236264295</datetime_created>
|
||||
<job>
|
||||
<xks_userid>[REDACTED]</xks_userid>
|
||||
<xks_user_name>[REDACTED]</xks_user_name>
|
||||
<xks_password>18837b706121a0ca</xks_password>
|
||||
<search_type>full_log</search_type>
|
||||
<query_name>Find_my_appid</query_name>
|
||||
<query_justification>Testing appid signature </query_justification>
|
||||
<datetime>
|
||||
<interval>6</interval>
|
||||
<offset>5</offset>
|
||||
</datetime>
|
||||
<sql>
|
||||
<where>
|
||||
<and>
|
||||
<field>fm_ip</field>
|
||||
<value>1.2.3.4</value>
|
||||
</and>
|
||||
<and>
|
||||
<field>to_ap</field>
|
||||
<value>80</value>
|
||||
</and>
|
||||
<and>
|
||||
<field>fingerprint</field>
|
||||
<value>search/google*</value>
|
||||
</and>
|
||||
</where>
|
||||
<group_by>to_ip</group_by>
|
||||
<indexes>unique key(to_ip)</indexes>
|
||||
</sql>
|
||||
<advanced>
|
||||
<content_must_exist>true</content_must_exist>
|
||||
<routing>
|
||||
<database>xks-jychan:q0</database>
|
||||
</routing>
|
||||
</advanced>
|
||||
|
||||
Cancel Save/Submit
|
||||
|
||||
Page 1 of 1
|
||||
Page Size: 30 Displaying 1 - 1 of 1
|
||||
|
||||
This system is audited for USSID 18 and Human Rights Act compliance
|
||||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, and NZL//20320108
|
||||
```
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
---
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
## Common mistakes
|
||||
|
||||
* From IP and To IP with the same value.
|
||||
* In this view, terms are ANDed together.
|
||||
* Use Multiple Field Search Tab.
|
||||
|
||||
```
|
||||
Workflow Central Request Wizard
|
||||
Add Search Fields
|
||||
|
||||
Search Values are ANDed by default.
|
||||
|
||||
To OR Search Fields:
|
||||
* Use the Multiple Field Search tab (below the input fields).
|
||||
* Select all the fields you wish to search.
|
||||
To OR Search Values:
|
||||
* Type 'OR' between each value (no quotes).
|
||||
|
||||
See Search Value Help below for more details or
|
||||
for a description of boolean logic to go here.
|
||||
|
||||
Search Field Search Value Remove
|
||||
From IP Address OR To IP Address 1.2.3.4
|
||||
Attribute Info
|
||||
From IP Address
|
||||
To IP Address
|
||||
From Port
|
||||
To Port
|
||||
|
||||
Single Field Search | Multiple Field Search
|
||||
Search Value Help
|
||||
|
||||
Cancel Prev Next Submit
|
||||
```
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
---
|
||||
|
||||
## Common mistakes
|
||||
|
||||
* Using the multiple field search does not break this up into 3 search<->value pairs.
|
||||
* Enter each term separately in the singe fieldsearch.
|
||||
|
||||
```
|
||||
Workflow Central Request Wizard
|
||||
Add Search Fields
|
||||
|
||||
Search Values are ANDed by default.
|
||||
|
||||
To OR Search Fields:
|
||||
* Use the Multiple Field Search tab (below the input fields).
|
||||
* Select all the fields you wish to search.
|
||||
To OR Search Values:
|
||||
* Type 'OR' between each value (no quotes).
|
||||
|
||||
See Search Value Help below for more details or
|
||||
for a description of boolean logic to go here.
|
||||
|
||||
Search Field Search Value Remove
|
||||
From IP Address 1.2.3.4
|
||||
To IP Address 5.6.7.8
|
||||
From Port 80
|
||||
|
||||
Single Field Search | Multiple Field Search
|
||||
Search Value Help
|
||||
|
||||
Cancel Prev Next Submit
|
||||
```
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
---
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
## Common mistakes
|
||||
|
||||
* This will return ALL casenotations.
|
||||
* "a" will be defeated by "!a" but "a" does equal "!b"
|
||||
* All the defeated values must be ANDed together.
|
||||
|
||||
```
|
||||
Workflow Central Request Wizard
|
||||
Add Search Fields
|
||||
|
||||
Search Values are ANDed by default.
|
||||
|
||||
To OR Search Fields:
|
||||
* Use the Multiple Field Search tab (below the input fields).
|
||||
* Select all the fields you wish to search.
|
||||
To OR Search Values:
|
||||
* Type 'OR' between each value (no quotes).
|
||||
|
||||
See Search Value Help below for more details or
|
||||
for a description of boolean logic to go here.
|
||||
|
||||
Search Field Search Value Remove
|
||||
Casenotation !a
|
||||
Casenotation !b
|
||||
Casenotation !c
|
||||
Casenotation !d
|
||||
|
||||
Single Field Search | Multiple Field Search
|
||||
Search Value Help
|
||||
```
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
---
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
## Common mistakes
|
||||
|
||||
* If you are selecting specific SIGADs, only select the sites that have data from that SIGAD.
|
||||
* Queries will return faster.
|
||||
* All sites selected
|
||||
* Less work for the system.
|
||||
|
||||
|
||||
```
|
||||
Workflow Central Request Wizard
|
||||
Add Search Fields
|
||||
|
||||
Search Values are ANDed by default.
|
||||
|
||||
To OR Search Fields:
|
||||
* Use the Multiple Field Search tab (below the input fields).
|
||||
* Select all the fields you wish to search.
|
||||
To OR Search Values:
|
||||
* Type 'OR' between each value (no quotes).
|
||||
|
||||
See Search Value Help below for more details or
|
||||
for a description of boolean logic to go here.
|
||||
|
||||
Search Field Search Value Remove
|
||||
Casenotation !c
|
||||
Casenotation !d
|
||||
SIGAD AUC-993
|
||||
|
||||
Select the Database(s) to query
|
||||
=AUS sites
|
||||
=F6 sites
|
||||
=NZ sites
|
||||
|
||||
Content must exist
|
||||
|
||||
Check All
|
||||
Uncheck All
|
||||
|
||||
Basic Features Help
|
||||
```
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
---
|
||||
|
||||
## Common mistakes
|
||||
|
||||
* If you select the SQL Report option, make sure you put a valid SQL statement!
|
||||
|
||||
SQL statement filled in:
|
||||
SELECT casenotation,
|
||||
count(\*)
|
||||
FROM %{OUTPUT_TABLE}
|
||||
WHERE casenotation!=''
|
||||
GROUP BY casenotation
|
||||
|
||||
```
|
||||
Workflow Central Request Wizard
|
||||
Follow-on Actions
|
||||
Would you like to add any follow on actions
|
||||
No
|
||||
Yes
|
||||
|
||||
Script
|
||||
SQL Report
|
||||
|
||||
Script Arguments
|
||||
Type: CSV
|
||||
Email To: analyst@work.com
|
||||
Email Subject: My Workflow Results
|
||||
Email Content: Bad SQL - empty
|
||||
Email Attachment: Email Attachment
|
||||
ROWR: Return Only With Results
|
||||
Filename:
|
||||
Mail Order Trigraph:
|
||||
SQL: SELECT casenotation, count(*)
|
||||
FROM %{OUTPUT_TABLE}
|
||||
WHERE casenotation!=''
|
||||
GROUP BY casenotation
|
||||
GZIP: Compress Contents
|
||||
Add
|
||||
|
||||
Cancel Prev Next Submit
|
||||
```
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
---
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
|
||||
## Questions?
|
||||
|
||||
xks_workflow@r1.r.nsa
|
||||
|
||||
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||||
Loading…
Add table
Add a link
Reference in a new issue