1411 lines
No EOL
24 KiB
Markdown
1411 lines
No EOL
24 KiB
Markdown
TOP SECRET //SI// REL USA,FVEY
|
||
|
||
# Real-Time Regional Gateway Cloud Analytics for Forward Users
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
SIGDEV
|
||
SIGINT DEVELOPMENT
|
||
|
||
[REDACTED]
|
||
|
||
TOP SECRET//SI//REL USA, FVEY
|
||
|
||
REAL-TIME REGIONAL GATEWAY
|
||
|
||
---
|
||
|
||
SECRET//REL USA, FVEY
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
SIGDEV
|
||
SIGINT DEVELOPMENT
|
||
|
||
## BLUF
|
||
|
||
RTRG:
|
||
|
||
...brings near real-time intelligence to the warfighter
|
||
|
||
..."grew up" supporting operations in Iraq
|
||
|
||
...RTRG is now a global architecture
|
||
|
||
...leveraging the emerging cloud architecture to answer questions we have not been able to do before
|
||
|
||
2
|
||
SECRET//REL USA, FVEY
|
||
|
||
---
|
||
|
||
TOP SECRET//SI//REL USA, FVEY
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
SIGDEV
|
||
SIGINT DEVELOPMENT
|
||
|
||
## RTRG in Afghanistan
|
||
|
||
(U)
|
||
|
||
(S//REL)
|
||
|
||
(S//REL) Area 82 – Bagram Air Base Home of RTRG AF1 & gmTote
|
||
|
||
* 20 days data retention from AFPAK
|
||
* 200-600 daily users
|
||
* New upgrades include two systems in Kabul
|
||
|
||
(TS//REL)
|
||
|
||
Mission Areas: Tracking high-value targets (HVT), Counter-Insurgency (COIN), Counter-IED (CIED)
|
||
|
||
Organizations Using RTRG:
|
||
|
||
* CSG* Afghanistan
|
||
* U.S. Marine Corps (USMC) 1st and 2nd Radio Battalions
|
||
* U.S. Army SIGINT analysts at BCT* level
|
||
* U.S. Air Force National Tactical Integration
|
||
* Jalalabad Fusion Cell (USMC)
|
||
* S2 TOPI
|
||
* NSA-G SWAN Counternarcotics Team
|
||
* All special operations task forces
|
||
|
||
(TS//SI//REL)
|
||
|
||
“RTRG is the most significant SIGINT support to the war fighter in the last decade"
|
||
- General David Petraeus
|
||
|
||
"USSOCOM has enduring and critical needs for the tools and data that RT-RG provides"
|
||
- Admiral William McRaven
|
||
|
||
*CST- Cryptologic Support Team
|
||
*BCT - Brigade Combat Team
|
||
*CSG - Cryptologic Support Group
|
||
|
||
TOP SECRET//SI//REL USA, FVEY
|
||
3
|
||
|
||
---
|
||
|
||
TOP SECRET//SI//REL USA,FVEY
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
SIGDEV
|
||
SIGINT DEVELOPMENT
|
||
|
||
## CSG Afghanistan Statistics
|
||
|
||
* In 2011, RTRG in Afghanistan
|
||
* Played a key role in 90% of all SIGINT developed operations
|
||
* Leading to 2270 capture/kill operations
|
||
* 6534 enemies killed in action
|
||
* 1117 detainees
|
||
|
||
4
|
||
TOP SECRET//SI//REL USA, FVEY
|
||
|
||
---
|
||
|
||
TOP SECRET//SI//REL USA, FVEY
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
SIGDEV
|
||
SIGINT DEVELOPMENT
|
||
|
||
## RTRG in the Gulf and Horn of Africa
|
||
|
||
Monitoring Iranian Navy (IRIN) in Straits of Hormuz
|
||
|
||
QESHM ISLAND
|
||
HIDARABI ISLAND
|
||
HENGAM ISLAND
|
||
BAR-E-LANGEH
|
||
FARUR ISLAND
|
||
GREATER TUNB
|
||
BANI FARUR
|
||
ABU MUSA
|
||
JASH
|
||
USS GEORGIA
|
||
SSGN 729
|
||
|
||
Photo of IRIN vessel from US Navy aircraft, located by RTRG
|
||
|
||
Supporting CENTCOM Maritime (NAVCENT)
|
||
Navy Information Operations Command- Bahrain (NIOC-B)
|
||
|
||
Missions supported:
|
||
* Iran, Yemen, Persian Gulf
|
||
* Recent successes include monitoring of Iranian naval assets
|
||
|
||
(TS//SI//REL)
|
||
|
||
RTRG Afloat on subsurface platform USS Georgia (SSGN-729)
|
||
|
||
(TS//SI//REL)
|
||
Missions supported:
|
||
* Horn of Africa: In first week of mission, system received 31 million GSM events, leading to 10 high-value target voice ID, and 90 tactical tip-offs
|
||
|
||
TOP SECRET//SI//REL USA, FVEY
|
||
5
|
||
|
||
---
|
||
|
||
TOP SECRET//SI//REL TO USA, FVEY
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
SIGDEV
|
||
SIGINT DEVELOPMENT
|
||
|
||
## RTRG Global Operations
|
||
|
||
NSA/CSS COLORADO
|
||
TECHNICAL SIGINT
|
||
(U//FOUO)
|
||
|
||
(U//FOUO)
|
||
|
||
US-2 - Denver
|
||
Global Maritime & ELINT
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
UNITED STATES OF AMERICA
|
||
|
||
US-1 Fort Meade
|
||
Mission Assurance
|
||
|
||
IQ2 Iraq*
|
||
COIN, CIED
|
||
|
||
AF-1,5 Bagram & Kabul
|
||
COIN,CIED
|
||
|
||
(S//REL)
|
||
|
||
USS Georgia & USS Florida
|
||
|
||
US-3 JFCOM
|
||
Joint Forces Command
|
||
|
||
BH-1 NIOC** Bahrain
|
||
AFRICOM, EUCOM, PACOM
|
||
|
||
GE-3 Germany**
|
||
AFPAK
|
||
Continuity-of-Operations
|
||
|
||
KO-1,2 USFK** Pangyo
|
||
PACOM & North Korea
|
||
|
||
US-8 NSA-Texas
|
||
Counternarcotics, Maritime, Mexico & SOUTHCOM Support
|
||
|
||
Texas Cryptologic Center
|
||
NSA/CSS TEXAS
|
||
|
||
\* In draw-down
|
||
\** ECC - NSA European Technical Center
|
||
USFK-U.S. Forces, Korea
|
||
NIOC - Navy Information Operations Command
|
||
|
||
TOP SECRET//SI//REL TO USA, FVEY
|
||
6
|
||
|
||
---
|
||
|
||
UNCLASSIFIED//FOR OFFICIAL USE ONLY
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
SIGDEV
|
||
SIGINT DEVELOPMENT
|
||
|
||
## Outline
|
||
|
||
* RTRG Mission Overview
|
||
* RTRG System: Today and Tomorrow
|
||
* Target-Centric and Network-Centric Cloud Analytics
|
||
* Future Work
|
||
|
||
7
|
||
UNCLASSIFIED//FOR OFFICIAL USE ONLY
|
||
|
||
---
|
||
|
||
SECRET//SI//REL TO USA, FVEY
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
SIGDEV
|
||
SIGINT DEVELOPMENT
|
||
|
||
## Current RTRG Architecture
|
||
|
||
RTRG System
|
||
|
||
Goldminer
|
||
Metadata Search
|
||
|
||
GeoT
|
||
Geospatial Tools
|
||
|
||
Agent Logic
|
||
Alerting Tools
|
||
|
||
Sharkfinn
|
||
Selector Enrichment
|
||
|
||
SKS
|
||
Report / Doc Manager
|
||
|
||
Panopticon
|
||
Target Management
|
||
|
||
Supporting Tactical Users
|
||
|
||
Forward Data Centers
|
||
|
||
Services Layer: Web services, authentication, auditing
|
||
|
||
Publish and Subscribe Messaging
|
||
|
||
Oracle relational database & dimensional data model
|
||
|
||
Ingest and Enrichment Pipeline:
|
||
Flexible, high-speed architecture for parser and data processors
|
||
|
||
Data Feeds:
|
||
* DRT
|
||
* JUGGERNAUT
|
||
* CIDNE
|
||
* LOPERS
|
||
* TALIS/MATTERHORN
|
||
* AIRHANDLER
|
||
* LYCANTHROPE
|
||
* KL
|
||
* VOICESAIL
|
||
* RETURNSPRING
|
||
... and a growing list of others*
|
||
|
||
(DNR, DNI collect, tipping and reporting)
|
||
|
||
A successful architecture for several years. Demand for more data feeds, longer retention, and data-intensive analytics has driven RTRG to seek new solutions
|
||
|
||
\*Based on Afghanistan RTRG data flow
|
||
|
||
SECRET//SI//REL TO USA, FVEY
|
||
|
||
---
|
||
|
||
SECRET//SI//REL TO USA, FVEY
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
SIGDEV
|
||
SIGINT DEVELOPMENT
|
||
|
||
## RTRG Data Challenges
|
||
|
||
Current Challenges
|
||
|
||
* Data Storage & Retention
|
||
* "Patterns of Life” analysis needs require 6+ months of data from world-wide collection
|
||
* A typical system has capacity for only 4-6 weeks of regional data (~90% user queries are within seven days of “now”)
|
||
* Data Use & Computation
|
||
* Analytic processes should make maximum use of all available data to find small signals
|
||
* Relational databases are unsuited to sophisticated analytics such as correlation and matching
|
||
|
||
* Data & Technology Heterogeneity
|
||
* New types of data must be added to the system continually
|
||
* With traditional databases, schema modifications are difficult
|
||
* Exotic data management solutions are difficult to adopt due to limited expertise
|
||
|
||
SECRET//SI//REL TO USA, FVEY
|
||
|
||
---
|
||
|
||
UNCLASSIFIED//FOR OFFICIAL USE ONLY
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
SIGDEV
|
||
SIGINT DEVELOPMENT
|
||
|
||
## Cloud Architectures for Analytics
|
||
|
||
Google
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
UNITED STATES OF AMERICA
|
||
|
||
hadoop
|
||
|
||
Apache
|
||
|
||
Cloudbase
|
||
Scalable BigTable Implementation with Security
|
||
|
||
Emerging NSA Cloud Reference Architecture is well-suited for developing analytics on intelligence data
|
||
|
||
**Scalable:** Distributed file systems and databases are built on clusters of commodity hardware, leveraging open source projects and industrial solutions
|
||
|
||
**Computable:** The MapReduce programming model simplifies writing efficient parallel computations that operate over large volumes of data
|
||
|
||
**Flexible:** Cloud technologies enable flexible schema and leverage large open-source efforts
|
||
|
||
UNCLASSIFIED//FOR OFFICIAL USE ONLY
|
||
|
||
---
|
||
|
||
TOP SECRET//SI//REL TO USA, FVEY
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
SIGDEV
|
||
SIGINT DEVELOPMENT
|
||
|
||
## Scalability & Computation
|
||
|
||
Data Challenges in AFPAK
|
||
|
||
Current RTRG (AF1)
|
||
* Current database is 27 terabytes (TB)
|
||
* Retention is ~30 days
|
||
|
||
Future Cloud enabled system
|
||
* Even a modest cloud system (3 rack) for storage will be at least 125 TB of storage
|
||
* 5x increase in available space
|
||
* Actual retention improvement depends on how the space resources are allocated
|
||
|
||
Cloud supports more data feeds & more days of historical data
|
||
|
||
Analytic Challenges from Iraq & AFPAK
|
||
|
||
* Many analytics used by RTRG are based on R6 SORTINGLEAD event summaries
|
||
* Event summaries were originally created on relational databases
|
||
* Collection increased dramatically, and a mapreduce implementation was needed
|
||
* For new analytics with present day collection volumes, a practical parallel execution model is crucial
|
||
|
||
Cloud supports large-scale analytics
|
||
|
||
TOP SECRET//SI//REL TO USA, FVEY
|
||
11
|
||
|
||
---
|
||
|
||
TOP SECRET//SI//REL USA, FVEY
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
SIGDEV
|
||
SIGINT DEVELOPMENT
|
||
|
||
## NSA Cloud Computing Enterprise
|
||
|
||
* 5-12 racks commodity hardware
|
||
* 150+ data nodes
|
||
* 16 GB RAM each
|
||
* Apache Hadoop
|
||
* 100s of terabytes of storage
|
||
* Stores 10s to 100s of billions of events
|
||
* NSA Cloudbase
|
||
* 100s of nodes serving BigTable implementation
|
||
* Stores 100s of billions of entries
|
||
|
||
gmAPPLE
|
||
Yakima
|
||
|
||
gmCAVE
|
||
gmPEACH
|
||
Utah Language Center
|
||
|
||
gmDEN
|
||
ADF-C
|
||
|
||
gmMOUNT
|
||
|
||
gmLIGHT
|
||
gmBALTIC
|
||
gmPLACE
|
||
gmPARK
|
||
gmHALO
|
||
gmALAMO
|
||
gmCARBON
|
||
|
||
Europe
|
||
|
||
gmROYALE
|
||
gmREGAL
|
||
gmGEIST
|
||
|
||
Iraq
|
||
|
||
gmIDEA (Camp Victory)
|
||
|
||
Afghanistan
|
||
|
||
gmTOTE (Bagram)
|
||
|
||
Korea & Japan
|
||
gmPEN
|
||
S. Korea
|
||
|
||
gmPAC
|
||
Misawa, Japan
|
||
|
||
gmCALM
|
||
Camp Humphreys
|
||
S. Korea
|
||
|
||
Australia
|
||
|
||
gmMATE
|
||
(S//REL) (Alice Springs)
|
||
|
||
GHOSTMACHINE is a data-intensive cloud system with many fielded instances worldwide
|
||
|
||
Hadoop clusters have been demonstrated with 10s of petabytes and over 10,000 cores
|
||
|
||
12
|
||
Map does not include all GHOSTMACHINE/SiteStore systems
|
||
TOP SECRET//SI//REL USA, FVEY
|
||
|
||
---
|
||
|
||
TOP SECRET//SI//REL TO USA, FVEY
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
SIGDEV
|
||
SIGINT DEVELOPMENT
|
||
|
||
## Today's "Hybrid” Architecture
|
||
|
||
GTM
|
||
GeoT
|
||
SKS
|
||
PANOPTICON
|
||
Applications
|
||
|
||
Services Layer
|
||
|
||
Publish and Subscribe Messaging
|
||
|
||
Relational Database
|
||
|
||
Ingest and Enrichment
|
||
|
||
hadoop
|
||
Large scale data preprocessing and analysis
|
||
|
||
Cloudbase
|
||
Scalable BigTable Implementation with Security
|
||
Massive storage and fast record lookup
|
||
|
||
GHOSTMACHINE
|
||
|
||
RTRG and GHOSTMACHINE systems are paired with one another:
|
||
MapReduce analytic results are fed back to RTRG relational database
|
||
|
||
TOP SECRET//SI//REL TO USA, FVEY
|
||
13
|
||
|
||
---
|
||
|
||
TOP SECRET//SI//REL TO USA, FVEY
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
SIGDEV
|
||
SIGINT DEVELOPMENT
|
||
|
||
## Tomorrow's Architecture
|
||
|
||
User interfaces
|
||
* W WidgetFramework
|
||
* GTM
|
||
* GeoT
|
||
* CHET
|
||
* H&S
|
||
* SKS
|
||
* PANOPTICON
|
||
* Web Tier
|
||
|
||
Services
|
||
* Application Services
|
||
* Alerting
|
||
* Ingest
|
||
* Real time Enrichment
|
||
* Analytics
|
||
* Streaming-based
|
||
* Analytics
|
||
* Map Reduce-based
|
||
|
||
Data
|
||
* Data Storage (NFS)
|
||
* Secure Data Access Services
|
||
* Casport, Wavelegal, etc.
|
||
* Relational Store
|
||
* Enrichment, User, & Auxiliary Data
|
||
* Cloudbase
|
||
* Event Data and Analytic Results
|
||
|
||
Operating System
|
||
* Java VM
|
||
* Java VM
|
||
* Operating System
|
||
* Hardened RH Linux
|
||
* Operating System
|
||
* Hardened RH Linux
|
||
* OpenStack VM
|
||
* Utility Cloud
|
||
* Hardware Layer
|
||
* Data/Analytic Cloud
|
||
* Hardware Layer
|
||
|
||
Data Storage (HDFS)
|
||
|
||
The Cloud will bring new data-intensive capabilities, support existing missions, and align RTRG installations with emerging NSA and IC standards
|
||
|
||
TOP SECRET//SI//REL TO USA, FVEY
|
||
|
||
---
|
||
|
||
UNCLASSIFIED//FOR OFFICIAL USE ONLY
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
SIGDEV
|
||
SIGINT DEVELOPMENT
|
||
|
||
## Outline
|
||
|
||
* RTRG Mission Overview
|
||
* RTRG System: Today and Tomorrow
|
||
* Target-Centric and Network-Centric Cloud Analytics
|
||
* Future Work
|
||
|
||
15
|
||
UNCLASSIFIED//FOR OFFICIAL USE ONLY
|
||
|
||
---
|
||
|
||
TOP SECRET//SI//REL TO USA, FVEY
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
SIGDEV
|
||
SIGINT DEVELOPMENT
|
||
|
||
## Analytics Overview
|
||
|
||
Target Development
|
||
|
||
Co-Travelers & Meetings
|
||
Contact Similarity
|
||
High Priority
|
||
Untasked Numbers
|
||
|
||
Graph & Network
|
||
|
||
Graph Triage
|
||
Handset Swap
|
||
Target Enrichment & Disambiguation
|
||
|
||
Beddown
|
||
Layercake
|
||
Heatmaps
|
||
|
||
Geo-Spatial
|
||
|
||
The data-intensive computing capabilities in the system enables a set of graph/network analytics and target development analytics
|
||
|
||
TOP SECRET//SI//REL TO USA, FVEY
|
||
16
|
||
|
||
---
|
||
|
||
TOP SECRET//SI//REL TO USA, FVEY
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
SIGDEV
|
||
SIGINT DEVELOPMENT
|
||
|
||
## Meeting Target Development Challenges
|
||
|
||
**Past** Analysts manually queried multiple, independent repositories, aggregating results in Excel, taking hours or work for search and refinement
|
||
|
||
**Now** RT-RG provides a streamlined, integrated workflow saving analyst effort
|
||
|
||
Tips
|
||
|
||
Selectors
|
||
|
||
GIST Report
|
||
|
||
Detainee Reports
|
||
|
||
Target Development
|
||
|
||
Event Type comms.gsm
|
||
Event Subtype call.*
|
||
Selectors IMSI:12345
|
||
|
||
NAI Kandahar
|
||
?
|
||
|
||
Submit Query Cancel Query Export To File...
|
||
|
||
Metadata Search
|
||
|
||
[REDACTED]
|
||
Profile Reports Network Changelog Recommendations
|
||
OBJ DEWEY BEACH ASSOCIATE
|
||
Bottom Line Pech Valley INS possibly in charge of a "company" sized element. Led the attack
|
||
Last Active: (TS//SI//REL TO USA, FVEY) (IMEI)
|
||
9 hours ago
|
||
Last Location: (TS//SI//REL TO USA, FVEY) (IMEI)
|
||
RTRG-Afghanistan Targets 9 hours ago Geo
|
||
Processor: AROCC
|
||
Last Report: (TS//SI//REL TO USA, FVEY) (MSISDN) view report
|
||
5 months ago
|
||
VID/SID: 1 year ago
|
||
|
||
Collaboration
|
||
|
||
TOP SECRET//COMINT//REL TO U
|
||
CST 17 FOB FENTY (PAN): AF IN
|
||
(Khaksar)
|
||
Thu, 10 May 2012 15:55:20 Zu
|
||
TOP SECRET//COMINT//REL TO
|
||
NZL//20320108
|
||
ASSET: JUGGERNAUT
|
||
|
||
Geospatial Alerting
|
||
|
||
|
||
Afghanistan/Insurgency:
|
||
Taliban Leaders in
|
||
Chaman, Pakistan,
|
||
Command Insurgent
|
||
Operations throughout
|
||
Southern Afghanistan-A
|
||
SIGINT Assessment (S//
|
||
SI//REL TO USA, FVEY)
|
||
|
||
Afghanistan: Influence of
|
||
Chamay on RC South,
|
||
Juge-July2010
|
||
|
||
IIR 6 223 0164 12/DOME
|
||
EXPOSE/DRAGON FIRE-
|
||
EXPLOITATION OFPAGE 3
|
||
|
||
Report Text for:
|
||
02992282
|
||
2017302 AUG 10
|
||
FM USJ-800
|
||
T O P S E C R E T UMBRA US/UK/CAN/AUS/NZ EYES ONLY
|
||
SERIAL: 3/CY/507710-10
|
||
SUBJ: Afghanistan/Insurgency: Taliban Leaders in Chaman,
|
||
Pakistan, Command Insurgent Operations throughout
|
||
Southem
|
||
Afghanistan A SIGINT Assessment (S//SI//REL TO
|
||
USA, FVEY)
|
||
|
||
20100119 08) BIMINI
|
||
(10020-3AFG)
|
||
_Nokia1650
|
||
|
||
kc taliban ircurg
|
||
|
||
Enrichment
|
||
|
||
Outcomes
|
||
|
||
Geolocation
|
||
|
||
Capture
|
||
|
||
TOP SECRET//SI//REL TO USA, FVEY
|
||
17
|
||
|
||
---
|
||
|
||
TOP SECRET//SI//REL TO USA, FVEY
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
SIGDEV
|
||
SIGINT DEVELOPMENT
|
||
|
||
## Target Development with Meetings
|
||
|
||
Who is at the same UCELLID at the same time?
|
||
|
||
Manual Process
|
||
|
||
* Take your selector and query for every unique location he has been and at what time
|
||
* Query for other selectors who have been at the same places at the same times (impossible or painful)
|
||
* OR compare to another known set of selectors to find overlap (excel / ArcGIS / JEMA) (limiting to what you know)
|
||
* Summary statistics on the matching IMSIs using excel or ArcGIS
|
||
|
||
Cloud Process
|
||
|
||
* Pre-calculates all UCELLID overlaps between tasked selectors
|
||
* Simply query your selector in cloud-generated QFD and view summary statistics
|
||
|
||
Counts
|
||
|
||
Meetings
|
||
Different Dates
|
||
Different Locations
|
||
|
||
Count
|
||
|
||
Selectors
|
||
|
||
TOP SECRET//SI//REL TO USA, FVEY
|
||
|
||
---
|
||
|
||
TOP SECRET//SI//REL TO USA, FVEY
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
SIGDEV
|
||
SIGINT DEVELOPMENT
|
||
|
||
## Target Development with Co-Travelers*
|
||
|
||
Is there a pair traveling together?
|
||
|
||
Manual Process
|
||
|
||
* You could use the same manual process from Meetings, however, this would not find co-travelers on different networks
|
||
* Manual comparison of pairs of **known** selectors is possible with ArcGIS or similar spatial tools - **You** **must know the pairs up front**
|
||
|
||
Cloud Process
|
||
|
||
* Measures miles-per-hour (MPH) between tasked selectors as they move around.
|
||
* Low average MPH = co-traveling.
|
||
* Simply query for your selector to view statistics on average MPH, days calculated, etc.
|
||
|
||
*Also known as "Sidekicks"
|
||
|
||
TOP SECRET//SI//REL TO USA, FVEY
|
||
|
||
---
|
||
|
||
TOP SECRET//SI//REL TO USA, FVEY
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
SIGDEV
|
||
SIGINT DEVELOPMENT
|
||
|
||
## Meeting Network Challenges
|
||
|
||
**Past** Manually query multiple repositories and build network with Analyst Notebook (ANB) - amount of labor can be prohibitive
|
||
|
||
**Now** RT-RG tools exist for contact chaining for selector-to-selector & selector-to-report graphs, with more analytics and tools to come
|
||
|
||
Southwest Asia/Narcotics:
|
||
Senior Traffickers from the AbdulAlim Ishaqzai Narcotics Network Located in HelmandProvince, Afghanistan, After Bahram Chah Raid, 5 November2010 (S//REL TO USA, FVEY)
|
||
|
||
UIM[REDACTED]
|
||
ASKS[REDACTED]
|
||
[REDACTED]IF HE
|
||
HAS HIS MISSING BAG.
|
||
|
||
[REDACTED]
|
||
|
||
[REDACTED]
|
||
|
||
[REDACTED] GETS
|
||
PASSED[REDACTED]
|
||
PHONE NUMBER.
|
||
|
||
Selector-to-Report Graph from Enrichment
|
||
|
||
Search graph...
|
||
|
||
Calls:2
|
||
Calls:3
|
||
|
||
[REDACTED] IMSI)
|
||
|
||
// 2040-06-18 // RC EAST
|
||
TIPPER // OBX GIANT
|
||
SQUID //
|
||
|
||
selectors
|
||
|
||
VF // 2010-06-16 // VHF
|
||
N2K TIPPER // OBJ GIANT
|
||
SQUID //
|
||
|
||
VF//2010-06-15//CSG
|
||
BAGRAM SELECTORS
|
||
TIPPER//OBJ GIANT
|
||
SQUID//
|
||
|
||
calls: 718
|
||
PREPAID PREPAID
|
||
PREPAID
|
||
|
||
THIS TACREP IS REL
|
||
USA AUS,CAN,DEU,DNK,F
|
||
RA,GBR,NLD,NO,NZL,SW
|
||
E
|
||
|
||
OBJ COOPER (OMEGA 10)
|
||
[REDACTED]
|
||
|
||
targets
|
||
|
||
[REDACTED]IMEI)
|
||
|
||
[REDACTED]
|
||
|
||
MAINWAY graph in RTRG UI
|
||
|
||
TOP SECRET//SI//REL TO USA, FVEY
|
||
20
|
||
|
||
---
|
||
|
||
TOP SECRET//SI//REL TO USA, FVEY
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
SIGDEV
|
||
SIGINT DEVELOPMENT
|
||
|
||
## Graph Analysis with Furious Chainsaw
|
||
|
||
* Graph representation is natural for DNR
|
||
* Result is Furious Chainsaw
|
||
* Prototype on Cloudbase
|
||
* Now supports contact chains and trends
|
||
* Will support other graph analytics in the future
|
||
* Triage capability for forward users to complement Enterprise databases
|
||
* Enables chaining and other analytics, provides foundation for graph algorithms
|
||
|
||
Metadata matrix in Cloudbase supports fast graph traversal
|
||
|
||
Graph View in Renoir – but many other analytics are possible
|
||
|
||
TOP SECRET//SI//REL TO USA, FVEY
|
||
21
|
||
|
||
---
|
||
|
||
TOP SECRET//SI//REL TO USA, FVEY
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
SIGDEV
|
||
SIGINT DEVELOPMENT
|
||
|
||
## Graph Triage: Multiple Views
|
||
|
||
Contact Neighborhood
|
||
Ozone Widget
|
||
|
||
Friends and Family
|
||
IMS[REDACTED]
|
||
test
|
||
[REDACTED]
|
||
[REDACTED]
|
||
[REDACTED]
|
||
[REDACTED]
|
||
[REDACTED]
|
||
[REDACTED]
|
||
[REDACTED]
|
||
[REDACTED]
|
||
[REDACTED]
|
||
[REDACTED]
|
||
ANB
|
||
|
||
Daily Call Trends in Panopticon
|
||
PANOPTICON Create Profiles Partners
|
||
Details
|
||
Relationships
|
||
Discussions
|
||
|
||
Bar
|
||
Event Count
|
||
100
|
||
80
|
||
60
|
||
40
|
||
20
|
||
0
|
||
0 1 2 3 4 5 6 7 8 9 1011121314151617181920212223
|
||
Hour of Day (Zulu)
|
||
|
||
Top Associates
|
||
Ozone Widget
|
||
|
||
Top 100 Associations
|
||
|
||
Top 100 Associations | Event Count
|
||
[REDACTED] | 81
|
||
[REDACTED] | 63
|
||
[REDACTED] | 61
|
||
[REDACTED] | 59
|
||
[REDACTED] | 49
|
||
[REDACTED] | 43
|
||
[REDACTED] | 37
|
||
[REDACTED] | 34
|
||
[REDACTED] | 27
|
||
[REDACTED] | 24
|
||
Showing 1 to 10 of 100 entries
|
||
Previous Next
|
||
|
||
Weekly Call Trends
|
||
Ozone Widget
|
||
|
||
Day of Week
|
||
Bar
|
||
Event Count
|
||
200
|
||
150
|
||
100
|
||
50
|
||
0
|
||
Mon
|
||
Tue
|
||
Wed
|
||
Thu
|
||
Fri
|
||
Sat
|
||
Sun
|
||
Day of Week
|
||
|
||
DNR graphs in Furious Chainsaw tables in Cloudbase support a wide range of fast queries and analytics
|
||
|
||
TOP SECRET//SI//REL TO USA, FVEY
|
||
22
|
||
|
||
---
|
||
|
||
TOP SECRET//SI//REL TO USA, FVEY
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
SIGDEV
|
||
SIGINT DEVELOPMENT
|
||
|
||
## Unstructured Data Exploitation
|
||
|
||
SHARKFINN
|
||
|
||
* Selector extraction, normalization, and enrichment
|
||
* Flexible free-text query interface
|
||
* Graph, text, and spreadsheet output formats
|
||
|
||
SKS Structured Knowledge Space
|
||
|
||
* Entity extraction (people, organizations, times, geos)
|
||
* Keyword, faceted, and people search
|
||
* Document clustering
|
||
* Arabic name expansion
|
||
|
||
sharkfury
|
||
|
||
* Integrated data ingest using Niagara Files (NiFi)
|
||
* SharkQuery: search by selector, entity, location, and keyword
|
||
* SharkDocs: query, sharing, and collaboration on user uploaded documents
|
||
* Visualization of results in query overview, table, graph, and map
|
||
* Cloudbase and HDFS for scalable text analytics platform
|
||
|
||
TOP SECRET//SI//REL TO USA, FVEY
|
||
|
||
---
|
||
|
||
TOP SECRET//SI//REL TO USA, FVEY
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
SIGDEV
|
||
SIGINT DEVELOPMENT
|
||
|
||
## Target Development - SharkFinn / SKS
|
||
|
||
SKS
|
||
|
||
"OBJ SMITHERS" AND OBJECTIVES: OBJ TOPPER
|
||
Previous 30 Days
|
||
|
||
1.Perform keyword search with seed OBJ on report library
|
||
|
||
OBJECTIVES
|
||
OBJ SMITHERS (19)
|
||
OBJ TOPPER (19)
|
||
OBJ SPRINGFIELD (12)
|
||
OBJ APU (6)
|
||
OBJ GROUNDSMAN WILLY (6)
|
||
OBJ WIGGUM (4)
|
||
OBJ MAYFIELD (3)
|
||
OBJ NEVIS (1)
|
||
OBJ TRAJAN (1)
|
||
|
||
2. Finding co-occurring objectives
|
||
|
||
SELECTORS
|
||
[REDACTED]
|
||
[REDACTED]
|
||
[REDACTED]
|
||
[REDACTED]
|
||
[REDACTED]
|
||
[REDACTED]
|
||
[REDACTED]
|
||
|
||
3.Selector co-occurrence
|
||
|
||
4.Additional Search & Filtering
|
||
|
||
SELECTOR
|
||
1 [REDACTED]
|
||
2 [REDACTED]
|
||
3 [REDACTED]
|
||
4 [REDACTED]
|
||
5 [REDACTED]
|
||
6 [REDACTED]
|
||
7 [REDACTED]
|
||
8 [REDACTED]
|
||
9 [REDACTED]
|
||
10 [REDACTED]
|
||
11 [REDACTED]
|
||
12 [REDACTED]
|
||
13 [REDACTED]
|
||
|
||
5. Correlate found selectors with Octave/UTT and Panopticon
|
||
|
||
6. Export selector & Report Graph
|
||
|
||
TOP SECRET//SI//REL TO USA, FVEY
|
||
|
||
---
|
||
|
||
TOP SECRET//SI//REL TO USA, FVEY
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
SIGDEV
|
||
SIGINT DEVELOPMENT
|
||
|
||
## Meeting Geolocation Challenges
|
||
|
||
**Past** Analysts manually correlated locations using map viewers or spreadsheets, aggregating data from multiple sources
|
||
|
||
**Now** Analytics and alerts push target information by subscription
|
||
|
||
412.001.00078.27161
|
||
UCELLIDS
|
||
412.001.00076.11402
|
||
/01
|
||
/10
|
||
/16
|
||
/22
|
||
/23
|
||
/24
|
||
/25
|
||
/26
|
||
/27
|
||
/28
|
||
/29
|
||
3/04
|
||
3/05
|
||
3/06
|
||
3/07
|
||
3/08
|
||
3/09
|
||
3/10
|
||
3/11
|
||
3/12
|
||
3/13
|
||
3/14
|
||
3/15
|
||
Day
|
||
RT-RG pattern of life analytic for target location cues
|
||
|
||
tio field
|
||
First Event
|
||
Last Event
|
||
|
||
MAHIM
|
||
Alükh
|
||
Qal'eh-ye
|
||
Adam Khan
|
||
Vazil
|
||
Niazi
|
||
Shiwak
|
||
Shiv
|
||
|
||
Analysts are notified by alerts, based on:
|
||
* Geospatial NAI*
|
||
* Tasking status
|
||
* SMS content
|
||
* Selectors, callsigns, frequencies
|
||
* ...
|
||
|
||
Op ATTENTION INS RC-C Kabul
|
||
(PAN):UNID ASW SUICIDE VEST KABUL 3
|
||
Thu, 10 May 2012 17:48:12 Zulu
|
||
TOP SECRET//COMINT//REL TO USA, AUS,
|
||
CAN, GBR, NZL//20320108
|
||
ASSET: LOPERS
|
||
EVENT TYPE: comms.gsm/call.orig
|
||
SIGAD: US-3309
|
||
PDDG: K2
|
||
SSD: L2
|
||
POEI: (Missing)
|
||
CASN: GCCA0153L5C0001
|
||
SELECTION MODE: STANDARD
|
||
COLLECTION AUTHORITY: EO12333
|
||
Target Location
|
||
Approx. Geo: 42S WD 21111 18789
|
||
UCELLID: [REDACTED]
|
||
Geo Data Source(s): GSM Geo-Sigdev
|
||
4 Item(s)
|
||
|
||
25
|
||
TOP SECRET//SI//REL TO USA, FVEY
|
||
* NAI = Named Area of Interest
|
||
|
||
---
|
||
|
||
TOP SECRET//SI//REL TO USA, FVEY
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
SIGDEV
|
||
SIGINT DEVELOPMENT
|
||
|
||
## Target Geolocation with Bed Down
|
||
|
||
Find the most consistent location of the day's first/last event
|
||
|
||
First Events
|
||
+
|
||
Last Events
|
||
=
|
||
Bed Down
|
||
|
||
Manual Process - One Selector At a Time
|
||
|
||
* Query all events for your selector.
|
||
* Mark first and last events manually.
|
||
* OR
|
||
* Enter in a tool like CheekyMonkey to view gaps in activity.
|
||
* Slow process to do one selector at a time
|
||
|
||
Cloud Process- All Tasked Selectors
|
||
|
||
* Pre-calculates first and last events in local time for ALL selectors.
|
||
* Will calculate estimated Bed Down at query time.
|
||
* Can query multiple selectors in seconds, find common overlap.
|
||
|
||
TOP SECRET//SI//REL TO USA, FVEY
|
||
|
||
---
|
||
|
||
TOP SECRET//SI//REL TO USA, FVEY
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
SIGDEV
|
||
SIGINT DEVELOPMENT
|
||
|
||
## Target Geolocation with LayerCake
|
||
|
||
LayerCake - find geospatial overlap of a set of targets.
|
||
|
||
Manual Process
|
||
|
||
* Query all events for your selectors.
|
||
* Display events spatially on mapping software (impossible to view polygon overlaps).
|
||
* Rasterize and do "raster math” to determine max overlap. (very complex and expensive task in most GIS tools).
|
||
|
||
Cloud Process
|
||
|
||
* Pre-calculates unique locations visited for ALL selectors.
|
||
* Raster heat maps drawn at query time.
|
||
|
||
TOP SECRET//SI//REL TO USA, FVEY
|
||
|
||
---
|
||
|
||
TOP SECRET//SI// REL USA,FVEY
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
SIGDEV
|
||
SIGINT DEVELOPMENT
|
||
|
||
## Spatial Visualization of Data
|
||
|
||
Using MapReduce, analytics can count and aggregate over large number of daily events to give a multi-resolution spatial visualization of the data
|
||
|
||
Google Earth
|
||
|
||
Cloudbase Tables for Geo Data
|
||
|
||
MapReduce for Binning, Counting
|
||
|
||
Event Data
|
||
|
||
Analytic Data Flow
|
||
|
||
Synoptic View
|
||
|
||
TS//SI//REL
|
||
|
||
Call volume map of RTRG (AF1) collection (A day in Dec, 2011)
|
||
|
||
Fewer DNR events
|
||
|
||
Fewer events
|
||
|
||
28
|
||
TOP SECRET//SI// REL USA, FVEY
|
||
|
||
---
|
||
|
||
TOP SECRET//SI // REL USA, FVEY
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
SIGDEV
|
||
SIGINT DEVELOPMENT
|
||
|
||
## Spatial Visualization of Data
|
||
|
||
Cloud enabled analytic allows viewing the spatial data at many levels-of-detail
|
||
|
||
“Meso-scale” View
|
||
|
||
Detail View – “Mashup” with heatmap
|
||
|
||
Kabul
|
||
Peshawar
|
||
Rawalpindi
|
||
Hyderabad
|
||
Karachi
|
||
SEDB
|
||
Faisalabad
|
||
Amritsar
|
||
Lahore
|
||
Srinagar
|
||
|
||
Heatmap of activity of a set of targets
|
||
|
||
[REDACTED]
|
||
ASSOC OF CH
|
||
[REDACTED]
|
||
NDS D124 DEP CHIEF[REDACTED]
|
||
ΜΟΙ ΖΟΝΕ 101 AS
|
||
KARZAI
|
||
AIDE TO MO
|
||
OCT 829
|
||
UPS AND INDIV
|
||
MOI CT OFFI
|
||
|
||
وز محشر شفا خواهی تمام مومنین است.
|
||
We will arrive on saturday
|
||
محمد در عرب سردار دین است چراغ آسمان
|
||
شمع زمین است. به فردای قیامت و
|
||
|
||
And Z the day of judgment
|
||
healing desiring all believers is.
|
||
We+will+arrive+on+saturday+
|
||
[REDACTED]
|
||
commander religion is: light
|
||
heaven/sky candle land is: to the
|
||
resurrection-day R
|
||
Directions: To here - From here
|
||
|
||
MFA
|
||
MOD SECRETARIAT
|
||
MP
|
||
DEVELOPMENT ICW IRGC QF DEPT 409
|
||
KHALEQ
|
||
MP
|
||
|
||
Using Cybertrans translation service on SMS messages, integrated with heatmaps
|
||
|
||
Fewer calls
|
||
More calls
|
||
|
||
TOP SECRET//SI // REL USA, FVEY
|
||
29
|
||
|
||
---
|
||
|
||
UNCLASSIFIED//FOR OFFICIAL USE ONLY
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
SIGDEV
|
||
SIGINT DEVELOPMENT
|
||
|
||
## Outline
|
||
|
||
* RTRG Mission Overview
|
||
* RTRG System: Today and Tomorrow
|
||
* Target-Centric and Network-Centric Cloud Analytics
|
||
* Future Work
|
||
|
||
30
|
||
UNCLASSIFIED//FOR OFFICIAL USE ONLY
|
||
|
||
---
|
||
|
||
TOP SECRET//SI//REL TO USA, FVEY
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
SIGDEV
|
||
SIGINT DEVELOPMENT
|
||
|
||
## RTRG Tomorrow
|
||
|
||
* Improved DNI capabilities; focus on convergence
|
||
* Integrating active SIGINT capabilities
|
||
* Increased CT and expeditionary capabilities
|
||
* Better tools for faster analytic development
|
||
* Incorporation of content analysis and HLT capabilities
|
||
* Improved integration between target and population analytics
|
||
|
||
31
|
||
TOP SECRET//SI//REL TO USA, FVEY
|
||
|
||
---
|
||
|
||
TOP SECRET//SI// REL USA, FVEY
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
SIGDEV
|
||
SIGINT DEVELOPMENT
|
||
|
||
## RTRG Planned Cloud Instances
|
||
|
||
NSA/CSS GEORGIA
|
||
FORT GORDON
|
||
|
||
gmSeminole
|
||
NSA Georgia
|
||
|
||
NAVY INFORMATION
|
||
OPERATIONS COMMAND BAHRAIN
|
||
NETWARCOM
|
||
|
||
gmGulf
|
||
NIOC-Bahrain
|
||
|
||
NCR AFGHANISTAN
|
||
SEMPER VIGILANS SEMPER PARATOS SEMPER VOLENS
|
||
|
||
gmZilla
|
||
Kabul
|
||
|
||
TOP SECRET//SI// REL USA, FVEY
|
||
32
|
||
|
||
---
|
||
|
||
TOP SECRET//SI//REL USA,FVEY
|
||
|
||
NATIONAL SECURITY AGENCY
|
||
SIGDEV
|
||
SIGINT DEVELOPMENT
|
||
|
||
## Summary
|
||
|
||
* RTRG has been a successful regional data store and exploitation system for COIN, CIED and other missions
|
||
* Moving to NSA Cloud infrastructure
|
||
* More historical data
|
||
* Deeper analysis using parallel programs
|
||
* Allows for more flexible deployments to IC, DoD service installations
|
||
* Continuing to support advanced analytics for current and future operations
|
||
|
||
TOP SECRET//SI//REL USA, FVEY
|
||
33 |