24 KiB
TOP SECRET //SI// REL USA,FVEY
Real-Time Regional Gateway Cloud Analytics for Forward Users
NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT
[REDACTED]
TOP SECRET//SI//REL USA, FVEY
REAL-TIME REGIONAL GATEWAY
SECRET//REL USA, FVEY
NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT
BLUF
RTRG:
...brings near real-time intelligence to the warfighter
..."grew up" supporting operations in Iraq
...RTRG is now a global architecture
...leveraging the emerging cloud architecture to answer questions we have not been able to do before
2 SECRET//REL USA, FVEY
TOP SECRET//SI//REL USA, FVEY
NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT
RTRG in Afghanistan
(U)
(S//REL)
(S//REL) Area 82 – Bagram Air Base Home of RTRG AF1 & gmTote
- 20 days data retention from AFPAK
- 200-600 daily users
- New upgrades include two systems in Kabul
(TS//REL)
Mission Areas: Tracking high-value targets (HVT), Counter-Insurgency (COIN), Counter-IED (CIED)
Organizations Using RTRG:
- CSG* Afghanistan
- U.S. Marine Corps (USMC) 1st and 2nd Radio Battalions
- U.S. Army SIGINT analysts at BCT* level
- U.S. Air Force National Tactical Integration
- Jalalabad Fusion Cell (USMC)
- S2 TOPI
- NSA-G SWAN Counternarcotics Team
- All special operations task forces
(TS//SI//REL)
“RTRG is the most significant SIGINT support to the war fighter in the last decade"
- General David Petraeus
"USSOCOM has enduring and critical needs for the tools and data that RT-RG provides"
- Admiral William McRaven
*CST- Cryptologic Support Team *BCT - Brigade Combat Team *CSG - Cryptologic Support Group
TOP SECRET//SI//REL USA, FVEY 3
TOP SECRET//SI//REL USA,FVEY
NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT
CSG Afghanistan Statistics
- In 2011, RTRG in Afghanistan
- Played a key role in 90% of all SIGINT developed operations
- Leading to 2270 capture/kill operations
- 6534 enemies killed in action
- 1117 detainees
4 TOP SECRET//SI//REL USA, FVEY
TOP SECRET//SI//REL USA, FVEY
NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT
RTRG in the Gulf and Horn of Africa
Monitoring Iranian Navy (IRIN) in Straits of Hormuz
QESHM ISLAND HIDARABI ISLAND HENGAM ISLAND BAR-E-LANGEH FARUR ISLAND GREATER TUNB BANI FARUR ABU MUSA JASH USS GEORGIA SSGN 729
Photo of IRIN vessel from US Navy aircraft, located by RTRG
Supporting CENTCOM Maritime (NAVCENT) Navy Information Operations Command- Bahrain (NIOC-B)
Missions supported:
- Iran, Yemen, Persian Gulf
- Recent successes include monitoring of Iranian naval assets
(TS//SI//REL)
RTRG Afloat on subsurface platform USS Georgia (SSGN-729)
(TS//SI//REL) Missions supported:
- Horn of Africa: In first week of mission, system received 31 million GSM events, leading to 10 high-value target voice ID, and 90 tactical tip-offs
TOP SECRET//SI//REL USA, FVEY 5
TOP SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT
RTRG Global Operations
NSA/CSS COLORADO TECHNICAL SIGINT (U//FOUO)
(U//FOUO)
US-2 - Denver Global Maritime & ELINT
NATIONAL SECURITY AGENCY UNITED STATES OF AMERICA
US-1 Fort Meade Mission Assurance
IQ2 Iraq* COIN, CIED
AF-1,5 Bagram & Kabul COIN,CIED
(S//REL)
USS Georgia & USS Florida
US-3 JFCOM Joint Forces Command
BH-1 NIOC** Bahrain AFRICOM, EUCOM, PACOM
GE-3 Germany** AFPAK Continuity-of-Operations
KO-1,2 USFK** Pangyo PACOM & North Korea
US-8 NSA-Texas Counternarcotics, Maritime, Mexico & SOUTHCOM Support
Texas Cryptologic Center NSA/CSS TEXAS
* In draw-down ** ECC - NSA European Technical Center USFK-U.S. Forces, Korea NIOC - Navy Information Operations Command
TOP SECRET//SI//REL TO USA, FVEY 6
UNCLASSIFIED//FOR OFFICIAL USE ONLY
NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT
Outline
- RTRG Mission Overview
- RTRG System: Today and Tomorrow
- Target-Centric and Network-Centric Cloud Analytics
- Future Work
7 UNCLASSIFIED//FOR OFFICIAL USE ONLY
SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT
Current RTRG Architecture
RTRG System
Goldminer Metadata Search
GeoT Geospatial Tools
Agent Logic Alerting Tools
Sharkfinn Selector Enrichment
SKS Report / Doc Manager
Panopticon Target Management
Supporting Tactical Users
Forward Data Centers
Services Layer: Web services, authentication, auditing
Publish and Subscribe Messaging
Oracle relational database & dimensional data model
Ingest and Enrichment Pipeline: Flexible, high-speed architecture for parser and data processors
Data Feeds:
- DRT
- JUGGERNAUT
- CIDNE
- LOPERS
- TALIS/MATTERHORN
- AIRHANDLER
- LYCANTHROPE
- KL
- VOICESAIL
- RETURNSPRING ... and a growing list of others*
(DNR, DNI collect, tipping and reporting)
A successful architecture for several years. Demand for more data feeds, longer retention, and data-intensive analytics has driven RTRG to seek new solutions
*Based on Afghanistan RTRG data flow
SECRET//SI//REL TO USA, FVEY
SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT
RTRG Data Challenges
Current Challenges
-
Data Storage & Retention
- "Patterns of Life” analysis needs require 6+ months of data from world-wide collection
- A typical system has capacity for only 4-6 weeks of regional data (~90% user queries are within seven days of “now”)
- Data Use & Computation
- Analytic processes should make maximum use of all available data to find small signals
- Relational databases are unsuited to sophisticated analytics such as correlation and matching
-
Data & Technology Heterogeneity
- New types of data must be added to the system continually
- With traditional databases, schema modifications are difficult
- Exotic data management solutions are difficult to adopt due to limited expertise
SECRET//SI//REL TO USA, FVEY
UNCLASSIFIED//FOR OFFICIAL USE ONLY
NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT
Cloud Architectures for Analytics
NATIONAL SECURITY AGENCY UNITED STATES OF AMERICA
hadoop
Apache
Cloudbase Scalable BigTable Implementation with Security
Emerging NSA Cloud Reference Architecture is well-suited for developing analytics on intelligence data
Scalable: Distributed file systems and databases are built on clusters of commodity hardware, leveraging open source projects and industrial solutions
Computable: The MapReduce programming model simplifies writing efficient parallel computations that operate over large volumes of data
Flexible: Cloud technologies enable flexible schema and leverage large open-source efforts
UNCLASSIFIED//FOR OFFICIAL USE ONLY
TOP SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT
Scalability & Computation
Data Challenges in AFPAK
Current RTRG (AF1)
- Current database is 27 terabytes (TB)
- Retention is ~30 days
Future Cloud enabled system
- Even a modest cloud system (3 rack) for storage will be at least 125 TB of storage
- 5x increase in available space
- Actual retention improvement depends on how the space resources are allocated
Cloud supports more data feeds & more days of historical data
Analytic Challenges from Iraq & AFPAK
- Many analytics used by RTRG are based on R6 SORTINGLEAD event summaries
- Event summaries were originally created on relational databases
- Collection increased dramatically, and a mapreduce implementation was needed
- For new analytics with present day collection volumes, a practical parallel execution model is crucial
Cloud supports large-scale analytics
TOP SECRET//SI//REL TO USA, FVEY 11
TOP SECRET//SI//REL USA, FVEY
NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT
NSA Cloud Computing Enterprise
- 5-12 racks commodity hardware
- 150+ data nodes
- 16 GB RAM each
- Apache Hadoop
- 100s of terabytes of storage
- Stores 10s to 100s of billions of events
- NSA Cloudbase
- 100s of nodes serving BigTable implementation
- Stores 100s of billions of entries
gmAPPLE Yakima
gmCAVE gmPEACH Utah Language Center
gmDEN ADF-C
gmMOUNT
gmLIGHT gmBALTIC gmPLACE gmPARK gmHALO gmALAMO gmCARBON
Europe
gmROYALE gmREGAL gmGEIST
Iraq
gmIDEA (Camp Victory)
Afghanistan
gmTOTE (Bagram)
Korea & Japan gmPEN S. Korea
gmPAC Misawa, Japan
gmCALM Camp Humphreys S. Korea
Australia
gmMATE (S//REL) (Alice Springs)
GHOSTMACHINE is a data-intensive cloud system with many fielded instances worldwide
Hadoop clusters have been demonstrated with 10s of petabytes and over 10,000 cores
12 Map does not include all GHOSTMACHINE/SiteStore systems TOP SECRET//SI//REL USA, FVEY
TOP SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT
Today's "Hybrid” Architecture
GTM GeoT SKS PANOPTICON Applications
Services Layer
Publish and Subscribe Messaging
Relational Database
Ingest and Enrichment
hadoop Large scale data preprocessing and analysis
Cloudbase Scalable BigTable Implementation with Security Massive storage and fast record lookup
GHOSTMACHINE
RTRG and GHOSTMACHINE systems are paired with one another: MapReduce analytic results are fed back to RTRG relational database
TOP SECRET//SI//REL TO USA, FVEY 13
TOP SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT
Tomorrow's Architecture
User interfaces
- W WidgetFramework
- GTM
- GeoT
- CHET
- H&S
- SKS
- PANOPTICON
- Web Tier
Services
- Application Services
- Alerting
- Ingest
- Real time Enrichment
- Analytics
- Streaming-based
- Analytics
- Map Reduce-based
Data
- Data Storage (NFS)
- Secure Data Access Services
- Casport, Wavelegal, etc.
- Relational Store
- Enrichment, User, & Auxiliary Data
- Cloudbase
- Event Data and Analytic Results
Operating System
- Java VM
- Java VM
- Operating System
- Hardened RH Linux
- Operating System
- Hardened RH Linux
- OpenStack VM
- Utility Cloud
- Hardware Layer
- Data/Analytic Cloud
- Hardware Layer
Data Storage (HDFS)
The Cloud will bring new data-intensive capabilities, support existing missions, and align RTRG installations with emerging NSA and IC standards
TOP SECRET//SI//REL TO USA, FVEY
UNCLASSIFIED//FOR OFFICIAL USE ONLY
NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT
Outline
- RTRG Mission Overview
- RTRG System: Today and Tomorrow
- Target-Centric and Network-Centric Cloud Analytics
- Future Work
15 UNCLASSIFIED//FOR OFFICIAL USE ONLY
TOP SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT
Analytics Overview
Target Development
Co-Travelers & Meetings Contact Similarity High Priority Untasked Numbers
Graph & Network
Graph Triage Handset Swap Target Enrichment & Disambiguation
Beddown Layercake Heatmaps
Geo-Spatial
The data-intensive computing capabilities in the system enables a set of graph/network analytics and target development analytics
TOP SECRET//SI//REL TO USA, FVEY 16
TOP SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT
Meeting Target Development Challenges
Past Analysts manually queried multiple, independent repositories, aggregating results in Excel, taking hours or work for search and refinement
Now RT-RG provides a streamlined, integrated workflow saving analyst effort
Tips
Selectors
GIST Report
Detainee Reports
Target Development
Event Type comms.gsm Event Subtype call.* Selectors IMSI:12345
NAI Kandahar ?
Submit Query Cancel Query Export To File...
Metadata Search
[REDACTED] Profile Reports Network Changelog Recommendations OBJ DEWEY BEACH ASSOCIATE Bottom Line Pech Valley INS possibly in charge of a "company" sized element. Led the attack Last Active: (TS//SI//REL TO USA, FVEY) (IMEI) 9 hours ago Last Location: (TS//SI//REL TO USA, FVEY) (IMEI) RTRG-Afghanistan Targets 9 hours ago Geo Processor: AROCC Last Report: (TS//SI//REL TO USA, FVEY) (MSISDN) view report 5 months ago VID/SID: 1 year ago
Collaboration
TOP SECRET//COMINT//REL TO U CST 17 FOB FENTY (PAN): AF IN (Khaksar) Thu, 10 May 2012 15:55:20 Zu TOP SECRET//COMINT//REL TO NZL//20320108 ASSET: JUGGERNAUT
Geospatial Alerting
Afghanistan/Insurgency: Taliban Leaders in Chaman, Pakistan, Command Insurgent Operations throughout Southern Afghanistan-A SIGINT Assessment (S// SI//REL TO USA, FVEY)
Afghanistan: Influence of Chamay on RC South, Juge-July2010
IIR 6 223 0164 12/DOME EXPOSE/DRAGON FIRE- EXPLOITATION OFPAGE 3
Report Text for: 02992282 2017302 AUG 10 FM USJ-800 T O P S E C R E T UMBRA US/UK/CAN/AUS/NZ EYES ONLY SERIAL: 3/CY/507710-10 SUBJ: Afghanistan/Insurgency: Taliban Leaders in Chaman, Pakistan, Command Insurgent Operations throughout Southem Afghanistan A SIGINT Assessment (S//SI//REL TO USA, FVEY)
20100119 08) BIMINI (10020-3AFG) _Nokia1650
kc taliban ircurg
Enrichment
Outcomes
Geolocation
Capture
TOP SECRET//SI//REL TO USA, FVEY 17
TOP SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT
Target Development with Meetings
Who is at the same UCELLID at the same time?
Manual Process
- Take your selector and query for every unique location he has been and at what time
- Query for other selectors who have been at the same places at the same times (impossible or painful)
- OR compare to another known set of selectors to find overlap (excel / ArcGIS / JEMA) (limiting to what you know)
- Summary statistics on the matching IMSIs using excel or ArcGIS
Cloud Process
- Pre-calculates all UCELLID overlaps between tasked selectors
- Simply query your selector in cloud-generated QFD and view summary statistics
Counts
Meetings Different Dates Different Locations
Count
Selectors
TOP SECRET//SI//REL TO USA, FVEY
TOP SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT
Target Development with Co-Travelers*
Is there a pair traveling together?
Manual Process
- You could use the same manual process from Meetings, however, this would not find co-travelers on different networks
- Manual comparison of pairs of known selectors is possible with ArcGIS or similar spatial tools - You must know the pairs up front
Cloud Process
- Measures miles-per-hour (MPH) between tasked selectors as they move around.
- Low average MPH = co-traveling.
- Simply query for your selector to view statistics on average MPH, days calculated, etc.
*Also known as "Sidekicks"
TOP SECRET//SI//REL TO USA, FVEY
TOP SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT
Meeting Network Challenges
Past Manually query multiple repositories and build network with Analyst Notebook (ANB) - amount of labor can be prohibitive
Now RT-RG tools exist for contact chaining for selector-to-selector & selector-to-report graphs, with more analytics and tools to come
Southwest Asia/Narcotics: Senior Traffickers from the AbdulAlim Ishaqzai Narcotics Network Located in HelmandProvince, Afghanistan, After Bahram Chah Raid, 5 November2010 (S//REL TO USA, FVEY)
UIM[REDACTED] ASKS[REDACTED] [REDACTED]IF HE HAS HIS MISSING BAG.
[REDACTED]
[REDACTED]
[REDACTED] GETS PASSED[REDACTED] PHONE NUMBER.
Selector-to-Report Graph from Enrichment
Search graph...
Calls:2 Calls:3
[REDACTED] IMSI)
// 2040-06-18 // RC EAST TIPPER // OBX GIANT SQUID //
selectors
VF // 2010-06-16 // VHF N2K TIPPER // OBJ GIANT SQUID //
VF//2010-06-15//CSG BAGRAM SELECTORS TIPPER//OBJ GIANT SQUID//
calls: 718 PREPAID PREPAID PREPAID
THIS TACREP IS REL USA AUS,CAN,DEU,DNK,F RA,GBR,NLD,NO,NZL,SW E
OBJ COOPER (OMEGA 10) [REDACTED]
targets
[REDACTED]IMEI)
[REDACTED]
MAINWAY graph in RTRG UI
TOP SECRET//SI//REL TO USA, FVEY 20
TOP SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT
Graph Analysis with Furious Chainsaw
- Graph representation is natural for DNR
- Result is Furious Chainsaw
- Prototype on Cloudbase
- Now supports contact chains and trends
- Will support other graph analytics in the future
- Triage capability for forward users to complement Enterprise databases
- Enables chaining and other analytics, provides foundation for graph algorithms
Metadata matrix in Cloudbase supports fast graph traversal
Graph View in Renoir – but many other analytics are possible
TOP SECRET//SI//REL TO USA, FVEY 21
TOP SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT
Graph Triage: Multiple Views
Contact Neighborhood Ozone Widget
Friends and Family IMS[REDACTED] test [REDACTED] [REDACTED] [REDACTED] [REDACTED] [REDACTED] [REDACTED] [REDACTED] [REDACTED] [REDACTED] [REDACTED] ANB
Daily Call Trends in Panopticon PANOPTICON Create Profiles Partners Details Relationships Discussions
Bar Event Count 100 80 60 40 20 0 0 1 2 3 4 5 6 7 8 9 1011121314151617181920212223 Hour of Day (Zulu)
Top Associates Ozone Widget
Top 100 Associations
Top 100 Associations | Event Count [REDACTED] | 81 [REDACTED] | 63 [REDACTED] | 61 [REDACTED] | 59 [REDACTED] | 49 [REDACTED] | 43 [REDACTED] | 37 [REDACTED] | 34 [REDACTED] | 27 [REDACTED] | 24 Showing 1 to 10 of 100 entries Previous Next
Weekly Call Trends Ozone Widget
Day of Week Bar Event Count 200 150 100 50 0 Mon Tue Wed Thu Fri Sat Sun Day of Week
DNR graphs in Furious Chainsaw tables in Cloudbase support a wide range of fast queries and analytics
TOP SECRET//SI//REL TO USA, FVEY 22
TOP SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT
Unstructured Data Exploitation
SHARKFINN
- Selector extraction, normalization, and enrichment
- Flexible free-text query interface
- Graph, text, and spreadsheet output formats
SKS Structured Knowledge Space
- Entity extraction (people, organizations, times, geos)
- Keyword, faceted, and people search
- Document clustering
- Arabic name expansion
sharkfury
- Integrated data ingest using Niagara Files (NiFi)
- SharkQuery: search by selector, entity, location, and keyword
- SharkDocs: query, sharing, and collaboration on user uploaded documents
- Visualization of results in query overview, table, graph, and map
- Cloudbase and HDFS for scalable text analytics platform
TOP SECRET//SI//REL TO USA, FVEY
TOP SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT
Target Development - SharkFinn / SKS
SKS
"OBJ SMITHERS" AND OBJECTIVES: OBJ TOPPER Previous 30 Days
1.Perform keyword search with seed OBJ on report library
OBJECTIVES OBJ SMITHERS (19) OBJ TOPPER (19) OBJ SPRINGFIELD (12) OBJ APU (6) OBJ GROUNDSMAN WILLY (6) OBJ WIGGUM (4) OBJ MAYFIELD (3) OBJ NEVIS (1) OBJ TRAJAN (1)
- Finding co-occurring objectives
SELECTORS [REDACTED] [REDACTED] [REDACTED] [REDACTED] [REDACTED] [REDACTED] [REDACTED]
3.Selector co-occurrence
4.Additional Search & Filtering
SELECTOR 1 [REDACTED] 2 [REDACTED] 3 [REDACTED] 4 [REDACTED] 5 [REDACTED] 6 [REDACTED] 7 [REDACTED] 8 [REDACTED] 9 [REDACTED] 10 [REDACTED] 11 [REDACTED] 12 [REDACTED] 13 [REDACTED]
-
Correlate found selectors with Octave/UTT and Panopticon
-
Export selector & Report Graph
TOP SECRET//SI//REL TO USA, FVEY
TOP SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT
Meeting Geolocation Challenges
Past Analysts manually correlated locations using map viewers or spreadsheets, aggregating data from multiple sources
Now Analytics and alerts push target information by subscription
412.001.00078.27161 UCELLIDS 412.001.00076.11402 /01 /10 /16 /22 /23 /24 /25 /26 /27 /28 /29 3/04 3/05 3/06 3/07 3/08 3/09 3/10 3/11 3/12 3/13 3/14 3/15 Day RT-RG pattern of life analytic for target location cues
tio field First Event Last Event
MAHIM Alükh Qal'eh-ye Adam Khan Vazil Niazi Shiwak Shiv
Analysts are notified by alerts, based on:
- Geospatial NAI*
- Tasking status
- SMS content
- Selectors, callsigns, frequencies
- ...
Op ATTENTION INS RC-C Kabul (PAN):UNID ASW SUICIDE VEST KABUL 3 Thu, 10 May 2012 17:48:12 Zulu TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108 ASSET: LOPERS EVENT TYPE: comms.gsm/call.orig SIGAD: US-3309 PDDG: K2 SSD: L2 POEI: (Missing) CASN: GCCA0153L5C0001 SELECTION MODE: STANDARD COLLECTION AUTHORITY: EO12333 Target Location Approx. Geo: 42S WD 21111 18789 UCELLID: [REDACTED] Geo Data Source(s): GSM Geo-Sigdev 4 Item(s)
25 TOP SECRET//SI//REL TO USA, FVEY
- NAI = Named Area of Interest
TOP SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT
Target Geolocation with Bed Down
Find the most consistent location of the day's first/last event
First Events + Last Events
Bed Down
Manual Process - One Selector At a Time
- Query all events for your selector.
- Mark first and last events manually.
- OR
- Enter in a tool like CheekyMonkey to view gaps in activity.
- Slow process to do one selector at a time
Cloud Process- All Tasked Selectors
- Pre-calculates first and last events in local time for ALL selectors.
- Will calculate estimated Bed Down at query time.
- Can query multiple selectors in seconds, find common overlap.
TOP SECRET//SI//REL TO USA, FVEY
TOP SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT
Target Geolocation with LayerCake
LayerCake - find geospatial overlap of a set of targets.
Manual Process
- Query all events for your selectors.
- Display events spatially on mapping software (impossible to view polygon overlaps).
- Rasterize and do "raster math” to determine max overlap. (very complex and expensive task in most GIS tools).
Cloud Process
- Pre-calculates unique locations visited for ALL selectors.
- Raster heat maps drawn at query time.
TOP SECRET//SI//REL TO USA, FVEY
TOP SECRET//SI// REL USA,FVEY
NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT
Spatial Visualization of Data
Using MapReduce, analytics can count and aggregate over large number of daily events to give a multi-resolution spatial visualization of the data
Google Earth
Cloudbase Tables for Geo Data
MapReduce for Binning, Counting
Event Data
Analytic Data Flow
Synoptic View
TS//SI//REL
Call volume map of RTRG (AF1) collection (A day in Dec, 2011)
Fewer DNR events
Fewer events
28 TOP SECRET//SI// REL USA, FVEY
TOP SECRET//SI // REL USA, FVEY
NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT
Spatial Visualization of Data
Cloud enabled analytic allows viewing the spatial data at many levels-of-detail
“Meso-scale” View
Detail View – “Mashup” with heatmap
Kabul Peshawar Rawalpindi Hyderabad Karachi SEDB Faisalabad Amritsar Lahore Srinagar
Heatmap of activity of a set of targets
[REDACTED] ASSOC OF CH [REDACTED] NDS D124 DEP CHIEF[REDACTED] ΜΟΙ ΖΟΝΕ 101 AS KARZAI AIDE TO MO OCT 829 UPS AND INDIV MOI CT OFFI
وز محشر شفا خواهی تمام مومنین است. We will arrive on saturday محمد در عرب سردار دین است چراغ آسمان شمع زمین است. به فردای قیامت و
And Z the day of judgment healing desiring all believers is. We+will+arrive+on+saturday+ [REDACTED] commander religion is: light heaven/sky candle land is: to the resurrection-day R Directions: To here - From here
MFA MOD SECRETARIAT MP DEVELOPMENT ICW IRGC QF DEPT 409 KHALEQ MP
Using Cybertrans translation service on SMS messages, integrated with heatmaps
Fewer calls More calls
TOP SECRET//SI // REL USA, FVEY 29
UNCLASSIFIED//FOR OFFICIAL USE ONLY
NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT
Outline
- RTRG Mission Overview
- RTRG System: Today and Tomorrow
- Target-Centric and Network-Centric Cloud Analytics
- Future Work
30 UNCLASSIFIED//FOR OFFICIAL USE ONLY
TOP SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT
RTRG Tomorrow
- Improved DNI capabilities; focus on convergence
- Integrating active SIGINT capabilities
- Increased CT and expeditionary capabilities
- Better tools for faster analytic development
- Incorporation of content analysis and HLT capabilities
- Improved integration between target and population analytics
31 TOP SECRET//SI//REL TO USA, FVEY
TOP SECRET//SI// REL USA, FVEY
NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT
RTRG Planned Cloud Instances
NSA/CSS GEORGIA FORT GORDON
gmSeminole NSA Georgia
NAVY INFORMATION OPERATIONS COMMAND BAHRAIN NETWARCOM
gmGulf NIOC-Bahrain
NCR AFGHANISTAN SEMPER VIGILANS SEMPER PARATOS SEMPER VOLENS
gmZilla Kabul
TOP SECRET//SI// REL USA, FVEY 32
TOP SECRET//SI//REL USA,FVEY
NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT
Summary
- RTRG has been a successful regional data store and exploitation system for COIN, CIED and other missions
- Moving to NSA Cloud infrastructure
- More historical data
- Deeper analysis using parallel programs
- Allows for more flexible deployments to IC, DoD service installations
- Continuing to support advanced analytics for current and future operations
TOP SECRET//SI//REL USA, FVEY 33