TOP SECRET //SI// REL USA,FVEY # Real-Time Regional Gateway Cloud Analytics for Forward Users NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT [REDACTED] TOP SECRET//SI//REL USA, FVEY REAL-TIME REGIONAL GATEWAY --- SECRET//REL USA, FVEY NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT ## BLUF RTRG: ...brings near real-time intelligence to the warfighter ..."grew up" supporting operations in Iraq ...RTRG is now a global architecture ...leveraging the emerging cloud architecture to answer questions we have not been able to do before 2 SECRET//REL USA, FVEY --- TOP SECRET//SI//REL USA, FVEY NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT ## RTRG in Afghanistan (U) (S//REL) (S//REL) Area 82 – Bagram Air Base Home of RTRG AF1 & gmTote * 20 days data retention from AFPAK * 200-600 daily users * New upgrades include two systems in Kabul (TS//REL) Mission Areas: Tracking high-value targets (HVT), Counter-Insurgency (COIN), Counter-IED (CIED) Organizations Using RTRG: * CSG* Afghanistan * U.S. Marine Corps (USMC) 1st and 2nd Radio Battalions * U.S. Army SIGINT analysts at BCT* level * U.S. Air Force National Tactical Integration * Jalalabad Fusion Cell (USMC) * S2 TOPI * NSA-G SWAN Counternarcotics Team * All special operations task forces (TS//SI//REL) “RTRG is the most significant SIGINT support to the war fighter in the last decade" - General David Petraeus "USSOCOM has enduring and critical needs for the tools and data that RT-RG provides" - Admiral William McRaven *CST- Cryptologic Support Team *BCT - Brigade Combat Team *CSG - Cryptologic Support Group TOP SECRET//SI//REL USA, FVEY 3 --- TOP SECRET//SI//REL USA,FVEY NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT ## CSG Afghanistan Statistics * In 2011, RTRG in Afghanistan * Played a key role in 90% of all SIGINT developed operations * Leading to 2270 capture/kill operations * 6534 enemies killed in action * 1117 detainees 4 TOP SECRET//SI//REL USA, FVEY --- TOP SECRET//SI//REL USA, FVEY NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT ## RTRG in the Gulf and Horn of Africa Monitoring Iranian Navy (IRIN) in Straits of Hormuz QESHM ISLAND HIDARABI ISLAND HENGAM ISLAND BAR-E-LANGEH FARUR ISLAND GREATER TUNB BANI FARUR ABU MUSA JASH USS GEORGIA SSGN 729 Photo of IRIN vessel from US Navy aircraft, located by RTRG Supporting CENTCOM Maritime (NAVCENT) Navy Information Operations Command- Bahrain (NIOC-B) Missions supported: * Iran, Yemen, Persian Gulf * Recent successes include monitoring of Iranian naval assets (TS//SI//REL) RTRG Afloat on subsurface platform USS Georgia (SSGN-729) (TS//SI//REL) Missions supported: * Horn of Africa: In first week of mission, system received 31 million GSM events, leading to 10 high-value target voice ID, and 90 tactical tip-offs TOP SECRET//SI//REL USA, FVEY 5 --- TOP SECRET//SI//REL TO USA, FVEY NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT ## RTRG Global Operations NSA/CSS COLORADO TECHNICAL SIGINT (U//FOUO) (U//FOUO) US-2 - Denver Global Maritime & ELINT NATIONAL SECURITY AGENCY UNITED STATES OF AMERICA US-1 Fort Meade Mission Assurance IQ2 Iraq* COIN, CIED AF-1,5 Bagram & Kabul COIN,CIED (S//REL) USS Georgia & USS Florida US-3 JFCOM Joint Forces Command BH-1 NIOC** Bahrain AFRICOM, EUCOM, PACOM GE-3 Germany** AFPAK Continuity-of-Operations KO-1,2 USFK** Pangyo PACOM & North Korea US-8 NSA-Texas Counternarcotics, Maritime, Mexico & SOUTHCOM Support Texas Cryptologic Center NSA/CSS TEXAS \* In draw-down \** ECC - NSA European Technical Center USFK-U.S. Forces, Korea NIOC - Navy Information Operations Command TOP SECRET//SI//REL TO USA, FVEY 6 --- UNCLASSIFIED//FOR OFFICIAL USE ONLY NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT ## Outline * RTRG Mission Overview * RTRG System: Today and Tomorrow * Target-Centric and Network-Centric Cloud Analytics * Future Work 7 UNCLASSIFIED//FOR OFFICIAL USE ONLY --- SECRET//SI//REL TO USA, FVEY NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT ## Current RTRG Architecture RTRG System Goldminer Metadata Search GeoT Geospatial Tools Agent Logic Alerting Tools Sharkfinn Selector Enrichment SKS Report / Doc Manager Panopticon Target Management Supporting Tactical Users Forward Data Centers Services Layer: Web services, authentication, auditing Publish and Subscribe Messaging Oracle relational database & dimensional data model Ingest and Enrichment Pipeline: Flexible, high-speed architecture for parser and data processors Data Feeds: * DRT * JUGGERNAUT * CIDNE * LOPERS * TALIS/MATTERHORN * AIRHANDLER * LYCANTHROPE * KL * VOICESAIL * RETURNSPRING ... and a growing list of others* (DNR, DNI collect, tipping and reporting) A successful architecture for several years. Demand for more data feeds, longer retention, and data-intensive analytics has driven RTRG to seek new solutions \*Based on Afghanistan RTRG data flow SECRET//SI//REL TO USA, FVEY --- SECRET//SI//REL TO USA, FVEY NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT ## RTRG Data Challenges Current Challenges * Data Storage & Retention * "Patterns of Life” analysis needs require 6+ months of data from world-wide collection * A typical system has capacity for only 4-6 weeks of regional data (~90% user queries are within seven days of “now”) * Data Use & Computation * Analytic processes should make maximum use of all available data to find small signals * Relational databases are unsuited to sophisticated analytics such as correlation and matching * Data & Technology Heterogeneity * New types of data must be added to the system continually * With traditional databases, schema modifications are difficult * Exotic data management solutions are difficult to adopt due to limited expertise SECRET//SI//REL TO USA, FVEY --- UNCLASSIFIED//FOR OFFICIAL USE ONLY NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT ## Cloud Architectures for Analytics Google NATIONAL SECURITY AGENCY UNITED STATES OF AMERICA hadoop Apache Cloudbase Scalable BigTable Implementation with Security Emerging NSA Cloud Reference Architecture is well-suited for developing analytics on intelligence data **Scalable:** Distributed file systems and databases are built on clusters of commodity hardware, leveraging open source projects and industrial solutions **Computable:** The MapReduce programming model simplifies writing efficient parallel computations that operate over large volumes of data **Flexible:** Cloud technologies enable flexible schema and leverage large open-source efforts UNCLASSIFIED//FOR OFFICIAL USE ONLY --- TOP SECRET//SI//REL TO USA, FVEY NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT ## Scalability & Computation Data Challenges in AFPAK Current RTRG (AF1) * Current database is 27 terabytes (TB) * Retention is ~30 days Future Cloud enabled system * Even a modest cloud system (3 rack) for storage will be at least 125 TB of storage * 5x increase in available space * Actual retention improvement depends on how the space resources are allocated Cloud supports more data feeds & more days of historical data Analytic Challenges from Iraq & AFPAK * Many analytics used by RTRG are based on R6 SORTINGLEAD event summaries * Event summaries were originally created on relational databases * Collection increased dramatically, and a mapreduce implementation was needed * For new analytics with present day collection volumes, a practical parallel execution model is crucial Cloud supports large-scale analytics TOP SECRET//SI//REL TO USA, FVEY 11 --- TOP SECRET//SI//REL USA, FVEY NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT ## NSA Cloud Computing Enterprise * 5-12 racks commodity hardware * 150+ data nodes * 16 GB RAM each * Apache Hadoop * 100s of terabytes of storage * Stores 10s to 100s of billions of events * NSA Cloudbase * 100s of nodes serving BigTable implementation * Stores 100s of billions of entries gmAPPLE Yakima gmCAVE gmPEACH Utah Language Center gmDEN ADF-C gmMOUNT gmLIGHT gmBALTIC gmPLACE gmPARK gmHALO gmALAMO gmCARBON Europe gmROYALE gmREGAL gmGEIST Iraq gmIDEA (Camp Victory) Afghanistan gmTOTE (Bagram) Korea & Japan gmPEN S. Korea gmPAC Misawa, Japan gmCALM Camp Humphreys S. Korea Australia gmMATE (S//REL) (Alice Springs) GHOSTMACHINE is a data-intensive cloud system with many fielded instances worldwide Hadoop clusters have been demonstrated with 10s of petabytes and over 10,000 cores 12 Map does not include all GHOSTMACHINE/SiteStore systems TOP SECRET//SI//REL USA, FVEY --- TOP SECRET//SI//REL TO USA, FVEY NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT ## Today's "Hybrid” Architecture GTM GeoT SKS PANOPTICON Applications Services Layer Publish and Subscribe Messaging Relational Database Ingest and Enrichment hadoop Large scale data preprocessing and analysis Cloudbase Scalable BigTable Implementation with Security Massive storage and fast record lookup GHOSTMACHINE RTRG and GHOSTMACHINE systems are paired with one another: MapReduce analytic results are fed back to RTRG relational database TOP SECRET//SI//REL TO USA, FVEY 13 --- TOP SECRET//SI//REL TO USA, FVEY NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT ## Tomorrow's Architecture User interfaces * W WidgetFramework * GTM * GeoT * CHET * H&S * SKS * PANOPTICON * Web Tier Services * Application Services * Alerting * Ingest * Real time Enrichment * Analytics * Streaming-based * Analytics * Map Reduce-based Data * Data Storage (NFS) * Secure Data Access Services * Casport, Wavelegal, etc. * Relational Store * Enrichment, User, & Auxiliary Data * Cloudbase * Event Data and Analytic Results Operating System * Java VM * Java VM * Operating System * Hardened RH Linux * Operating System * Hardened RH Linux * OpenStack VM * Utility Cloud * Hardware Layer * Data/Analytic Cloud * Hardware Layer Data Storage (HDFS) The Cloud will bring new data-intensive capabilities, support existing missions, and align RTRG installations with emerging NSA and IC standards TOP SECRET//SI//REL TO USA, FVEY --- UNCLASSIFIED//FOR OFFICIAL USE ONLY NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT ## Outline * RTRG Mission Overview * RTRG System: Today and Tomorrow * Target-Centric and Network-Centric Cloud Analytics * Future Work 15 UNCLASSIFIED//FOR OFFICIAL USE ONLY --- TOP SECRET//SI//REL TO USA, FVEY NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT ## Analytics Overview Target Development Co-Travelers & Meetings Contact Similarity High Priority Untasked Numbers Graph & Network Graph Triage Handset Swap Target Enrichment & Disambiguation Beddown Layercake Heatmaps Geo-Spatial The data-intensive computing capabilities in the system enables a set of graph/network analytics and target development analytics TOP SECRET//SI//REL TO USA, FVEY 16 --- TOP SECRET//SI//REL TO USA, FVEY NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT ## Meeting Target Development Challenges **Past** Analysts manually queried multiple, independent repositories, aggregating results in Excel, taking hours or work for search and refinement **Now** RT-RG provides a streamlined, integrated workflow saving analyst effort Tips Selectors GIST Report Detainee Reports Target Development Event Type comms.gsm Event Subtype call.* Selectors IMSI:12345 NAI Kandahar ? Submit Query Cancel Query Export To File... Metadata Search [REDACTED] Profile Reports Network Changelog Recommendations OBJ DEWEY BEACH ASSOCIATE Bottom Line Pech Valley INS possibly in charge of a "company" sized element. Led the attack Last Active: (TS//SI//REL TO USA, FVEY) (IMEI) 9 hours ago Last Location: (TS//SI//REL TO USA, FVEY) (IMEI) RTRG-Afghanistan Targets 9 hours ago Geo Processor: AROCC Last Report: (TS//SI//REL TO USA, FVEY) (MSISDN) view report 5 months ago VID/SID: 1 year ago Collaboration TOP SECRET//COMINT//REL TO U CST 17 FOB FENTY (PAN): AF IN (Khaksar) Thu, 10 May 2012 15:55:20 Zu TOP SECRET//COMINT//REL TO NZL//20320108 ASSET: JUGGERNAUT Geospatial Alerting Afghanistan/Insurgency: Taliban Leaders in Chaman, Pakistan, Command Insurgent Operations throughout Southern Afghanistan-A SIGINT Assessment (S// SI//REL TO USA, FVEY) Afghanistan: Influence of Chamay on RC South, Juge-July2010 IIR 6 223 0164 12/DOME EXPOSE/DRAGON FIRE- EXPLOITATION OFPAGE 3 Report Text for: 02992282 2017302 AUG 10 FM USJ-800 T O P S E C R E T UMBRA US/UK/CAN/AUS/NZ EYES ONLY SERIAL: 3/CY/507710-10 SUBJ: Afghanistan/Insurgency: Taliban Leaders in Chaman, Pakistan, Command Insurgent Operations throughout Southem Afghanistan A SIGINT Assessment (S//SI//REL TO USA, FVEY) 20100119 08) BIMINI (10020-3AFG) _Nokia1650 kc taliban ircurg Enrichment Outcomes Geolocation Capture TOP SECRET//SI//REL TO USA, FVEY 17 --- TOP SECRET//SI//REL TO USA, FVEY NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT ## Target Development with Meetings Who is at the same UCELLID at the same time? Manual Process * Take your selector and query for every unique location he has been and at what time * Query for other selectors who have been at the same places at the same times (impossible or painful) * OR compare to another known set of selectors to find overlap (excel / ArcGIS / JEMA) (limiting to what you know) * Summary statistics on the matching IMSIs using excel or ArcGIS Cloud Process * Pre-calculates all UCELLID overlaps between tasked selectors * Simply query your selector in cloud-generated QFD and view summary statistics Counts Meetings Different Dates Different Locations Count Selectors TOP SECRET//SI//REL TO USA, FVEY --- TOP SECRET//SI//REL TO USA, FVEY NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT ## Target Development with Co-Travelers* Is there a pair traveling together? Manual Process * You could use the same manual process from Meetings, however, this would not find co-travelers on different networks * Manual comparison of pairs of **known** selectors is possible with ArcGIS or similar spatial tools - **You** **must know the pairs up front** Cloud Process * Measures miles-per-hour (MPH) between tasked selectors as they move around. * Low average MPH = co-traveling. * Simply query for your selector to view statistics on average MPH, days calculated, etc. *Also known as "Sidekicks" TOP SECRET//SI//REL TO USA, FVEY --- TOP SECRET//SI//REL TO USA, FVEY NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT ## Meeting Network Challenges **Past** Manually query multiple repositories and build network with Analyst Notebook (ANB) - amount of labor can be prohibitive **Now** RT-RG tools exist for contact chaining for selector-to-selector & selector-to-report graphs, with more analytics and tools to come Southwest Asia/Narcotics: Senior Traffickers from the AbdulAlim Ishaqzai Narcotics Network Located in HelmandProvince, Afghanistan, After Bahram Chah Raid, 5 November2010 (S//REL TO USA, FVEY) UIM[REDACTED] ASKS[REDACTED] [REDACTED]IF HE HAS HIS MISSING BAG. [REDACTED] [REDACTED] [REDACTED] GETS PASSED[REDACTED] PHONE NUMBER. Selector-to-Report Graph from Enrichment Search graph... Calls:2 Calls:3 [REDACTED] IMSI) // 2040-06-18 // RC EAST TIPPER // OBX GIANT SQUID // selectors VF // 2010-06-16 // VHF N2K TIPPER // OBJ GIANT SQUID // VF//2010-06-15//CSG BAGRAM SELECTORS TIPPER//OBJ GIANT SQUID// calls: 718 PREPAID PREPAID PREPAID THIS TACREP IS REL USA AUS,CAN,DEU,DNK,F RA,GBR,NLD,NO,NZL,SW E OBJ COOPER (OMEGA 10) [REDACTED] targets [REDACTED]IMEI) [REDACTED] MAINWAY graph in RTRG UI TOP SECRET//SI//REL TO USA, FVEY 20 --- TOP SECRET//SI//REL TO USA, FVEY NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT ## Graph Analysis with Furious Chainsaw * Graph representation is natural for DNR * Result is Furious Chainsaw * Prototype on Cloudbase * Now supports contact chains and trends * Will support other graph analytics in the future * Triage capability for forward users to complement Enterprise databases * Enables chaining and other analytics, provides foundation for graph algorithms Metadata matrix in Cloudbase supports fast graph traversal Graph View in Renoir – but many other analytics are possible TOP SECRET//SI//REL TO USA, FVEY 21 --- TOP SECRET//SI//REL TO USA, FVEY NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT ## Graph Triage: Multiple Views Contact Neighborhood Ozone Widget Friends and Family IMS[REDACTED] test [REDACTED] [REDACTED] [REDACTED] [REDACTED] [REDACTED] [REDACTED] [REDACTED] [REDACTED] [REDACTED] [REDACTED] ANB Daily Call Trends in Panopticon PANOPTICON Create Profiles Partners Details Relationships Discussions Bar Event Count 100 80 60 40 20 0 0 1 2 3 4 5 6 7 8 9 1011121314151617181920212223 Hour of Day (Zulu) Top Associates Ozone Widget Top 100 Associations Top 100 Associations | Event Count [REDACTED] | 81 [REDACTED] | 63 [REDACTED] | 61 [REDACTED] | 59 [REDACTED] | 49 [REDACTED] | 43 [REDACTED] | 37 [REDACTED] | 34 [REDACTED] | 27 [REDACTED] | 24 Showing 1 to 10 of 100 entries Previous Next Weekly Call Trends Ozone Widget Day of Week Bar Event Count 200 150 100 50 0 Mon Tue Wed Thu Fri Sat Sun Day of Week DNR graphs in Furious Chainsaw tables in Cloudbase support a wide range of fast queries and analytics TOP SECRET//SI//REL TO USA, FVEY 22 --- TOP SECRET//SI//REL TO USA, FVEY NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT ## Unstructured Data Exploitation SHARKFINN * Selector extraction, normalization, and enrichment * Flexible free-text query interface * Graph, text, and spreadsheet output formats SKS Structured Knowledge Space * Entity extraction (people, organizations, times, geos) * Keyword, faceted, and people search * Document clustering * Arabic name expansion sharkfury * Integrated data ingest using Niagara Files (NiFi) * SharkQuery: search by selector, entity, location, and keyword * SharkDocs: query, sharing, and collaboration on user uploaded documents * Visualization of results in query overview, table, graph, and map * Cloudbase and HDFS for scalable text analytics platform TOP SECRET//SI//REL TO USA, FVEY --- TOP SECRET//SI//REL TO USA, FVEY NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT ## Target Development - SharkFinn / SKS SKS "OBJ SMITHERS" AND OBJECTIVES: OBJ TOPPER Previous 30 Days 1.Perform keyword search with seed OBJ on report library OBJECTIVES OBJ SMITHERS (19) OBJ TOPPER (19) OBJ SPRINGFIELD (12) OBJ APU (6) OBJ GROUNDSMAN WILLY (6) OBJ WIGGUM (4) OBJ MAYFIELD (3) OBJ NEVIS (1) OBJ TRAJAN (1) 2. Finding co-occurring objectives SELECTORS [REDACTED] [REDACTED] [REDACTED] [REDACTED] [REDACTED] [REDACTED] [REDACTED] 3.Selector co-occurrence 4.Additional Search & Filtering SELECTOR 1 [REDACTED] 2 [REDACTED] 3 [REDACTED] 4 [REDACTED] 5 [REDACTED] 6 [REDACTED] 7 [REDACTED] 8 [REDACTED] 9 [REDACTED] 10 [REDACTED] 11 [REDACTED] 12 [REDACTED] 13 [REDACTED] 5. Correlate found selectors with Octave/UTT and Panopticon 6. Export selector & Report Graph TOP SECRET//SI//REL TO USA, FVEY --- TOP SECRET//SI//REL TO USA, FVEY NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT ## Meeting Geolocation Challenges **Past** Analysts manually correlated locations using map viewers or spreadsheets, aggregating data from multiple sources **Now** Analytics and alerts push target information by subscription 412.001.00078.27161 UCELLIDS 412.001.00076.11402 /01 /10 /16 /22 /23 /24 /25 /26 /27 /28 /29 3/04 3/05 3/06 3/07 3/08 3/09 3/10 3/11 3/12 3/13 3/14 3/15 Day RT-RG pattern of life analytic for target location cues tio field First Event Last Event MAHIM Alükh Qal'eh-ye Adam Khan Vazil Niazi Shiwak Shiv Analysts are notified by alerts, based on: * Geospatial NAI* * Tasking status * SMS content * Selectors, callsigns, frequencies * ... Op ATTENTION INS RC-C Kabul (PAN):UNID ASW SUICIDE VEST KABUL 3 Thu, 10 May 2012 17:48:12 Zulu TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108 ASSET: LOPERS EVENT TYPE: comms.gsm/call.orig SIGAD: US-3309 PDDG: K2 SSD: L2 POEI: (Missing) CASN: GCCA0153L5C0001 SELECTION MODE: STANDARD COLLECTION AUTHORITY: EO12333 Target Location Approx. Geo: 42S WD 21111 18789 UCELLID: [REDACTED] Geo Data Source(s): GSM Geo-Sigdev 4 Item(s) 25 TOP SECRET//SI//REL TO USA, FVEY * NAI = Named Area of Interest --- TOP SECRET//SI//REL TO USA, FVEY NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT ## Target Geolocation with Bed Down Find the most consistent location of the day's first/last event First Events + Last Events = Bed Down Manual Process - One Selector At a Time * Query all events for your selector. * Mark first and last events manually. * OR * Enter in a tool like CheekyMonkey to view gaps in activity. * Slow process to do one selector at a time Cloud Process- All Tasked Selectors * Pre-calculates first and last events in local time for ALL selectors. * Will calculate estimated Bed Down at query time. * Can query multiple selectors in seconds, find common overlap. TOP SECRET//SI//REL TO USA, FVEY --- TOP SECRET//SI//REL TO USA, FVEY NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT ## Target Geolocation with LayerCake LayerCake - find geospatial overlap of a set of targets. Manual Process * Query all events for your selectors. * Display events spatially on mapping software (impossible to view polygon overlaps). * Rasterize and do "raster math” to determine max overlap. (very complex and expensive task in most GIS tools). Cloud Process * Pre-calculates unique locations visited for ALL selectors. * Raster heat maps drawn at query time. TOP SECRET//SI//REL TO USA, FVEY --- TOP SECRET//SI// REL USA,FVEY NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT ## Spatial Visualization of Data Using MapReduce, analytics can count and aggregate over large number of daily events to give a multi-resolution spatial visualization of the data Google Earth Cloudbase Tables for Geo Data MapReduce for Binning, Counting Event Data Analytic Data Flow Synoptic View TS//SI//REL Call volume map of RTRG (AF1) collection (A day in Dec, 2011) Fewer DNR events Fewer events 28 TOP SECRET//SI// REL USA, FVEY --- TOP SECRET//SI // REL USA, FVEY NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT ## Spatial Visualization of Data Cloud enabled analytic allows viewing the spatial data at many levels-of-detail “Meso-scale” View Detail View – “Mashup” with heatmap Kabul Peshawar Rawalpindi Hyderabad Karachi SEDB Faisalabad Amritsar Lahore Srinagar Heatmap of activity of a set of targets [REDACTED] ASSOC OF CH [REDACTED] NDS D124 DEP CHIEF[REDACTED] ΜΟΙ ΖΟΝΕ 101 AS KARZAI AIDE TO MO OCT 829 UPS AND INDIV MOI CT OFFI وز محشر شفا خواهی تمام مومنین است. We will arrive on saturday محمد در عرب سردار دین است چراغ آسمان شمع زمین است. به فردای قیامت و And Z the day of judgment healing desiring all believers is. We+will+arrive+on+saturday+ [REDACTED] commander religion is: light heaven/sky candle land is: to the resurrection-day R Directions: To here - From here MFA MOD SECRETARIAT MP DEVELOPMENT ICW IRGC QF DEPT 409 KHALEQ MP Using Cybertrans translation service on SMS messages, integrated with heatmaps Fewer calls More calls TOP SECRET//SI // REL USA, FVEY 29 --- UNCLASSIFIED//FOR OFFICIAL USE ONLY NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT ## Outline * RTRG Mission Overview * RTRG System: Today and Tomorrow * Target-Centric and Network-Centric Cloud Analytics * Future Work 30 UNCLASSIFIED//FOR OFFICIAL USE ONLY --- TOP SECRET//SI//REL TO USA, FVEY NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT ## RTRG Tomorrow * Improved DNI capabilities; focus on convergence * Integrating active SIGINT capabilities * Increased CT and expeditionary capabilities * Better tools for faster analytic development * Incorporation of content analysis and HLT capabilities * Improved integration between target and population analytics 31 TOP SECRET//SI//REL TO USA, FVEY --- TOP SECRET//SI// REL USA, FVEY NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT ## RTRG Planned Cloud Instances NSA/CSS GEORGIA FORT GORDON gmSeminole NSA Georgia NAVY INFORMATION OPERATIONS COMMAND BAHRAIN NETWARCOM gmGulf NIOC-Bahrain NCR AFGHANISTAN SEMPER VIGILANS SEMPER PARATOS SEMPER VOLENS gmZilla Kabul TOP SECRET//SI// REL USA, FVEY 32 --- TOP SECRET//SI//REL USA,FVEY NATIONAL SECURITY AGENCY SIGDEV SIGINT DEVELOPMENT ## Summary * RTRG has been a successful regional data store and exploitation system for COIN, CIED and other missions * Moving to NSA Cloud infrastructure * More historical data * Deeper analysis using parallel programs * Allows for more flexible deployments to IC, DoD service installations * Continuing to support advanced analytics for current and future operations TOP SECRET//SI//REL USA, FVEY 33