first commit
This commit is contained in:
commit
0d90363116
6510 changed files with 679146 additions and 0 deletions
65
data/rt10-overview/metadata.json
Normal file
65
data/rt10-overview/metadata.json
Normal file
|
|
@ -0,0 +1,65 @@
|
|||
{
|
||||
"title": "RT10 Overview",
|
||||
"type": "REPORTED",
|
||||
"description": "This RT10 overview document is a high-level briefing on the implementation of a real-time SIGINT architecture designed to modernize intelligence operations in the Iraq theater, specifically Baghdad. The project aims to replace legacy, \"manual-intensive\" processes with a \"SIGINT Brain\"—a distributed database and automated analytic framework capable of processing over 50 million GSM metadata events per day.",
|
||||
"long_description": "The RT10 initiative was born out of a critical need to handle the increasing \"velocity and volume\" of converged network data. Before RT10, analysts faced substantial lags, non-integrated tools, and a lack of comprehensive theater knowledge. The transition is described as moving from the static \"Atlantic Monthly\" style of analysis to the high-speed \"CNN scrolling bar\" model.\n\nKey technical pillars of RT10 include:\n\n* VoiceRT: A voice-processing engine designed to perform phonetic indexing on 1 million voice cuts per day and match them against 1,000 specific voice prints. This was intended to optimize linguist time by filtering for high-value targets automatically.\n* Advanced Sensors & Checkpoints: The implementation of \"active cell phone interrogation\" and RF illumination at military checkpoints (such as Route Irish) to fingerprint vehicles and identify threats like IED triggers or artillery shells.\n* Automation: The \"One-Click\" philosophy, allowing analysts to instantly generate relationship-chaining diagrams, frequent calling lists, and geospatial trends directly from alerts.\n* Connectivity: A robust local network utilizing Wi-Max (35 Mbps), Wireless Mesh, and Trojan Spirit to connect analytic nodes at the \"Perfume Palace,\" JIOC-I, and NSA-G.\n\nThe implementation followed a \"Spin\" methodology—90-day iterative cycles to expand capabilities. Spin 1 (August 2006) focused on initial installation and JUGGERNAUT data integration, while later spins incorporated FORNSAT (Foreign Satellite) data and next-generation analytical tools. The architecture specifically targeted various points in the GSM network, from tactical \"G-Box\" air-interface collection to national-level targeting at the Abis and A-Link interfaces using systems like OBELISK.",
|
||||
"tags": [
|
||||
"GSM",
|
||||
"metadata collection",
|
||||
"phonetic indexing",
|
||||
"real-time alerting",
|
||||
"geospatial analysis",
|
||||
"geotemporal analysis",
|
||||
"tactical intelligence",
|
||||
"target development",
|
||||
"theater intelligence",
|
||||
"surveillance architecture"
|
||||
],
|
||||
"document": {
|
||||
"creator": "NSA",
|
||||
"date_created": "2006-06",
|
||||
"classification": [
|
||||
"TOP SECRET",
|
||||
"COMINT"
|
||||
],
|
||||
"dissemination": [],
|
||||
"banner": "TOPSECRET//COMINT//20291123",
|
||||
"genre": "slides",
|
||||
"programs": [
|
||||
"RT10",
|
||||
"OBELISK",
|
||||
"WISPYKNIT",
|
||||
"VICTORYUNIFORM",
|
||||
"JUGGERNAUT",
|
||||
"COBRAFOCUS",
|
||||
"MATTERHORN",
|
||||
"EXPIATION",
|
||||
"VoiceRT",
|
||||
"GREENDRAGON",
|
||||
"TAREX",
|
||||
"KEYCARD"
|
||||
],
|
||||
"countries": [
|
||||
"Iraq"
|
||||
],
|
||||
"people": []
|
||||
},
|
||||
"publication": [
|
||||
{
|
||||
"publisher": "The Intercept",
|
||||
"date_published": "2015-05-05",
|
||||
"title": "How the NSA Converts Spoken Words Into Searchable Text",
|
||||
"link": "https://theintercept.com/2015/05/05/nsa-speech-recognition-snowden-searchable-text/",
|
||||
"reporters": [
|
||||
"Froomkin, Dan"
|
||||
],
|
||||
"contributors": [
|
||||
"Fishman, Andrew"
|
||||
],
|
||||
"doc_source": "https://theintercept.com/document/2015/05/05/rt10-overview/",
|
||||
"doc_url": "https://s3.documentcloud.org/documents/2072024/rt10-overview.pdf"
|
||||
}
|
||||
],
|
||||
"pdf_sha256": "d965e7b2be55a01c3d6fe6fda0531ab6f9467a7766adf905f717eb9d0aa8caaf",
|
||||
"pages": 15
|
||||
}
|
||||
288
data/rt10-overview/rt10-overview.md
Normal file
288
data/rt10-overview/rt10-overview.md
Normal file
|
|
@ -0,0 +1,288 @@
|
|||
# RT10 Overview
|
||||
|
||||
## June 2006
|
||||
|
||||
TOPSECRET//COMINT//20291123
|
||||
|
||||
---
|
||||
|
||||
TOPSECRET//COMINT//20291123
|
||||
|
||||
## Current Challenges – Why Innovate?
|
||||
|
||||
* Network is converging and dataflow is increasing
|
||||
* Pushing everything back means front-end filtering → dropped data
|
||||
* Substantial lag to process, store, and query data
|
||||
* Restricted geo-spatial capabilities (not just because of the hardware)
|
||||
* Manual correlation between SIGINT, HUMINT, SIGACTS
|
||||
* Non-integrated toolsets
|
||||
* No comprehensive theater knowledge base
|
||||
* Non-optimal collaboration between analysts
|
||||
* Manually intensive production processes
|
||||
* Cannot scale / work targets in volume
|
||||
* Reaching limits of legacy systems
|
||||
* Analysis takes a lot of time
|
||||
|
||||
Think: Atlantic Monthly vs. CNN scrolling bar
|
||||
|
||||
---
|
||||
|
||||
TOPSECRET//COMINT//20291123
|
||||
|
||||
## RT-10 Goals
|
||||
|
||||
* Overall Objective: Order of magnitude improvements in real-time SIGINT architecture for the U.S. Cryptologic System, initially focused on national and tactical intelligence in Baghdad, to enable **better** decisions in **less** time
|
||||
|
||||
* Providing:
|
||||
* Access to **more comprehensive** data
|
||||
* Immediate access to local data sets, with query back to NSAW
|
||||
* Integrated Analytic Workflow, with **better** tools
|
||||
* Real time Alerting: National and Tactical
|
||||
* Automation of tasks → Query to Dissemination
|
||||
* Distributed Analytic Collaboration
|
||||
* Scalability
|
||||
* **Integration** across brigade-level SIGINT capabilities
|
||||
|
||||
TOPSECRET//COMINT//20291123
|
||||
|
||||
---
|
||||
|
||||
TOPSECRET//COMINT//20291123
|
||||
|
||||
## Tools and Workflow
|
||||
|
||||
* Better Tools
|
||||
* New relationship visualization w/temporal capabilities
|
||||
* Real-time geo-spatial alerting framework
|
||||
* Web-based applications optimized for speed in a distributed environment
|
||||
* Partnership with Green Dragon to identify and inject new COTS/GOTS technologies in **much less** time
|
||||
|
||||
* Integrated Framework
|
||||
* Work in any tool of choice, seamlessly switch to alternative views (think development → presentation)
|
||||
* Automated, one-click mentality from query to dissemination
|
||||
* Developers available to react to analyst needs and inject new capabilities
|
||||
|
||||
---
|
||||
|
||||
TOPSECRET//COMINT//20291123
|
||||
|
||||
## Substantial Improvements in Data Access
|
||||
|
||||
* Initial
|
||||
* Traditional Data Sources (Global Reach touching NSA databases)
|
||||
* SCS GSM collection
|
||||
* Tactical GSM Accesses
|
||||
* Checkpoint Data
|
||||
* HUMINT / All-Source derived SIGINT Selectors (parsed CIA TDS, DOD IIRs, TAREX, DOCEX)
|
||||
* Local knowledge base
|
||||
|
||||
* Future
|
||||
* Fully-integrated Iraqi DNI Data flows (initially accessible through separate web interface)
|
||||
* OBELISK / LETC GSM Coverage
|
||||
* WISPYKNIT, VICTORYUNIFORM and other special source
|
||||
|
||||
Think: Know everything we collectively know, and faster
|
||||
|
||||
---
|
||||
|
||||
TOPSECRET//COMINT//20291123
|
||||
|
||||
## VoiceRT: Index / Search of Voice Cuts
|
||||
|
||||
* Goal: Better filtering and selection using latest generation of voice-processing technologies
|
||||
* Perform phonetic indexing on 1 million voice cuts per day
|
||||
* Run incoming cuts against 1000 individual voice prints to drive real-time filtering and selection
|
||||
* Optimizes linguist scarce time, does not replace linguists
|
||||
|
||||
* Increases efficiency of available linguists
|
||||
* Allows analysts to affect collection priorities and react to changing linguistic / word patterns
|
||||
* Possible future integration with checkpoint collection (voice / biometrics)
|
||||
|
||||
TOPSECRET//COMINT//20291123
|
||||
|
||||
---
|
||||
|
||||
TOPSECRET//COMINT//20291123
|
||||
|
||||
## Real time Alerting: National and Tactical
|
||||
|
||||
* Real-time alerting on hard selectors
|
||||
* Creating a knowledge base within the collection architecture
|
||||
* Drives selection and filtering
|
||||
* Provides relevant information to war fighter in seconds
|
||||
|
||||
* Algorithms to Detect and Alert from Patterns of Interest
|
||||
* Constant enrichment of incoming data flows based on NSA and GCHQ-developed algorithms
|
||||
* Robust framework to allow analysts to submit / modify / reject existing techniques
|
||||
* Capability to extend algorithms to correlate and react to friendly actions, geospatial and geotemporal vicinity, etc.
|
||||
|
||||
Dedicated effort to identify and detect new potential targets based on known behavioral patterns
|
||||
|
||||
---
|
||||
|
||||
TOPSECRET//COMINT//20291123
|
||||
|
||||
## Automation of Standard Tasks
|
||||
|
||||
* One-Click Report Generation
|
||||
* Chaining diagram
|
||||
* Products containing target
|
||||
* Frequent calling list
|
||||
* Temporal trends
|
||||
* Geospatial trends
|
||||
|
||||
* One-Click Alert-to-Analysis
|
||||
* Alerting framework fully integrated with analytical toolsets
|
||||
* Geospatial / Temporal / Network views of data
|
||||
|
||||
* One-Click Analyst Actions
|
||||
* Drive collection through interface to EDB / Keycard
|
||||
* Effortlessly affect knowledge base confidence / details
|
||||
|
||||
---
|
||||
|
||||
TOPSECRET//COMINT//20291123
|
||||
|
||||
## Checkpoints
|
||||
|
||||
* Provide advanced sensors to generate checkpoint metadata:
|
||||
* Active cell phone interrogation
|
||||
* Active RF Illumination: goal to fingerprint vehicles, identify threats (artillery shells, ammunition, gun barrels, electronic triggers)
|
||||
* 360 degree imagery
|
||||
* Chemical and radiological detectors
|
||||
* Fed real time to tip and cue other Ints
|
||||
|
||||
* Proof of concept in vicinity of BIAP, tentatively checkpoint 538 on Route Irish
|
||||
* Operational test, tentatively checkpoint 502 near Abu Ghurayb
|
||||
|
||||
---
|
||||
|
||||
TOPSECRET//COMINT//20291123
|
||||
|
||||
## Implementation
|
||||
|
||||
* Construct the JIOC-I "SIGINT Brain"
|
||||
* Distributed Databases in Baghdad and Ft Meade
|
||||
* Aggregate Metadata from tactical and national collection, focusing on GSM for initial efforts
|
||||
* Massive data flows: 50 Million+ GSM metadata events / day
|
||||
* Content access from all possible collectors
|
||||
* Integration: "Know what we know"
|
||||
|
||||
* Timeline
|
||||
* End June – Initial site surveys, theater coordination
|
||||
* 15 July – Hardware Ships
|
||||
* 1 August – Hardware Arrives, People begin arrving
|
||||
* 15 August – System Online
|
||||
* 31 August – Data Flowing
|
||||
* 15 September – IOC
|
||||
|
||||
TOPSECRET//COMINT//20291123
|
||||
|
||||
---
|
||||
|
||||
TOPSECRET//COMINT//20291123
|
||||
|
||||
## Achieving Success with Spins
|
||||
|
||||
* Spin Methodology:
|
||||
* Iterative activity consisting of a series of spins
|
||||
* Each 90-day spin expands capability
|
||||
* Demonstration of integrated capabilities
|
||||
* Application of new and existing technologies
|
||||
* Make discoveries and apply lessons learned to future spins
|
||||
|
||||
SPIN 1 SPIN 2 SPIN 3
|
||||
|
||||
TOPSECRET//COMINT//20291123
|
||||
|
||||
---
|
||||
|
||||
TOPSECRET//COMINT//20291123
|
||||
|
||||
## RT10 Spin Schedule
|
||||
|
||||
Spin 1: August 2006
|
||||
* System Installed
|
||||
* JUGGERNAUT Data
|
||||
* Initial Software Testing
|
||||
|
||||
Spin 2: Nov 2006
|
||||
* Demonstration of integrated capabilities
|
||||
* Cable / FORNSAT Integration
|
||||
* Enhance Checkpoint Capability
|
||||
* Analyst-Identified Areas of Improvement
|
||||
|
||||
Spin 3: Jan 2007
|
||||
* Analyst-Driven Modifications
|
||||
* Next-Generation Analytical Tools
|
||||
|
||||
SPIN 1 SPIN 2 SPIN 3
|
||||
|
||||
TOPSECRET//COMINT//20291123
|
||||
|
||||
---
|
||||
|
||||
TOPSECRET//COMINT//20291123
|
||||
|
||||
## RT10 Line of Sight Microwave Network
|
||||
## Wi-Max, Mesh, NSAnet Connectivity
|
||||
|
||||
Wi-Max (35 Mbps)
|
||||
Wireless MESH Network
|
||||
Koala House
|
||||
SIGINT Brain
|
||||
Trojan Spirit (8 Mpbs 4 Mbps JWICS, 4 Mbps NSANet)
|
||||
JIOC-I,CSG,IFC
|
||||
Perfume Palace
|
||||
NSANET 100Mbps
|
||||
TACLANE Encryption
|
||||
CP Prosperity
|
||||
CST - 4ID
|
||||
Al Faw Palace
|
||||
Villa
|
||||
|
||||
---
|
||||
|
||||
TOPSECRET//COMINT//20291123
|
||||
|
||||
## RT10 Analytic Nodes
|
||||
|
||||
* Iraq
|
||||
* MOC
|
||||
* NSA Product Lines
|
||||
* NSA-G
|
||||
* COBRA FOCUS
|
||||
|
||||
TOPSECRET//COMINT//20291123
|
||||
|
||||
---
|
||||
|
||||
TOP SECRET//SI//X1
|
||||
TOPSECRET//COMINT//20291123
|
||||
|
||||
## GSM Architecture & Access Points
|
||||
|
||||
Tactical Access Points
|
||||
[G-Box]
|
||||
Air Interface
|
||||
BTS
|
||||
[G-Box]
|
||||
|
||||
[DRT]
|
||||
[Matterhorn]
|
||||
[Expiation]
|
||||
|
||||
Primary National-level Targeting Points
|
||||
Abis Interface
|
||||
A Link
|
||||
[JUG]
|
||||
BSC
|
||||
[JUG]
|
||||
MSC
|
||||
[OBELISK]
|
||||
GMSC
|
||||
PSTN
|
||||
PSTN
|
||||
|
||||
TOPSECRET//COMINT//20291123
|
||||
TOP SECRET//SI//X1
|
||||
Loading…
Add table
Add a link
Reference in a new issue