first commit

This commit is contained in:
libroot 2026-11-19 04:39:11 +02:00
commit 0d90363116
Signed by: libroot
GPG key ID: 2AC68C029EBE9ACE
6510 changed files with 679146 additions and 0 deletions

View file

@ -0,0 +1,61 @@
{
"title": "GHOSTMACHINE: Identifier Lead Triage with ECHOBASE",
"type": "REPORTED",
"description": "This June 2012 presentation introduces ECHOBASE, an NSA analytic architecture designed to automate the triage of massive identifier lead lists (ranging from 50 to 10,000+ entries). By applying behavior-based analytics and \"yes/no\" behavioral queries against raw SIGINT, the system allows analysts to prioritize high-interest targets and discard irrelevant leads at scale.",
"long_description": "The document addresses a critical bottleneck in the SIGINT lifecycle: the transition from bulk lead generation to manual analysis. Under traditional tradecraft, analysts were often overwhelmed by lists of \"possible interest\" identifiers that required tedious manual vetting. ECHOBASE solves this by implementing Bulk Lead Triage via Behavior Analytics, which categorizes selectors into priority levels based on specific behavioral triggers—such as direct communication with known targets, presence in captured media, or activity in specific geographic regions like France.\n\nThe technical backbone of this system relies on the GHOSTMACHINE framework and the Identifier Scoreboard tool. These platforms aggregate data from various databases (including OCTAVE, UTT, and FOREMAN) to provide a \"SIGINT Business\" enrichment layer, covering targeting, foreignness, and compliance. The architecture is designed for cross-enterprise sharing, specifically highlighting a joint initiative for the 2012 London Olympics. During this event, the NSA and GCHQ synchronized their analytic feeds, allowing the Leads Evaluation Cell to leverage combined datasets to monitor potential threats in real-time under a unified \"Olympics Sharing\" framework.",
"tags": [
"SIGDEV",
"triage",
"behavioral analysis",
"olympics",
"London 2012 Olympics",
"Lead Evaluation Cell",
"QFD",
"tradecraft"
],
"document": {
"creator": "NSA",
"date_created": "2012-06",
"classification": [
"TOP SECRET",
"COMINT"
],
"dissemination": [
"REL TO USA, CAN, AUS, GBR, NZL"
],
"banner": "TOP SECRET//COMINT//REL TO USA, CAN, AUS, GBR, NZL",
"genre": "slides",
"programs": [
"GHOSTMACHINE",
"ECHOBASE",
"IDENTIFIERSCOREBOARD",
"FOREMAN",
"CONTRAOCTAVE",
"OCTAVE",
"UTT",
"WAVELEGAL",
"CASPORT",
"PSC"
],
"countries": [
"France"
],
"people": []
},
"publication": [
{
"publisher": "The Intercept",
"date_published": "2014-04-30",
"title": "British Spy Chiefs Secretly Begged to Play in NSA's Data Pools",
"link": "https://theintercept.com/2014/04/30/gchq-prism-nsa-fisa-unsupervised-access-snowden/",
"reporters": [
"Gallagher, Ryan"
],
"contributors": [],
"doc_source": "https://theintercept.com/document/2014/04/30/ghostmachine-identifier-lead-triage-echobase/",
"doc_url": "https://s3.documentcloud.org/documents/1150436/ghostmachine-identifier-lead-triage-with-echobase.pdf"
}
],
"pdf_sha256": "5a3fffc3563686b5f39974f65e0e1930c2a852b2b6a0e25eb81a22e2e72eff44",
"pages": 14
}

View file

@ -0,0 +1,577 @@
TOP SECRET//COMINT//REL TO USA, CAN, AUS, GBR, NZL
# Identifier Lead Triage with ECHOBASE
[REDACTED]
JUN 2012
NSA - S2I51
NSA - T1442
TOP SECRET//COMINT//REL TO USA, CAN, AUS, GBR, NZL
---
TOP SECRET//COMINT//REL TO USA, CAN, AUS, GBR, NZL
## The Problem
SIGINT is very good at 2 things:
1. Establishing lists of potential leads (50-10k+)
2. Manual analysis to vet individual targets
Potential leads
50-10k+
????
Manual analysis
TOP SECRET//COMINT//REL TO USA, CAN, AUS, GBR, NZL
2
---
TOP SECRET//COMINT//REL TO USA, CAN, AUS, GBR, NZL
## Tradecraft
A common model for identifier lead lists, today:
Input
* Seed List Provided to SIGDEV
Phase 2
* Normalize and Expand Selectors
Phase 3
* Foreignness and Compliance Check
Phase 4
* SIGINT Queries on Selector activity and behavior attributes
Bulk enrichment of 'SIGINT business knowledge'
????
Manual analysis
TOP SECRET//COMINT//REL TO USA, CAN, AUS, GBR, NZL
3
---
TOP SECRET//COMINT//REL TO USA, CAN, AUS, GBR, NZL
## Triage Today
After initial enrichment checks, the analyst is often left with too many identifiers of "possible interest"
No Further Analysis Needed 20%
Definite Interest (Pri 1) 5%
Possible Interest (Pri 2) 75%
Percentages are conceptual
TOP SECRET//COMINT//REL TO USA, CAN, AUS, GBR, NZL
4
---
TOP SECRET//COMINT//REL TO USA, CAN, AUS, GBR, NZL
## Bulk Lead Triage via Behavior Analytics
* Hundreds or thousands of selectors to go through high level vetting very quickly
* Better triage prioritization allows for highly adjustable thresholds to be set for follow -on analysis
* Compliance can be inserted at both the “batch result” and “query” level
* Potentially utilize multiple clouds & cross-enterprise analytics
No Further Analysis Needed 20%
Definite Interest (Pri. 1) 5%
High Interest (Pri 2) 15%
Low Interest (Pri 4) 25%
Medium Interest (Pri 3) 35%
TOP SECRET//COMINT//REL TO USA, CAN, AUS, GBR, NZL
5
---
TOP SECRET//COMINT//REL TO USA, CAN, AUS, GBR, NZL
## Identifier 'SIGINT Business' Enrichment
Bulk gathering, via Identifier Scoreboard (phase 2/phase 3)
* Targeting
* Authorities
* Reporting
* Targets
* Knowledge
* Foreignness
* Compliance
...not a raw SIGINT query
```
Dynamic Page Highest Possible Classification Is TOP SECRET//SI//REL TO USA, FVEY
About | Contact | A little help?
Airport Threat
IDENTIFIER
SCOREBOARD
Hide lineup
Add identifiers
Validate >>
Change realm | Delete
Added | Context | Realm
[REDACTED] | 12147 | [REDACTED] X
[REDACTED] | [REDACTED] | yahoo X
[REDACTED]<skypeUser> | [REDACTED] | skypeUser X
Import identifiers
Targeting tag
flag
Targeting category
Targeting zipcode
Query/criteria list
[REDACTED] X
Priority Email X
Choose...
Target
[REDACTED] X
Choose...
Choose...
Retrieve up-to-date FOREMAN data
Include correlations
Clear Save Search
---
SIGINT
ANALYTICS
Identifier view
Target view
Create target | Customize | Send to...
Targeting | Foreignness | Knowledge
```
| Normalized identifier | Comms | PSC | Tip | TAO | CNE | POCS | CONTRAOCTAVE | FOREMAN | Comments | Location | Reports | Actions |
| --------------------- | ----- | --- | --- | --- | --- | ------------------ | ------------ | ------- | -------- | ---------- | ------- | ------- |
| [REDACTED] | | | | | | 1 | n/a | unknown | | | | |
| [REDACTED] | | | | | | 1 | n/a | unknown | | | 10 | |
| [REDACTED] | | | | | | [REDACTED] (S2112) | n/a | unknown | | [REDACTED] | | |
| [REDACTED] | | | | | | [REDACTED] (F52) | n/a | unknown | | [REDACTED] | | |
| [REDACTED] | | | | | | 4 | n/a | unknown | 3 | | | |
```
Last updated: 2012-Mar-01 09:58:46 Z
Displaying 1-100 of 154 page 1 of 2
Dynamic Page Highest Possible Classification Is TOP SECRET//SI//REL TO USA, FVEY
```
TOP SECRET//COMINT//REL TO USA, CAN, AUS, GBR, NZL
6
---
TOP SECRET//COMINT//REL TO USA, CAN, AUS, GBR, NZL
## 'Yes/No' Identifier Behavior
Bulk triage, via SIGINT Analytics Mode (start of phase 4)
Core set of 'yes/no' behavioral questions about a set of identifier leads
...against raw SIGINT!
```
Dynamic Page Highest Possible Classification Is TOP SECRET//SI//REL TO USA, FVEY
About | Contact | A little help?
SIGINT
Airport Threat
Scoreboard
Has this identifier been in direct communications with a targeted identifier?
Identifiers supported: DNI and DNR
Legal authorities: EO12333_S, EO12333_X, FAA_702_CP, FAA_
702_CT, FAA 02_FG, FAA 704/705B
Results updated: Daily
more...
Identifier view
Target view
Customize | Send to...
```
| Normalized identifier | Positive Results | Direct Comms with Target? | Targeted Contact? | In Captured Media? | [REDACTED] | Seen | Actions |
|-----------------------|------------------|---------------------------|-------------------|----------------------|------------|------|---------|
| [REDACTED] | 2 | * | n/a | n/a | n/a | | |
| [REDACTED] | n/a | | | | | | |
| [REDACTED]<msnpassport>View detailed analytic results | 4 | Targets:<br> [REDACTED]<yahoo> TP<br>[REDACTED]<facebookname><br><br>TP First heard: 25 Oct 2011<br>Last Heard: 09 Mar 2012 | | | [REDACTED]First heard: 03 Dec 2011<br>Last heard: 29 Feb 2012 | [REDACTED] | |
<msnpassport>
| [REDACTED]<yahoo> | 3 | * | | | | | |
| [REDACTED]<emailAddr> | n/a | | | | | | |
| [REDACTED]<msnpassport> | 4 | * | * | | * | | |
| [REDACTED]<facebookname | 2 | | * | | | | |
| [REDACTED]<skypeUser> | 0 | | | | | | |
| [REDACTED]<yahoo> | 0 | | | | | | |
| [REDACTED]<yahoo> | 2 | * | | | | | |
| [REDACTED]<emailAddr> | 2 | | * | | | | |
| [REDACTED]<msnpassport> | 0 | | | | | | |
| [REDACTED]<google> | 0 | | | | | | |
| [REDACTED] | 0 | | n/a | n/a | n/a | | |
| [REDACTED] | 2 | * | | | | | |
| [REDACTED] | 4 | * | * | * | | | |
| [REDACTED] | 2 | * | n/a | n/a | n/a | | |
```
Last ran on: 2012-Mar-01 09:58:46 Z
Displaying 1-100 of 148 page 1 of 2
(U//FOUO) Page Publisher: Identifier Scoreboard, T1422 | Content [REDACTED] T14 | Last Reviewed: 15-Dec-2010 | Last Modified: 23-Feb-2012
(U//FOUO) Derived From: NSA/CSSM 1-52 | Declassify On: 20320108 | Dated: 08 January 2007
Dynamic Page Highest Possible Classification is TOP SECRET//SI//REL TO USA, FVEY
```
TOP SECRET//COMINT//REL TO USA, CAN, AUS, GBR, NZL
7
---
TOP SECRET//COMINT//REL TO USA, CAN, AUS, GBR, NZL
## SIGINT Analytics Mode
Triage by aggregate behaviors
| Normalized identifier | Positive Results | Direct Comms with Target? | Targeted Contact? | In Captured Media? | [REDACTED] | Se |
|-----------------------|------------------|---------------------------|-------------------|----------------------|------------|------|
| [REDACTED] | 2 | * | n/a | n/a | n/a | |
| [REDACTED] | n/a | | | | | |
| [REDACTED]<msnpassport>View detailed analytic results | 4 | Targets:<br> [REDACTED]<yahoo> TP<br>[REDACTED]<facebookname><br><br>TP First heard: 25 Oct 2011<br>Last Heard: 09 Mar 2012 | | | [REDACTED]First heard: 03 Dec 2011<br>Last heard: 29 Feb 2012 | [REDACTED] | |
<msnpassport>
| [REDACTED]<yahoo> | 3 | * | | | | |
| [REDACTED]<emailAddr> | n/a | | | | | |
| [REDACTED]<msnpassport> | 4 | * | * | | * | |
| [REDACTED]<facebookname> | 2 | | * | | | |
| [REDACTED]<skypeUser> | 0 | | | | | |
| [REDACTED]<yahoo> | 0 | | | | | |
| [REDACTED]<yahoo> | 2 | * | | | | |
| [REDACTED]<emailAddr> | 2 | | * | | | |
One column per 'yes/no' question
Quickly zero in on worthy leads
TOP SECRET//COMINT//REL TO USA, CAN, AUS, GBR, NZL
8
---
TOP SECRET//COMINT//REL TO USA, CAN, AUS, GBR, NZL
## SIGINT Analytics Mode - Detailed View
```
Dynamic Page - Highest Possible Classification Is TOP SECRET//SI//REL TO USA, FVEY
About | Contact | A little help?
SIGINT
ANALYTICS
Analytic Results: [REDACTED]<msnpassport> 2012-Mar-01 09:58:46 Z
[REDACTED] View in Target Profiler
Had direct communications with a targeted identifier?
[REDACTED]
Seen in captured media? (no results)
Seen in CNE? (no results)
Seen in France?
Seen on a targeted identifier's contact lists, or has a targeted contact? (no results)
[REDACTED]
Had direct communications with a targeted identifier?
First heard: 2012-Mar-01 08:58:42 Z Last heard: 2012-Mar-05 07:55:40 Z
```
| Date | Description | Source |
|------------------------|---------------------------------------------------------|--------|
| 2012-Mar-05 07:55:40 Z | [REDACTED]<msnpassport> cc'd email to [REDACTED]<yahoo> | UUID: [REDACTED]<br>SIGAD/PDDG: DS-200A/C4<br>Case notation: [REDACTED]<br>Legal authority category: EO12333 |
| 2012-Mar-04 09:58:46 Z | [REDACTED]<msnpassport> was bcc'd on email from [REDACTED]<msnpassport><br>[REDACTED]<msnpassport> | UUID: [REDACTED]<br>SIGAD/PDDG: DS-200A/C4<br>Case notation: [REDACTED]<br>Legal authority category: EO12333<br> |
| 2012-Mar-02 10:56:43 Z | [REDACTED]<msnpassport> received email from [REDACTED]<msnpassport> | UUID: [REDACTED]<br>SIGAD/PDDG: DS-200B/C4<br>Case notation: [REDACTED]<br>Legal authority category: EO12333 |
| 2012-Mar-01 08:58:42 Z |[REDACTED]<msnpassport> sent email to [REDACTED]<yahoo> | UUID: [REDACTED]<br>SIGAD/PDDG: US-3171/T8<br>Case notation: [REDACTED]<br>Legal authority category: EO12333 |
```
Back to top
(U//FOUO) Page Publisher: Identifier Scoreboard, T1422 | Content [REDACTED] T14 | Last Reviewed: 15-Dec-2010 | Last Modified: 23-Feb-2012
(U//FOUO) Derived From: NSA/CSSM 1-52 | Declassify On: 20320108 | Dated: 08 January 2007
Dynamic Page Highest Possible Classification is TOP SECRET//SI//REL TO USA, FVEY
```
TOP SECRET//COMINT//REL TO USA, CAN, AUS, GBR, NZL
9
---
TOP SECRET//COMINT//REL TO USA, CAN, AUS, GBR, NZL
## SIGINT Analytics Mode - Detailed View
```
Had direct communications with a targeted identifier?
First heard: 2012-Mar-01 08:58:42 Z Last heard: 2012-Mar-05 07:55:40 Z
```
| Date | Description | Source |
|------------------------|---------------------------------------------------------|--------|
| 2012-Mar-05 07:55:40 Z | [REDACTED]<msnpassport> cc'd email to [REDACTED]<yahoo> | UUID: [REDACTED]<br>SIGAD/PDDG: DS-200A/C4<br>Case notation: [REDACTED]<br>Legal authority category: EO12333 |
| 2012-Mar-04 09:58:46 Z | [REDACTED]<msnpassport> was bcc'd on email from [REDACTED]<msnpassport><br>[REDACTED]<msnpassport> | UUID: [REDACTED]<br>SIGAD/PDDG: DS-200A/C4<br>Case notation: [REDACTED]<br>Legal authority category: EO12333<br> |
| 2012-Mar-02 10:56:43 Z | [REDACTED]<msnpassport> received email from [REDACTED]<msnpassport> | UUID: [REDACTED]<br>SIGAD/PDDG: DS-200B/C4<br>Case notation: [REDACTED]<br>Legal authority category: EO12333 |
| 2012-Mar-01 08:58:42 Z |[REDACTED]<msnpassport> sent email to [REDACTED]<yahoo> | UUID: [REDACTED]<br>SIGAD/PDDG: US-3171/T8<br>Case notation: [REDACTED]<br>Legal authority category: EO12333 |
```
Back to top
```
Go view target knowledge
Go view content
Add new knowledge
External links to guide next steps in analysis
TOP SECRET//COMINT//REL TO USA, CAN, AUS, GBR, NZL
10
---
TOP SECRET//COMINT//REL TO USA, CAN, AUS, GBR, NZL
## ECHOBASE Analytics Architecture
Initial set of analytic questions
* Most running within GHOSTMACHINE framework
* Limited contributors
* GHOSTMACHINE Analytic Engine provides
* QFD hosting of analytic results
* RESTful query interface
Future analytics
* multiple organizations/frameworks
Daily Feeds
Targeting
* OCTAVE
* UTT
Targeted identifiers
GHOSTMACHINE
Selector List
Seeds
Seeded Analytic
Seeded Analytic
Analytic
Analytic
GM Analytic Engine
QFD
QFD
QFD
QFD
QFD
QFD
Query
QFDs Svc
Bulk feeds of analytics results
T12
CDP
User DN, justification, leads & which QFDs ("domains")
Log queries
WAVELEGAL
Check user authorizations
CASport
Check user authorizations
Direct service query
Future analytic service
Future analytic
?
Bulk feed of analytic results
Non-GM Analytic
FGS
TOP SECRET//COMINT//REL TO USA, CAN, AUS, GBR, NZL
11
---
TOP SECRET//COMINT//REL TO USA, CAN, AUS, GBR, NZL
## 2012 Olympics Sharing
GCHQ
Releasable targeted identifiers
Seeded Analytic
Seeded Analytic
Seeded Analytic
Seeded Analytic
Job Tracker
(GCHQ architecture details omitted)
NSA
Lineup query details
User DN, justification, leads & which QFDs ("domains")
Daily Feeds
Targeting
* OCTAVE
* UTT
Targeted identifiers
GHOSTMACHINE
Selector List
Seeds
Seeded Analytic
Seeded Analytic
Analytic
Analytic
GM Analytic Engine
QFD
QFD
QFD
QFD
QFD
QFD
Query
QFDs Svc
Bulk feeds of analytics results
T12
CDP
User DN, justification, leads & which QFDs ("domains")
Log queries
WAVELEGAL
Check user authorizations
CASport
Check user authorizations
Bulk feed of analytic results
Non-GM Analytic
FGS
TOP SECRET//COMINT//REL TO USA, CAN, AUS, GBR, NZL
12
---
TOP SECRET//COMINT//REL TO USA, CAN, AUS, GBR, NZL
## 2012 Olympics Support
* NSA SID Leads Evaluation Cell
* Triage of Olympics-based leads through the event
* Leverage both NSA and GCHQ-produced analytics
* Greater SID-wide usage following the Olympic period
TOP SECRET//COMINT//REL TO USA, CAN, AUS, GBR, NZL
13
---
TOP SECRET//COMINT//REL TO USA, CAN, AUS, GBR, NZL
## Contact/Information
- Briefers:
- [REDACTED]
- [REDACTED]
- ECHOBASE Alias:
- [REDACTED]
- NSA WikiInfo page:
- [REDACTED]
TOP SECRET//COMINT//REL TO USA, CAN, AUS, GBR, NZL
14