511 lines
No EOL
20 KiB
Markdown
511 lines
No EOL
20 KiB
Markdown
APPLIED RESEARCH
|
||
|
||
# Target Detection Identifiers
|
||
|
||
[REDACTED]
|
||
|
||
March 2009
|
||
|
||
© This information is exempt under the Freedom of Information Act 2000 (FOIA) and may be exempt under other UK information legislation. Refer any FOIA queries to [REDACTED].
|
||
|
||
Slide 1
|
||
|
||
APPLIED
|
||
RESEARCH
|
||
|
||
---
|
||
|
||
UK SECRET STRAP2 COMINT ORCON
|
||
|
||
## High-Speed Internet Processing
|
||
|
||
```
|
||
TCP SYN
|
||
|
||
GET /
|
||
User-Agent: Mozilla 4.1, IE5
|
||
Host:www.google.com
|
||
Cookie:ik=xzxsrzczccZ
|
||
|
||
TCP FIN
|
||
|
||
...
|
||
09:28:01 2008-10-13 [REDACTED] 7776 80 GET / Cookie: ik= qyzwww.....
|
||
09:28:13 2008-10-13 [REDACTED] 3456 80 GET / Cookie:ik=xzxsrzczccz
|
||
...
|
||
|
||
Google™
|
||
|
||
Event data sent to bulk store
|
||
```
|
||
|
||
© Crown Copyright. All rights reserved. This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to [REDACTED]
|
||
Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.
|
||
|
||
UK SECRET STRAP2 COMINT ORCON
|
||
Slide 2
|
||
|
||
---
|
||
|
||
UK SECRET STRAP2 COMINT ORCON
|
||
|
||
## High-Speed Internet Processing
|
||
|
||
* Bulk events key to SIGINT success on Internet
|
||
|
||
* Event types that are valuable for Intelligence change (quickly)
|
||
* 2000 SMTP/POP3
|
||
* 2001 Webmail
|
||
* ...
|
||
* 2007 vBulletin
|
||
* 2008 Social Networks,...,?
|
||
|
||
* GCHQ's Applied Research are pioneering ways of dealing with this:
|
||
* Presence Events (TDI)
|
||
* Very large scale high speed flat file storage to bulk store TDIs
|
||
* Just enough data marts
|
||
|
||
© Crown Copyright. All rights reserved. This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to [REDACTED]
|
||
Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.
|
||
|
||
UK SECRET STRAP2 COMINT ORCON
|
||
Slide 3
|
||
|
||
---
|
||
|
||
UK SECRET STRAP2 COMINT ORCON
|
||
|
||
## IP Packet Information
|
||
|
||
* Many possible types of information
|
||
|
||
* Many techniques available
|
||
|
||
* HTTP Get requests dominate cutting edge techniques
|
||
|
||
* To get Intelligence value Information must relate to a person or device... a TDI
|
||
|
||
© Crown Copyright. All rights reserved. This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to [REDACTED]
|
||
Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.
|
||
|
||
UK SECRET STRAP2 COMINT ORCON
|
||
Slide 4
|
||
|
||
---
|
||
|
||
UK SECRET STRAP2 COMINT ORCON
|
||
|
||
## TDI ...?
|
||
|
||
;
|
||
|
||
© Crown Copyright. A [REDACTED] other UK information legislation. Refer disclosure requests [REDACTED] under
|
||
Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.
|
||
|
||
UK SECRET STRAP2 COMINT ORCON
|
||
Slide 5
|
||
|
||
---
|
||
|
||
UK SECRET STRAP2 COMINT ORCON
|
||
|
||
## TDI ...?
|
||
|
||
;
|
||
|
||
© Crown Copyright. [REDACTED]
|
||
Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.
|
||
|
||
UK SECRET STRAP2 COMINT ORCON
|
||
Slide 6
|
||
|
||
---
|
||
|
||
UK SECRET STRAP2 COMINT ORCON
|
||
|
||
## TDI
|
||
|
||
Target
|
||
Detection
|
||
Identifier
|
||
|
||
© Crown Copyright. All rights reserved. This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to [REDACTED]
|
||
Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.
|
||
|
||
UK SECRET STRAP2 COMINT ORCON
|
||
Slide 7
|
||
|
||
---
|
||
|
||
UK SECRET STRAP2 COMINT ORCON
|
||
|
||
## TDI
|
||
|
||
Target
|
||
Detection
|
||
Identifier
|
||
|
||
Who
|
||
When
|
||
Where
|
||
(doing) What
|
||
|
||
© Crown Copyright. All rights reserved. This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to [REDACTED]
|
||
Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.
|
||
|
||
UK SECRET STRAP2 COMINT ORCON
|
||
Slide 8
|
||
|
||
---
|
||
|
||
UK SECRET STRAP2 COMINT ORCON
|
||
|
||
## TDI
|
||
|
||
Target
|
||
Detection
|
||
Identifier
|
||
|
||
Who
|
||
When
|
||
Where
|
||
(doing) What
|
||
|
||
## Fundamental atom of the Internet age.
|
||
|
||
© Crown Copyright. All rights reserved. This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to [REDACTED]
|
||
Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.
|
||
|
||
UK SECRET STRAP2 COMINT ORCON
|
||
Slide 9
|
||
|
||
---
|
||
|
||
UK SECRET STRAP2 COMINT ORCON
|
||
|
||
## Target Detection Identifiers
|
||
|
||
* DEFINITION
|
||
* TDIs are definite indicators of presence, that are unique and persistent for a user/machine.
|
||
|
||
* Built on the familiar
|
||
* Telephony +44 – international phone code
|
||
* Signalling tells us this phone user is 'online'
|
||
|
||
* Target Detection Identifiers
|
||
* Started with the Internet, mobile networks too.
|
||
* TDI is a 'SIGINT standardised code'.
|
||
* Not a standard managed by the ITU/ETSI.
|
||
* Extraction from packets much more complex.
|
||
|
||
© Crown Copyright. All rights reserved. This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to [REDACTED]
|
||
Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.
|
||
|
||
UK SECRET STRAP2 COMINT ORCON
|
||
Slide 10
|
||
|
||
---
|
||
|
||
UK SECRET STRAP2 COMINT ORCON
|
||
|
||
## TDI sources
|
||
|
||
```
|
||
hi5
|
||
BBC
|
||
YAHOO!
|
||
msn Hotmail
|
||
YouTube
|
||
reddit
|
||
Windows Live
|
||
MAKTOOb
|
||
First Arabic /English e-mail
|
||
facebook
|
||
Ask
|
||
Google
|
||
Windows
|
||
amazon.com
|
||
yellow
|
||
vBulletin
|
||
WORDPRESS
|
||
Google Earth
|
||
CNN
|
||
```
|
||
|
||
© Crown Copyright. All rights re [REDACTED] disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to [REDACTED]
|
||
Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.
|
||
|
||
UK SECRET STRAP2 COMINT ORCON
|
||
Slide 11
|
||
|
||
---
|
||
|
||
UK SECRET STRAP2 COMINT ORCON
|
||
|
||
## Target Detection Identifiers
|
||
|
||
* 70 distinct TDI types discovered.
|
||
|
||
* 2500 TDIs/sec (GET, de-duplicated)
|
||
|
||
* => 200 Million per day per 10Gbps
|
||
|
||
* De-dupe rate ???
|
||
|
||
* Cost – 250 hours per TDI
|
||
|
||
* Automated discovery prototype
|
||
|
||
| TDI Type | TDI Location | User/Machine |
|
||
| :------------------- | :----------- | :----------- |
|
||
| Yahoo-Y-Cookie | Cookie | User |
|
||
| Yahoo-B-Cookie | Coookie | Machine |
|
||
| Google-IK | Request-URI | User |
|
||
| Paltalk-Nickname | Request-URI | User |
|
||
| MS-MUID-Cookie | Cookie | Machine |
|
||
| Google-SID-Cookie | Cookie | Machine |
|
||
| Maktoob-MEUser-Cookie| Cookie | User |
|
||
| Orkut-PREFID-Cookie | Cookie | User |
|
||
| Cloob-Username | Cookie | User |
|
||
|
||
© Crown Copyright. All rights reserved. This information is exempt from disclosure under the Fr [REDACTED] other UK information legislation. Refer disclosure requests to [REDACTED]
|
||
Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.
|
||
|
||
UK SECRET STRAP2 COMINT ORCON
|
||
Slide 12
|
||
|
||
---
|
||
|
||
SECRET
|
||
|
||
```
|
||
[REDACTED] Search GCHQ
|
||
CHQ Homepage | B AutoTDI
|
||
GET data
|
||
|
||
Report is for the 7-day period from 04/10/08 (16:20) to 11/10/08 (16:20).
|
||
A cross-subnet threshold of ≤ 10% has been applied.
|
||
A mean user transmission threshold of ≥ 10 has been applied.
|
||
Putative selectors http://, photos/, images/, files/ are blacklisted from this plot.
|
||
Click to sort by a given column.
|
||
```
|
||
|
||
|
||
| Domain | Context | Technology Selector | Example value | Bearer count | Observation count | Mean user transmission frequency | Cross-/16 percentage |
|
||
| ------------ | ------- | ------------------- | ------------- | ------------ | ----------------- | -------------------------------- | -------------------- |
|
||
| facebook | Cookie | datr= | [REDACTED] | 8 | 671 | 12.98 | 3.51 |
|
||
| facebook | Cookie | c_user= | [REDACTED] | 8 | 651 | 12.09 | 3.51 |
|
||
| facebook | Cookie | __utmz= | [REDACTED] | 7 | 609 | 12.14 | 4.25 |
|
||
| facebook | Cookie | __utma= | [REDACTED] | 7 | 609 | 12.44 | 3.56 |
|
||
| facebook | Cookie | h_user= | [REDACTED] | 7 | 601 | 12.37 | 3.74 |
|
||
| facebook | Cookie | __qca= | [REDACTED] | 6 | 364 | 10.38 | 4.97 |
|
||
| reuters | Cookie | lv= | [REDACTED] | 6 | 336 | 10.67 | 0.24 |
|
||
| facebook | Cookie | next_path= | [REDACTED] | 6 | 323 | 18.63 | 9.18 |
|
||
| live | Cookie | MUID= | [REDACTED] | 7 | 321 | 10.81 | 3.24 |
|
||
| reuters | Cookie | id= | [REDACTED] | 6 | 312 | 21.59 | 0.45 |
|
||
| google | URI | q= | [REDACTED] | 7 | 311 | 15.02 | 5.81 |
|
||
| reuters | Cookie | ss= | [REDACTED] | 6 | 309 | 16.83 | 0.39 |
|
||
| yahoo | Cookie | B= | [REDACTED] | 7 | 307 | 10.76 | 2.76 |
|
||
| yahoo | Cookie | d= | [REDACTED] | 6 | 306 | 10.60 | 7.79 |
|
||
| youporn | Cookie | sid= | [REDACTED] | 5 | 290 | 24.90 | 1.96 |
|
||
| youporn | Cookie | __utma= | [REDACTED] | 5 | 282 | 24.23 | 1.65 |
|
||
| youporn | Cookie | __utmz= | [REDACTED] | 5 | 281 | 22.92 | 4.60 |
|
||
| reuters | Cookie | anonId= | [REDACTED] | 6 | 279 | 16.22 | 0.46 |
|
||
| youporn | Cookie | __qca= | [REDACTED] | 5 | 277 | 24.40 | 1.69 |
|
||
| yahoo | URI | p= | [REDACTED] | 7 | 277 | 31.18 | 6.89 |
|
||
| bebo | Cookie | bdaysession= | [REDACTED] | 7 | 275 | 27.19 | 2.06 |
|
||
| google | Cookie | LM= | [REDACTED] | 7 | 272 | 16.85 | 7.31 |
|
||
| google | Cookie | ID= | [REDACTED] | 7 | 271 | 16.80 | 3.73 |
|
||
| google | Cookie | TM= | [REDACTED] | 7 | 270 | 17.07 | 6.57 |
|
||
| bebo | Cookie | Username= | [REDACTED] | 7 | 268 | 27.18 | 2.21 |
|
||
| bebo | Cookie | Email= | [REDACTED] | 7 | 268 | 27.67 | 2.24 |
|
||
| yahoo | Cookie | I_s= | [REDACTED] | 4 | 264 | 78.61 | 3.00 |
|
||
| google | Cookie | S= | [REDACTED] | 6 | 264 | 39.35 | 3.82 |
|
||
| yahoo | Cookie | ip= | [REDACTED] | 3 | 253 | 14.24 | 2.54 |
|
||
| yieldmanager | Cookie | uid= | [REDACTED] | 7 | 251 | 66.03 | 1.01 |
|
||
| reuters | Cookie | RaptTracker= | [REDACTED] | 6 | 242 | 14.24 | 0.48 |
|
||
| yahoo | Referer | p= | [REDACTED] | 4 | 242 | 17.85 | 7.19 |
|
||
|
||
---
|
||
|
||
UK SECRET STRAP2 COMINT ORCON
|
||
|
||
## TDI Applications
|
||
|
||
* Bulk store of all TDIs seen in last 6 months [MUTANT BROTH]
|
||
|
||
* Bulk store TDI correlations (6 months) [AUTO ASSOC]
|
||
|
||
* Bulk store TDI <-> website correlations (6 months) [KARMA POLICE]
|
||
|
||
* Bulk store TDI vBulletin activity [INFINITE MONKEYS]
|
||
|
||
* Bulk store TDI Social Networking Site activity [SOCIAL ANIMAL]
|
||
|
||
* Bulk store web search requests [MEMORY HOLE]
|
||
|
||
* Bulk store Google Earth requests [MARBLED GECKO]
|
||
|
||
* Bulk store of Host.Referer references [HRMAP]
|
||
|
||
© Crown [REDACTED] Copyright. All rights reserved. This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to [REDACTED]
|
||
Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.
|
||
|
||
UK SECRET STRAP2 COMINT ORCON
|
||
Slide 14
|
||
|
||
---
|
||
|
||
SECRET
|
||
|
||
```
|
||
Edit View History Bookmarks Tools Help
|
||
B [REDACTED] Search GCHQ
|
||
s wiki 10gR2 fsrf PE ct blog /em/ im AR b13sw eci wiki ft wikipedia
|
||
|
||
MUTANT BROTH | 401 Authorization Required
|
||
|
||
UTANT BROTH
|
||
|
||
Identifier Search | IP Address Search | Password Search
|
||
|
||
Welcome
|
||
|
||
Logged in as [REDACTED] all queries logged for audit.
|
||
Database currently contains identifiers from the period Tue Dec 25 16:26:40 2007 to Fri Jun 20 22:13:19 2008 (18.41 billion rows of as 07-JUN-08).
|
||
Warning: data for period(s):-
|
||
* Fri Jun 20 22:13:21 2008 - Tue Jun 24 09:33:20 2008
|
||
is loaded, but currently unavailable for query due to index building. The rest of the database can be queried as usual during the rebuild.
|
||
|
||
Search for Identifiers
|
||
* If allow wildcards is ticked, % and _ are multi-character (bob%) and single-character (b_b@hotmail.com) wildcards.
|
||
* Queries are always case-sensitive (bob@hotmail.com ≠ BOB@hotmail.com ≠ BOB@HOTMAIL.COM). There is an option to automatically convert to lowercase.
|
||
* For bulk queries, paste in a list of identifiers separated by newlines (one per line).
|
||
* You can enter a minimum/maximum date for the search: default is to search all available selectors
|
||
|
||
[REDACTED]
|
||
|
||
MIRANDA 20135
|
||
JIC 1
|
||
Purpose NS
|
||
Reason demo
|
||
Execute
|
||
|
||
Allow wildcards
|
||
Convert to lowercase before searching
|
||
|
||
Matching Identifiers
|
||
|
||
The following identifiers have been found in the MUTANT BROTH database.
|
||
Select those that match your target(s) to generate a summary of target activity.
|
||
```
|
||
|
||
| TDI type | TDI value |
|
||
| :---------------- | :---------------- |
|
||
| Chat-MS-Messenger | [REDACTED]@hotmail.fr |
|
||
|
||
---
|
||
|
||
SECRET
|
||
|
||
```
|
||
Edit View History Bookmarks Tools Help
|
||
B [REDACTED] Search GCHQ
|
||
s wiki 10gR2 fsrf PE ct blog /em/ im AR b13sw eci wiki ft wikipedia
|
||
|
||
MUTANT BROTH | 401 Authorization Required
|
||
|
||
UTANT BROTH
|
||
|
||
Identifier Search | IP Address Search | Password Search
|
||
|
||
Welcome
|
||
|
||
Logged in as [REDACTED] all queries logged for audit.
|
||
Database currently contains identifiers from the period Tue Dec 25 16:26:40 2007 to Fri Jun 20 22:13:19 2008 (18.41 billion rows of as 07-JUN-08).
|
||
|
||
Warning: data for period(s):-
|
||
* Fri Jun 20 22:1
|
||
is loaded, but curre rebuild.
|
||
|
||
(18.41 billion rows of as 07-JUN-08).
|
||
|
||
Search for Ident
|
||
* If allow wildcards is
|
||
* Queries are always case-sensitive (bob@hotmail.com ≠ BOB@hotmail.com ≠ BOB@HOTMAIL.COM). There is an option to automatically convert to lowercase.
|
||
* For bulk queries, paste in a list of identifiers separated by newlines (one per line).
|
||
* You can enter a minimum/maximum date for the search: default is to search all available selectors
|
||
|
||
[REDACTED]
|
||
|
||
MIRANDA 20135
|
||
JIC 1
|
||
Purpose NS
|
||
Reason demo
|
||
Execute
|
||
|
||
Allow wildcards
|
||
Convert to lowercase before searching
|
||
|
||
Matching Identifiers
|
||
|
||
The following identifiers have been found in the MUTANT BROTH database.
|
||
Select those that match your target(s) to generate a summary of target activity.
|
||
```
|
||
|
||
| TDI type | TDI value |
|
||
| :---------------- | :---------------- |
|
||
| Chat-MS-Messenger | [REDACTED]@hotmail.fr |
|
||
|
||
---
|
||
|
||
SECRET
|
||
|
||
WHEN
|
||
WHERE
|
||
WHAT
|
||
WHO
|
||
|
||
```
|
||
Edit View History Bookmarks Tools Help
|
||
B [REDACTED] Search GCHQ
|
||
wiki 10gR2 fsrf PE ct blog /em/ im AR b13sw eci wiki ft wikipedia
|
||
|
||
MUTANT BROTH | 401 Authorization Required
|
||
|
||
[REDACTED] Chat Identifiers), the source IP Geo-Location identifies the other end of the communications link to the
|
||
|
||
Save as CSV
|
||
```
|
||
|
||
| Date | Time | Source IP | HHFP | Source IP Geo | Identifier Type | Identifier Value | Passw |
|
||
| :----------- | :------- | :-------- | :------- | :----------------------------------- | :-------------- | :--------------- | :---- |
|
||
| 17/06/2008 | 17:08:44 | [REDACTED]| 6de32bb0 | 41.02;28.96;ISTANBUL; TR; 5MMM | Hi5-Email-Cookie| [REDACTED]@hotmail.com | |
|
||
| 17/08 | [REDACTED]| | 6de32bb0 | 41.02[REDACTED] M | Hi5-Email-Cookie | [REDACTED]@hotmail.com | |
|
||
| 17/06/2008 | 16:55:21 | | 6de32bb0 | 41.9022;-87.6726; CHICAGO; US;5 | [REDACTED] | [REDACTED]@hotmail.com | |
|
||
| 17/06/2008 | 16:55:16 | [REDACTED]| 6de32bb0 | 41.02;28.96; ISTANBUL; TR; 5MMM | Hi5-Email-Cookie| [REDACTED]@hotmail.com | |
|
||
| 17/06/2008 | 16:54:47 | | 6de32bb0 | 41.9022;-87.6726; CHICAGO; US;5MMM | Hi5-Email-Cookie| [REDACTED]@hotmail.com | |
|
||
| 17/06/2008 | 16:52:13 | [REDACTED]| 6de32bb0 | 39.94;32.86; ANKARA; TR; 5MMM | Hi5-Email-Cookie| [REDACTED]@hotmail.com | |
|
||
| 15/06/2008 | 19:20:33 | [REDACTED]| de8bdc48 | 33.5;36.3; DIMASHQ;SY; 5MLV | Hi5-Email-Cookie| [REDACTED]@hotmail.com | |
|
||
|
||
---
|
||
|
||
UK SECRET STRAP2 COMINT ORCON
|
||
|
||
## Other Bulk Event Applications
|
||
|
||
* Most events that can be associated back to TDIs:
|
||
|
||
* File Transfer Signature (eg proof of life videos)
|
||
|
||
* Detection by Internet profile – eg 'Dead Letter Drop'.
|
||
|
||
* Yahoo webcam images
|
||
|
||
* Airline reservation confirmation emails
|
||
|
||
© Crown Copyright. All rights reserved. This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to [REDACTED]
|
||
Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.
|
||
|
||
UK SECRET STRAP2 COMINT ORCON
|
||
Slide 18 |