20 KiB
APPLIED RESEARCH
Target Detection Identifiers
[REDACTED]
March 2009
© This information is exempt under the Freedom of Information Act 2000 (FOIA) and may be exempt under other UK information legislation. Refer any FOIA queries to [REDACTED].
Slide 1
APPLIED RESEARCH
UK SECRET STRAP2 COMINT ORCON
High-Speed Internet Processing
TCP SYN
GET /
User-Agent: Mozilla 4.1, IE5
Host:www.google.com
Cookie:ik=xzxsrzczccZ
TCP FIN
...
09:28:01 2008-10-13 [REDACTED] 7776 80 GET / Cookie: ik= qyzwww.....
09:28:13 2008-10-13 [REDACTED] 3456 80 GET / Cookie:ik=xzxsrzczccz
...
Google™
Event data sent to bulk store
© Crown Copyright. All rights reserved. This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to [REDACTED] Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.
UK SECRET STRAP2 COMINT ORCON Slide 2
UK SECRET STRAP2 COMINT ORCON
High-Speed Internet Processing
-
Bulk events key to SIGINT success on Internet
-
Event types that are valuable for Intelligence change (quickly)
- 2000 SMTP/POP3
- 2001 Webmail
- ...
- 2007 vBulletin
- 2008 Social Networks,...,?
-
GCHQ's Applied Research are pioneering ways of dealing with this:
- Presence Events (TDI)
- Very large scale high speed flat file storage to bulk store TDIs
- Just enough data marts
© Crown Copyright. All rights reserved. This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to [REDACTED] Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.
UK SECRET STRAP2 COMINT ORCON Slide 3
UK SECRET STRAP2 COMINT ORCON
IP Packet Information
-
Many possible types of information
-
Many techniques available
-
HTTP Get requests dominate cutting edge techniques
-
To get Intelligence value Information must relate to a person or device... a TDI
© Crown Copyright. All rights reserved. This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to [REDACTED] Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.
UK SECRET STRAP2 COMINT ORCON Slide 4
UK SECRET STRAP2 COMINT ORCON
TDI ...?
;
© Crown Copyright. A [REDACTED] other UK information legislation. Refer disclosure requests [REDACTED] under Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.
UK SECRET STRAP2 COMINT ORCON Slide 5
UK SECRET STRAP2 COMINT ORCON
TDI ...?
;
© Crown Copyright. [REDACTED] Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.
UK SECRET STRAP2 COMINT ORCON Slide 6
UK SECRET STRAP2 COMINT ORCON
TDI
Target Detection Identifier
© Crown Copyright. All rights reserved. This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to [REDACTED] Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.
UK SECRET STRAP2 COMINT ORCON Slide 7
UK SECRET STRAP2 COMINT ORCON
TDI
Target Detection Identifier
Who When Where (doing) What
© Crown Copyright. All rights reserved. This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to [REDACTED] Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.
UK SECRET STRAP2 COMINT ORCON Slide 8
UK SECRET STRAP2 COMINT ORCON
TDI
Target Detection Identifier
Who When Where (doing) What
Fundamental atom of the Internet age.
© Crown Copyright. All rights reserved. This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to [REDACTED] Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.
UK SECRET STRAP2 COMINT ORCON Slide 9
UK SECRET STRAP2 COMINT ORCON
Target Detection Identifiers
-
DEFINITION
- TDIs are definite indicators of presence, that are unique and persistent for a user/machine.
-
Built on the familiar
- Telephony +44 – international phone code
- Signalling tells us this phone user is 'online'
-
Target Detection Identifiers
- Started with the Internet, mobile networks too.
- TDI is a 'SIGINT standardised code'.
- Not a standard managed by the ITU/ETSI.
- Extraction from packets much more complex.
© Crown Copyright. All rights reserved. This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to [REDACTED] Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.
UK SECRET STRAP2 COMINT ORCON Slide 10
UK SECRET STRAP2 COMINT ORCON
TDI sources
hi5
BBC
YAHOO!
msn Hotmail
YouTube
reddit
Windows Live
MAKTOOb
First Arabic /English e-mail
facebook
Ask
Google
Windows
amazon.com
yellow
vBulletin
WORDPRESS
Google Earth
CNN
© Crown Copyright. All rights re [REDACTED] disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to [REDACTED] Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.
UK SECRET STRAP2 COMINT ORCON Slide 11
UK SECRET STRAP2 COMINT ORCON
Target Detection Identifiers
-
70 distinct TDI types discovered.
-
2500 TDIs/sec (GET, de-duplicated)
-
=> 200 Million per day per 10Gbps
-
De-dupe rate ???
-
Cost – 250 hours per TDI
-
Automated discovery prototype
| TDI Type | TDI Location | User/Machine |
|---|---|---|
| Yahoo-Y-Cookie | Cookie | User |
| Yahoo-B-Cookie | Coookie | Machine |
| Google-IK | Request-URI | User |
| Paltalk-Nickname | Request-URI | User |
| MS-MUID-Cookie | Cookie | Machine |
| Google-SID-Cookie | Cookie | Machine |
| Maktoob-MEUser-Cookie | Cookie | User |
| Orkut-PREFID-Cookie | Cookie | User |
| Cloob-Username | Cookie | User |
© Crown Copyright. All rights reserved. This information is exempt from disclosure under the Fr [REDACTED] other UK information legislation. Refer disclosure requests to [REDACTED] Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.
UK SECRET STRAP2 COMINT ORCON Slide 12
SECRET
[REDACTED] Search GCHQ
CHQ Homepage | B AutoTDI
GET data
Report is for the 7-day period from 04/10/08 (16:20) to 11/10/08 (16:20).
A cross-subnet threshold of ≤ 10% has been applied.
A mean user transmission threshold of ≥ 10 has been applied.
Putative selectors http://, photos/, images/, files/ are blacklisted from this plot.
Click to sort by a given column.
| Domain | Context | Technology Selector | Example value | Bearer count | Observation count | Mean user transmission frequency | Cross-/16 percentage |
|---|---|---|---|---|---|---|---|
| Cookie | datr= | [REDACTED] | 8 | 671 | 12.98 | 3.51 | |
| Cookie | c_user= | [REDACTED] | 8 | 651 | 12.09 | 3.51 | |
| Cookie | __utmz= | [REDACTED] | 7 | 609 | 12.14 | 4.25 | |
| Cookie | __utma= | [REDACTED] | 7 | 609 | 12.44 | 3.56 | |
| Cookie | h_user= | [REDACTED] | 7 | 601 | 12.37 | 3.74 | |
| Cookie | __qca= | [REDACTED] | 6 | 364 | 10.38 | 4.97 | |
| reuters | Cookie | lv= | [REDACTED] | 6 | 336 | 10.67 | 0.24 |
| Cookie | next_path= | [REDACTED] | 6 | 323 | 18.63 | 9.18 | |
| live | Cookie | MUID= | [REDACTED] | 7 | 321 | 10.81 | 3.24 |
| reuters | Cookie | id= | [REDACTED] | 6 | 312 | 21.59 | 0.45 |
| URI | q= | [REDACTED] | 7 | 311 | 15.02 | 5.81 | |
| reuters | Cookie | ss= | [REDACTED] | 6 | 309 | 16.83 | 0.39 |
| yahoo | Cookie | B= | [REDACTED] | 7 | 307 | 10.76 | 2.76 |
| yahoo | Cookie | d= | [REDACTED] | 6 | 306 | 10.60 | 7.79 |
| youporn | Cookie | sid= | [REDACTED] | 5 | 290 | 24.90 | 1.96 |
| youporn | Cookie | __utma= | [REDACTED] | 5 | 282 | 24.23 | 1.65 |
| youporn | Cookie | __utmz= | [REDACTED] | 5 | 281 | 22.92 | 4.60 |
| reuters | Cookie | anonId= | [REDACTED] | 6 | 279 | 16.22 | 0.46 |
| youporn | Cookie | __qca= | [REDACTED] | 5 | 277 | 24.40 | 1.69 |
| yahoo | URI | p= | [REDACTED] | 7 | 277 | 31.18 | 6.89 |
| bebo | Cookie | bdaysession= | [REDACTED] | 7 | 275 | 27.19 | 2.06 |
| Cookie | LM= | [REDACTED] | 7 | 272 | 16.85 | 7.31 | |
| Cookie | ID= | [REDACTED] | 7 | 271 | 16.80 | 3.73 | |
| Cookie | TM= | [REDACTED] | 7 | 270 | 17.07 | 6.57 | |
| bebo | Cookie | Username= | [REDACTED] | 7 | 268 | 27.18 | 2.21 |
| bebo | Cookie | Email= | [REDACTED] | 7 | 268 | 27.67 | 2.24 |
| yahoo | Cookie | I_s= | [REDACTED] | 4 | 264 | 78.61 | 3.00 |
| Cookie | S= | [REDACTED] | 6 | 264 | 39.35 | 3.82 | |
| yahoo | Cookie | ip= | [REDACTED] | 3 | 253 | 14.24 | 2.54 |
| yieldmanager | Cookie | uid= | [REDACTED] | 7 | 251 | 66.03 | 1.01 |
| reuters | Cookie | RaptTracker= | [REDACTED] | 6 | 242 | 14.24 | 0.48 |
| yahoo | Referer | p= | [REDACTED] | 4 | 242 | 17.85 | 7.19 |
UK SECRET STRAP2 COMINT ORCON
TDI Applications
-
Bulk store of all TDIs seen in last 6 months [MUTANT BROTH]
-
Bulk store TDI correlations (6 months) [AUTO ASSOC]
-
Bulk store TDI <-> website correlations (6 months) [KARMA POLICE]
-
Bulk store TDI vBulletin activity [INFINITE MONKEYS]
-
Bulk store TDI Social Networking Site activity [SOCIAL ANIMAL]
-
Bulk store web search requests [MEMORY HOLE]
-
Bulk store Google Earth requests [MARBLED GECKO]
-
Bulk store of Host.Referer references [HRMAP]
© Crown [REDACTED] Copyright. All rights reserved. This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to [REDACTED] Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.
UK SECRET STRAP2 COMINT ORCON Slide 14
SECRET
Edit View History Bookmarks Tools Help
B [REDACTED] Search GCHQ
s wiki 10gR2 fsrf PE ct blog /em/ im AR b13sw eci wiki ft wikipedia
MUTANT BROTH | 401 Authorization Required
UTANT BROTH
Identifier Search | IP Address Search | Password Search
Welcome
Logged in as [REDACTED] all queries logged for audit.
Database currently contains identifiers from the period Tue Dec 25 16:26:40 2007 to Fri Jun 20 22:13:19 2008 (18.41 billion rows of as 07-JUN-08).
Warning: data for period(s):-
* Fri Jun 20 22:13:21 2008 - Tue Jun 24 09:33:20 2008
is loaded, but currently unavailable for query due to index building. The rest of the database can be queried as usual during the rebuild.
Search for Identifiers
* If allow wildcards is ticked, % and _ are multi-character (bob%) and single-character (b_b@hotmail.com) wildcards.
* Queries are always case-sensitive (bob@hotmail.com ≠ BOB@hotmail.com ≠ BOB@HOTMAIL.COM). There is an option to automatically convert to lowercase.
* For bulk queries, paste in a list of identifiers separated by newlines (one per line).
* You can enter a minimum/maximum date for the search: default is to search all available selectors
[REDACTED]
MIRANDA 20135
JIC 1
Purpose NS
Reason demo
Execute
Allow wildcards
Convert to lowercase before searching
Matching Identifiers
The following identifiers have been found in the MUTANT BROTH database.
Select those that match your target(s) to generate a summary of target activity.
| TDI type | TDI value |
|---|---|
| Chat-MS-Messenger | [REDACTED]@hotmail.fr |
SECRET
Edit View History Bookmarks Tools Help
B [REDACTED] Search GCHQ
s wiki 10gR2 fsrf PE ct blog /em/ im AR b13sw eci wiki ft wikipedia
MUTANT BROTH | 401 Authorization Required
UTANT BROTH
Identifier Search | IP Address Search | Password Search
Welcome
Logged in as [REDACTED] all queries logged for audit.
Database currently contains identifiers from the period Tue Dec 25 16:26:40 2007 to Fri Jun 20 22:13:19 2008 (18.41 billion rows of as 07-JUN-08).
Warning: data for period(s):-
* Fri Jun 20 22:1
is loaded, but curre rebuild.
(18.41 billion rows of as 07-JUN-08).
Search for Ident
* If allow wildcards is
* Queries are always case-sensitive (bob@hotmail.com ≠ BOB@hotmail.com ≠ BOB@HOTMAIL.COM). There is an option to automatically convert to lowercase.
* For bulk queries, paste in a list of identifiers separated by newlines (one per line).
* You can enter a minimum/maximum date for the search: default is to search all available selectors
[REDACTED]
MIRANDA 20135
JIC 1
Purpose NS
Reason demo
Execute
Allow wildcards
Convert to lowercase before searching
Matching Identifiers
The following identifiers have been found in the MUTANT BROTH database.
Select those that match your target(s) to generate a summary of target activity.
| TDI type | TDI value |
|---|---|
| Chat-MS-Messenger | [REDACTED]@hotmail.fr |
SECRET
WHEN WHERE WHAT WHO
Edit View History Bookmarks Tools Help
B [REDACTED] Search GCHQ
wiki 10gR2 fsrf PE ct blog /em/ im AR b13sw eci wiki ft wikipedia
MUTANT BROTH | 401 Authorization Required
[REDACTED] Chat Identifiers), the source IP Geo-Location identifies the other end of the communications link to the
Save as CSV
| Date | Time | Source IP | HHFP | Source IP Geo | Identifier Type | Identifier Value | Passw |
|---|---|---|---|---|---|---|---|
| 17/06/2008 | 17:08:44 | [REDACTED] | 6de32bb0 | 41.02;28.96;ISTANBUL; TR; 5MMM | Hi5-Email-Cookie | [REDACTED]@hotmail.com | |
| 17/08 | [REDACTED] | 6de32bb0 | 41.02[REDACTED] M | Hi5-Email-Cookie | [REDACTED]@hotmail.com | ||
| 17/06/2008 | 16:55:21 | 6de32bb0 | 41.9022;-87.6726; CHICAGO; US;5 | [REDACTED] | [REDACTED]@hotmail.com | ||
| 17/06/2008 | 16:55:16 | [REDACTED] | 6de32bb0 | 41.02;28.96; ISTANBUL; TR; 5MMM | Hi5-Email-Cookie | [REDACTED]@hotmail.com | |
| 17/06/2008 | 16:54:47 | 6de32bb0 | 41.9022;-87.6726; CHICAGO; US;5MMM | Hi5-Email-Cookie | [REDACTED]@hotmail.com | ||
| 17/06/2008 | 16:52:13 | [REDACTED] | 6de32bb0 | 39.94;32.86; ANKARA; TR; 5MMM | Hi5-Email-Cookie | [REDACTED]@hotmail.com | |
| 15/06/2008 | 19:20:33 | [REDACTED] | de8bdc48 | 33.5;36.3; DIMASHQ;SY; 5MLV | Hi5-Email-Cookie | [REDACTED]@hotmail.com |
UK SECRET STRAP2 COMINT ORCON
Other Bulk Event Applications
-
Most events that can be associated back to TDIs:
-
File Transfer Signature (eg proof of life videos)
-
Detection by Internet profile – eg 'Dead Letter Drop'.
-
Yahoo webcam images
-
Airline reservation confirmation emails
© Crown Copyright. All rights reserved. This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to [REDACTED] Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.
UK SECRET STRAP2 COMINT ORCON Slide 18