snowchive-meta/data/full-log-vs-http/full-log-vs-http.md
2026-11-19 04:39:11 +02:00

28 KiB
Raw Blame History

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20291123

XKEYSCORE

Full Log vs. HTTP

11 June 2009

Derived From: NSA/CSSM 1-52 Dated: 20070108 Declassify On: 20291123

DERIVED FROM: NSA/CSSM 1-52 TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20291123 08


TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL

Full Log - Pros

  • The Full Log search gives you access to all DNI sessions collected by X-KEYSCORE
  • Data is indexed by the basic meta-data like IP Address, Country Codes Port, Casenotation, Application ID/Fingerprints etc
  • If you're only interested in content, Full Log will give you access to everything

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL


TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL

Full Log - Cons

  • However, in most cases there will be too many results in XKS to look through every piece of content by hand
  • To be more efficient, it's important to utilize the meta-data contained in the other search forms (E-mail Addresses, HTTP Activity, Extracted Files, Document Meta-data etc.)

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL


TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL

HTTP Activity

  • HTTP Activity is essentially all web-based activity from a user's internet browser (with some exceptions)
  • It includes, web-surfing, Internet Searching (like Google), Mapping Website (Google Earth/Maps) etc.
  • Most of this data will not contain a strong selector like E-mail address

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL


TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL

HTTP Activity

  • HTTP activity comes in two types: Client-to-Server "requests" cnn.com Server Server-to-Client "responses" User

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL


TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL

HTTP Activity Client-to-Server

GET /search?tab=urdu&order=sortboth&q=musharraf&start=3&scope=urdu&link=next HTTP/1.1
Accept: */*
Referer: http://search.bbc.co.uk/search?tab=urdu&order=sortboth&q=musharraf&start=2&scope=urdu
Accept-Language: en-us
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
Host: search.bbc.co.uk
Cookie: BBC-UID=b479a5f4ad230a53063d513630203acb22684634a0e0b164c45f96efc054cf950Mozilla%2f4%2e0%20%28co
Cache-Control: max-stale=U
Connection: Keep-Alive
X-BlueCoat-Via: 66808702E9A98546
Host URL Path URL Args
search.bbc.co.uk /search tab=urdu&order=sortboth&q=musharraf&start=3&scope=urdu&link=next
Search Terms Language Browser Via
musharraf en Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) 66808702E9A98546

Referer http://search.bbc.co.uk/search?tab=urdu&order=sortboth&q=musharraf&start=2&scope=urdu

Cookie BBC-UID=b479a5f4ad230a53063d513630203acb22684634a0e0b164c45f96efc054cf950Mozilla%2f4%2e0%20%28com

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL


TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL

Overlap

  • Full log contains basic information on every single DNI session XKS processes.
  • HTTP activity contains more detailed information on the subset of that data which is web-based (aka port 80 "internet browser" traffic)

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL


TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL

How the Search Forms Fit Together

Full Log of all DNI sessions collected Sessions from web based HTTP Activity

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL


TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL

Example #1

  • Analysis of 14 May Internet session of PK based target started in MARINA
TS USERID PHONE USER_A ACTIVITY USER_B
20090514 132353Z [REDACTED] logged in (im) 119.[REDACTED]
20090514 132416Z [REDACTED]@hotmail.com logged in (im) 119.[REDACTED]
20090514 132419Z [REDACTED]@hotmail.com logged in (im) 119.[REDACTED]
20090514 132834Z [REDACTED]@hotmail.com logged in (im) 119.[REDACTED]
20090514 132843Z [REDACTED]@hotmail.com logged in (im) 119.[REDACTED]
20090514 133517Z [REDACTED]@hotmail.com logged in (im) 119.[REDACTED]
20090514 133522Z [REDACTED]@hotmail.com logged in (im) 119.[REDACTED]

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL


TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL

Understand what is behind the IP

  • Ensure Activity on IP can be associated with Target
  • Understand IP usage Dynamic/Static
  • Research IP using Foxtrail/NKB
  • Is it a Proxy, DVBLAN, Dial-Up, DSL, etc
  • Is it Client to Server or Server to Client
  • Still not sure? User Activity pull for 5 minute period on Foreign IP

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL

Derived From: NSA/CSSM 1-52 Dated: 20070108 Declassify On: 20340601


TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL

HTTP with a Proxy

Request sent to Proxy Request sent to Server Response sent to Proxy Response sent to User

  • Performance: Proxy can cache responses for static pages
  • Censorship: Proxy can filter traffic
  • Security: Proxy can look for malware
  • Access-Control: Proxy can control access to restricted content

Proxy can be run by

  • a user
  • an ISP
  • a web-hosting company
  • a content-delivery network (i.e. Akamai)

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL


TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL

Proxies on the Internet

Web-Server Short-lived connections Single-User

Web-Servers Short-lived connections Multiple-users multiplexed

Web-Servers Long-lived connections Multiple-users multiplexed

The Internet

Direct-Connect

Mixed-Gateway

National-Level Proxy

Proxy-to-Proxy

Cache

User-to-Proxy

TOP SECRET//COMINT//REL TO USA, A


TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL

Example #1

  • The analyst then did an HTTP activity query to find all web surfing from that IP address within the same rough timeframe.
Search: HTTP Activity
Query Name: 14_may_activity
Justification: PK based IP address used by CT target
Datetime: Custom Start: 2009-05-14 13:30 Stop: 2009-05-14 14:15
IP Address: 119.[REDACTED] Either
IP Address: To
Port: From
Port: To

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL


14 May - Strange HTTP Activity

  • HTTP meta-data indicated strange web-based activity
Host URL Path
infoservice.inf.tu-dresden.de /mixcascadestatus/F30905FCD73B6B30CB5FEFD3250FD66EF4B32591
infoservice.inf.tu-dresden.de /infoservices
Browser
RPT-HTTPClient/0.4-dev
RPT-HTTPClient/0.4-dev

GET /mixcascadestatus/F30905FCD73B6B30CB5FEFD3250FD66EF4B32591 HTTP/1.1
Host: infoservice.inf.tu-dresden.de
Connection: Keep-Alive, TE
TE: trailers, deflate, gzip, compress
User-Agent: RPT-HTTPClient/0.4-dev
Cache-Control: no-cache
Pragma: no-cache

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL


TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL

14 May - Strange HTTP Activity

  • Indications from the HTTP activity
Browser
RPT-HTTPClient/0.4-dev
RPT-HTTPClient/0.4-dev

Note the strange User Agent/Browser

GET /mixcascadestatus/F30905FCD73B6B30CB5FEFD3250FD66EF4B32591 HTTP/1.1
Host: infoservice.inf.tu-dresden.de
Connection: Keep-Alive, TE
TE: trailers, deflate, gzip, compress
User-Agent: RPT-HTTPClient/0.4-dev
Cache-Control: no-cache
Pragma: no-cache

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL


TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL

Tip off to possible anonymizer

  • Open Source research indicated that this user agent was indicative of multi-cast traffic. likely tip off that this was some type of anonymizer
Browser
RPT-HTTPClient/0.4-dev
RPT-HTTPClient/0.4-dev

GET /mixcascadestatus/F30905FCD73B6B30CB5FEFD3250FD66EF4B32591 HTTP/1.1
Host: infoservice.inf.tu-dresden.de
Connection: Keep-Alive, TE
TE: trailers, deflate, gzip, compress
User-Agent: RPT-HTTPClient/0.4-dev
Cache-Control: no-cache
Pragma: no-cache

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL


TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL

HTTP Results led to Full Log Query

  • The two tu-dresden.de requests were the only HTTP activity seen within that timeframe, but given the open source research suggesting that the user agent was an anonymizing proxy a full log query was ran to identify all other traffic originating from that same IP address during the same time
Datetime End Fm IP
2009-05-14 13:47:33 119.[REDACTED]
2009-05-14 13:48:07 119.[REDACTED]
Host URL Path
infoservice.inf.tu-dresden.de /mixcascadestatus/F30905FCD73B6B30CB5FEFD3250FD66EF4B32591
infoservice.inf.tu-dresden.de /infoservices

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL


TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL

Full Log Results

Datetime End Fm IP To IP Fm Por To Po Application Type Application
2009-05-14 13:47:33 119.[REDACTED] 141.[REDACTED] 1495 80 web http/get
2009-05-14 13:48:07 119.[REDACTED] 65.5[REDACTED] 1502 443 network_encryption network_encryption/https
2009-05-14 13:48:07 119.[REDACTED] 141.[REDACTED] 1494 80 web http/get
2009-05-14 13:48:35 119.[REDACTED] 141.[REDACTED] 1134 6544 unknown unknown/tcp
2009-05-14 13:49:02 119.[REDACTED] 141.[REDACTED] 1134 6544 unknown unknown/tcp

Note the two HTTP activity (port 80) sessions were seen, but in addition there was one SSL (port 443) session and two unknown port 6544 sessions

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL


TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL

Full Log Results

Fm IP To IP Fm Por To Po Application Type Application Data Length Session Length
119.[REDACTED] 141.[REDACTED] 1134 6544 unknown unknown/tcp 6986 7803
119.[REDACTED] 141.[REDACTED] 1134 6544 unknown unknown/tcp 93088 93901

Of the unknown port 6544 traffic, the data length of the sessions indicated that a significant amount of data was leaving the Pakistan IP used by our target

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL


TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL

Full Log Results

The content appeared unreadable. Further analysis by CES and open source research showed that the content was encrypted

DNI Display | Raw Data | DNI Format

Services

No presentation available for this type of data. Try sending it to another service for a better view. Below is an attempt to display the document as plain text.

*bh00000-701&@MjKi80f1xSg1/-9001[F
li Elg?Kvp1OHO
POUsatdVq219010VZ) v100?~X;bbnUc8a+NCOr
"]
z.GFBnAsj<nh 40 E Ag/150161?piz/f4pFj10
Gj50K/4-UNqzCXjJQH14
TOXZeeeenetlemele Le80000/00elle..........00000000000
cwg/3日[8kzN?
X1.XkcYs
Hvj_3031!/UC5+X!p#0
NO M6K: 11,5XU2X2
*1ZY"fq:dR?:gRy2c+jtUVXJx4['3cAI/$13
ct❤❤❤❤❤[4+6-oswg_n2h+x+*$"z8|5:
*#VkOeyd $005/XdfDnH.5(+
-10
"C01r0+bfou-
oh_d❤❤❤❤❤❤+❤❤
ys:"bz4
<j614 C.UZ R099 500101
Ot+gWtP210/Ox-2 le<6C>5DN<44>i<EFBFBD>**=pY6D***********27(EIX-看
R

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL


TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL

HTTP Results led to Full Log Query

  • While we were ultimately unable to identify what was underneath the 150K of encrypted traffic, we were at the least able to identify that our target was using an anonymizing service to mask a portion of his Internet activity
SERI SERIAL: *3/00/513109-09

German Anonymizing Proxy (TS//SI/REL TO USA, FVEY)

(TS//SI//OC/REL TO USA, FVEY) During the 14, 16, and 18 May sessions on the [REDACTED] telephone, the user(s) were using a free German-based anonymizing proxy (http://infoservice.inf.tu-dresden.de), presumably to mask the source of Internet traffic. Use of the anonymizing proxy occurred primarily during the times in which [REDACTED] 's @hotmail.com and [REDACTED]@yahoo.com accounts were accessed.

COMMENT: (TS//SI//OC/REL TO USA, FVEY) The German-based proxy is a Java Anonymous Proxy (JAP), which was developed by the Technical University of Dresden as a free and open source anonymity tool. The proxy functions in a manner similar to The Onion Router (TOR) network. Given his background in computer science and networking, it is not surprising that [REDACTED] would use an anonymizing proxy to secure his Internet activity.

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL


TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL

Why not only use Full Log?

  • If the Full Log query gave us the HTTP traffic in addition to the other non web based traffic, why don't we only use the Full Log query?
    • Because the meta-data options in the full log table are limited
Datetime End Fm IP To IP Fm Por To Po Application Type Application
2009-05-14 13:47:33 119.[REDACTED] 141.[REDACTED] 1495 80 web http/get
2009-05-14 13:48:07 119.[REDACTED] 65.5[REDACTED] 1502 443 network_encryption network_encryption/https
2009-05-14 13:48:07 119.[REDACTED] 141.[REDACTED] 1494 80 web http/get
2009-05-14 13:48:35 119.[REDACTED] 141.[REDACTED] 1134 6544 unknown unknown/tcp
2009-05-14 13:49:02 119.[REDACTED] 141.[REDACTED] 1134 6544 unknown unknown/tcp

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL


TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL

Example #2

  • Starting with MARINA results of a 20 May Internet session of an Iran based target
TS USERID PHONE USER_A ACTIVITY USER_B COOKIE
20090520 092139Z [REDACTED] logged in (email) 213.[REDACTED] fuq8af14q5kjt
20090520 092139Z [REDACTED] used xforip 192.[REDACTED] fuq8af14q5kjt
20090520 092130Z [REDACTED] logged in (email) 213.[REDACTED] fuq8af14q5kjt
20090520 092130Z [REDACTED] used xforip 192.[REDACTED] fuq8af14q5kjt

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL


TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL

Example #2

  • The analyst then did a full log query based off the IP & X-Forwarded-IP pair
Classic A-M
   ASF and WMV Metadata
   Alert
   BlackBerry
   CNE
   Call Logs
   Category DNI
   Cellular DNI
   Cisco Passwords
   DNS
   Document Metadata
   Document Tagging
   Email Addresses
   Extracted Files
   Full Log DNI
   HTTP Activity
   IKE Parser
   IRC Cafe Geolocation
   Logins and Passwords
   Microplugin Metadata

Search: Full Log
Query Name: iran_target
Justification: Iranian IP Address used by cyber target
Datetime: Custom Start: 2009-05-20 09:00 Stop: 2009-05-20 11:00
Client IP (X-Fowarded-For): 192.[REDACTED]
Username:
Attribute Info: [Field Builder]
IP Address: 213.[REDACTED] From
IP Address:

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL

Example #2

  • Full Log table contains the standard DNI meta-data with some but not all information from other plug-ins included (ie. Username from User Activity and Application Info contains some HTTP activity)
Application Info Username Fm Fm City To To City (IP) Datetime ▲ Datetime End Fm IP To IP Fm Por To Po
http://update.nai.com/Products/Cc [REDACTED] IR TEHRAN US NEWYORK 2009-05-20 10:05:16 2009-05-20 10:09:16 213.[REDACTED] 72.[REDACTED] 34847 80
http://platform.ak.facebook.com/v @gmail.com IR TEHRAN DE FRANKFURT 2009-05-20 10:06:54 2009-05-20 10:10:16 213.[REDACTED] 212.[REDACTED] 42806 80
http://platform.ak.facebook.com/v @gmail.col IR TEHRAN DE FRANKFURT 2009-05-20 10:06:54 2009-05-20 10:10:16 213.[REDACTED] 212.[REDACTED] 42806 80
http://platform.ak.facebook.com/v @gmail.com IR TEHRAN DE FRANKFURT 2009-05-20 10:06:54 2009-05-20 10:10:16 213.[REDACTED] 212.[REDACTED] 42806 80
http://newsrss.bbc.co.uk/rss/new [REDACTED] IR TEHRAN GB LONDON 2009-05-20 10:07:43 2009-05-20 10:07:54 213.[REDACTED] 212.[REDACTED] 37459 80
http://b.static.ak.fbcdn.net/inbox/i [REDACTED] IR TEHRAN DE FRANKFURT 2009-05-20 10:08:31 2009-05-20 10:11:33 213.[REDACTED] 212.[REDACTED] 41092 80
http://b.static.ak.fbcdn.net/rsrc.ph [REDACTED] IR TEHRAN DE FRANKFURT 2009-05-20 10:08:31 2009-05-20 10:09:57 213.[REDACTED] 212.[REDACTED] 41092 80
http://platform.ak.facebook.com/v @gmail.com IR TEHRAN DE FRANKFURT 2009-05-20 10:09:39 2009-05-20 10:12:12 213.[REDACTED] 195.[REDACTED] 49648 80
http://photos-d.ak.fbcdn.net/phot [REDACTED] IR TEHRAN NL AMSTERDAM 2009-05-20 10:09:56 2009-05-20 10:12:05 213.[REDACTED] 195.[REDACTED] 41696 80
http://photos-d.ak.fbcdn.net/photo [REDACTED] IR TEHRAN NL AMSTERDAM 2009-05-20 10:09:56 2009-05-20 10:12:05 213.[REDACTED] 195.[REDACTED] 41696 80
http://b.static.ak.fbcdn.net/inbox/i [REDACTED] IR TEHRAN DE FRANKFURT 2009-05-20 10:09:57 2009-05-20 10:10:09 213.[REDACTED] 195.[REDACTED] 34400 80
http://b.static.ak.fbcdn.net/images [REDACTED] IR TEHRAN DE FRANKFURT 2009-05-20 10:09:57 2009-05-20 10:10:09 213.[REDACTED] 195.[REDACTED] 34400 80
http://b.static.ak.fbcdn.net/images [REDACTED] IR TEHRAN DE FRANKFURT 2009-05-20 10:09:57 2009-05-20 10:10:09 213.[REDACTED] 195.[REDACTED] 34400 80
http://b.static.ak.fbcdn.net/images [REDACTED] IR TEHRAN DE FRANKFURT 2009-05-20 10:10:00 2009-05-20 10:12:09 213.[REDACTED] 195.[REDACTED] 44898 80
http://b.static.ak.fbcdn.net/inbox/i [REDACTED] IR TEHRAN DE FRANKFURT 2009-05-20 10:10:00 2009-05-20 10:12:09 213.[REDACTED] 195.[REDACTED] 44898 80
http://newsrss.bbc.co.uk/rss/new [REDACTED] IR TEHRAN GB LONDON 2009-05-20 10:16:12 2009-05-20 10:16:24 213.[REDACTED] 212.[REDACTED] 48804 80

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL


TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL

Example #2

  • The analyst wanted to know if the From IP & X-Forwarded-For IP pair was representing a single computer or if there were multiple users on multiple computers in this data.
  • Full log only provides the bare minimum meta-data to make this determination
ID Datetime Application Info
70 2009-05-20 10:21:45 http://us.mg1.mail.yahoo.com/dc/rs?log=ActivityMaxIdleTime:High&&.gx=1/login_webmail
61 2009-05-20 10:22:17 http://0.channel31.facebook.com/x/3023227576/false/p_1406565350=0/login_webmail
Fm IP To IP Fm Port To Po Application Type Application Data Length Session Length
213.[REDACTED] 209.191.106.109 55828 80 mail mail/webmail/yahoo 1446 1968
213.[REDACTED] 69.63.176.213 55435 80 social social/facebook 3402 3922

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL


TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL

Example #2

  • MARINA provided this information:
TS USERID PHONE USER_A ACTIVITY USER_B COOKIE
20090520 102145Z [REDACTED] previously logged in 213.[REDACTED] fuq8af14q5kjt
20090520 102145Z [REDACTED] previously logged in 213.[REDACTED] fuq8af14q5kjt
20090520 102145Z 213.[REDACTED] used xforip 192.168.36.1 fuq8af14q5kjt
20090520 102217Z [REDACTED] used xforip 192.168.36.1
20090520 102217Z [REDACTED] registered with [REDACTED]@gmail.com
20090520 102217Z [REDACTED] logged in (forum) 213.[REDACTED]
  • The Yahoo and Facebook activity came from the same proxy IP and the same X-Forwarded-For-IP and around the same time but was it from the same computer?

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL


TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL

HTTP Activity Query

  • Let's query that same date time range and IP and XFF IP pair in the HTTP Activity query to see what we get
Classic A-M
   ASF and WMV Metadata
   Alert
   BlackBerry
   CNE
   Call Logs
   Category DNI
   Cellular DNI
   Cisco Passwords
   DNS
   Document Metadata
   Document Tagging
   Email Addresses
   Extracted Files
   Full Log DNI
   HTTP Activity
   IKE Parser
   IRC Cafe Geolocation
   Logins and Passwords
   Microplugin Metadata


Search: HTTP Activity
Query Name: iran_target_http
Justification: Iranian IP address used by NTOC target
Datetime: Custom Start: 2009-05-14 13:30 Stop: 2009-05-14 14:15
X Forwarded For: 192.168.36.1
IP Address: 213.[REDACTED] From
IP Address: To

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL

Now view the HTTP Activity results

  • We saw this meta-data in the Full Log results:
ID Datetime Application Info
70 2009-05-20 10:21:45 http://us.mg1.mail.yahoo.com/dc/rs?log=ActivityMaxIdleTime:High&&.gx=1/login_webmail
61 2009-05-20 10:22:17 http://0.channel31.facebook.com/x/3023227576/false/p_1406565350=0/login_webmail
Fm IP To IP Fm Port To Po Application Type Application Data Length Session Length
213.[REDACTED] 209.191.106.109 55828 80 mail mail/webmail/yahoo 1446 1968
213.[REDACTED] 69.63.176.213 55435 80 social social/facebook 3402 3922
  • And then these three fields are among the unique (and valuable) fields only found in the HTTP activity table:
Cookie Referer Browser
YM[REDACTED]fpwidth=165&suc=3 http://us.mg1.mail.yahoo.com/dc/launc Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.10
datr=1242109330-dc7046de296a31363cbe218f6 http://0.channel31.facebook.com/ifrarr Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.9.0.10

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL


TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL

Now view the HTTP Activity

  • Of interest, note the differences between the two user agents

Browser Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10

  • This indicates different versions of Windows, so unless they did an upgrade within the 1 minute difference of activity, there were at least two different computers behind that Proxy and XFF IP pair

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL


TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL

Moral of the story

  • You should be use both the HTTP activity and Full Log queries to help discover everything your target does when he's online
  • HTTP Activity will give you great meta-data for quick analysis of "web-based" (port 80) activity
  • But not all DNI is done through an Internet Browsers, so it's important to look at the Full Log query results for indications of the use of other applications

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL


TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL

Moral of the story

  • The Multi-Search page gives you the ability to search full log and HTTP activity based on an IP address at the same time

Simply enter in an IP address, choose any or all "roles" (ie. from/to/xff) and then choose what search forms you want.

IP Address: 119.[REDACTED]
IP Role:
        From
        To
        X-Forwarded-For

Search Forms:
User Activity
Phone Number Extractor
Email Addresses
Extracted Files
HTTP Activity
Full Log
Web Proxy
Search
   Classic
      MultiSearch
         IP Addresses
         Mac Addresses
         Username
      Classic A-M
         Alert
         BlackBerry
         CNE
         Call Logs
         Category DNI
         Cellular DNI
         Cisco Passwords
         DNS
         Document Metadata
         Document Tagging
         Email Addresses
         Extracted Files
         Full Log DNI
         HTTP Activity
         IKE Parser
         IRC Cafe Geolocation
         Logins and Passwords
         Microplugin Metadata

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL


TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL

Moral of the story

  • It will submit the multiple searches at the same time, you can either view the results separately or view them as a merged table

My Recent Results

Datetime Submitted Query Name Status Num Results Query Type
2009-06-08 14:33:40 pk_ip_16_may finished 605 full_log
2009-06-08 14:33:40 pk_ip_16_may finished 475 http_parser

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL