1388 lines
No EOL
37 KiB
Markdown
1388 lines
No EOL
37 KiB
Markdown
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
# Introduction to Context Sensitive Scanning with X-KEYSCORE Fingerprints
|
||
|
||
[REDACTED]
|
||
May 2010
|
||
|
||
Derived From: NSA/CSSM [REDACTED]
|
||
Dated 200 [REDACTED]
|
||
Declassify On [REDACTED]
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Opening Question:
|
||
|
||
How do you find your target's activity in DNI traffic?
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Opening Question:
|
||
|
||
What if you don't know your targets E-mail address? Or you're trying to find new ones they may be using?
|
||
|
||
What if the traffic you're interested in doesn't even contain an E-mail address?
|
||
|
||
What do you do then?
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Opening Question:
|
||
|
||
You may try to look for keywords or patterns to help find your target.
|
||
|
||
But how do we scan for keywords in the large volumes of data we see in DNI collection? Won't we get too many false hits?
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Context Sensitive Scanning
|
||
|
||
Context sensitive scanning gives analysts a powerful way to surgically target the traffic you're interested in, by only applying the keywords in the manner in which the analyst intended them to be applied
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Context Sensitive Scanning
|
||
|
||
* For example, think about these scenarios:
|
||
* "I want to look for documents from Iran that mention a banned item"
|
||
* "I want to look for people doing web searches on Jihad from Kabul"
|
||
* "I want to look for people using Mojahedeen Secrets encryption from an IPhone"
|
||
* "I want to look for documents containing this regular expression"
|
||
* "I want to look for E-mails that mention words from various categories of interest to CP"
|
||
* How would you go about targeting those in passive DNI?
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## XKS Fingerprints can help!
|
||
|
||
* Fingerprints are an extremely flexible way to target DNI traffic without the foreknowledge of a strong selector
|
||
* They take advantage of X-KEYSCORE's context sensitive scanning engine that has over 70 unique contexts that can be targeted.
|
||
* An XKS Fingerprint is simply a meta-data tag that gets applied to a session when a certain criteria is met
|
||
* Think of fingerprints as analyst-defined "attributes" of a session
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## "There's an App for that!"
|
||
|
||
* There are currently almost 10,000 AppIDs and Fingerprints in X-KEYSCORE – the full list is available from the NSA XKS Home Page
|
||
* Odds are there may already be a fingerprint for the traffic you're interested in.
|
||
* If not you can easily create your own!
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## For example
|
||
|
||
* I'm an analyst in CT – I want to find anytime Mojahadeen Secrets 2 is seen in DNI Traffic.
|
||
* I'm an analyst in CP – I want to find E-mails or Documents relating to the Iranian Nuclear Procurement network
|
||
* I'm an analyst in NDIST/NTOC – I want to find traffic from a known botnet
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Use Fingerprints!
|
||
|
||
```
|
||
Field Builder
|
||
AppID (+Fingerprints)
|
||
topic/wmd/iran/irisl
|
||
topic/wmd/iran/irisl/edi1/chat_body
|
||
topic/wmd/iran/irisl/edi1/document_body
|
||
topic/wmd/iran/irisl/edi1/email_body
|
||
topic/wmd/iran/irisl/edi1/filename
|
||
topic/wmd/iran/irisl/edi1/url_path
|
||
topic/wmd/iran/irisl/edi2
|
||
topic/wmd/iran/irisl/edi3
|
||
|
||
Field Builder
|
||
AppID (+Fingerprints)
|
||
mojahe
|
||
encryption/mojaheden2
|
||
encryption/mojaheden2/encodedheader
|
||
encryption/mojaheden2/hidden
|
||
encryption/mojaheden2/hidden2
|
||
encryption/mojaheden2/hidden44
|
||
encryption/mojaheden2/secure_file_encoded
|
||
encryption/mojaheden2/securefile
|
||
|
||
Field Builder
|
||
AppID (+Fingerprints)
|
||
botnet/black
|
||
botnet/blackenergybot/command/die
|
||
botnet/blackenergybot/command/flood
|
||
botnet/blackenergybot/command/icmp
|
||
botnet/blackenergybot/command/stop
|
||
botnet/blackenergybot/command/syn
|
||
botnet/blackenergybot/command/wait
|
||
```
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Who is writing fingerprints?
|
||
|
||
* Ext
|
||
* F32/GCHQ
|
||
* F5
|
||
* F6
|
||
* F77
|
||
* FGS
|
||
* FHS
|
||
* FHV
|
||
* FTV
|
||
* I73
|
||
* I75
|
||
* MISCELLANEOUS
|
||
* R1
|
||
* R1, R4
|
||
* R1,IDA_CCS
|
||
* R22
|
||
* R4
|
||
* S
|
||
* S2B
|
||
* S2C
|
||
* S2D
|
||
* S2E
|
||
* S2G
|
||
* S2H
|
||
* S2I
|
||
* S31
|
||
* S32
|
||
* S33
|
||
* SSG
|
||
* ST
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Getting Started
|
||
|
||
* What are the basics of XKS Fingerprints?
|
||
* Simple XKS fingerprints are keyword or regular expression based signatures that are evaluated across the data collected and processed by X-KEYSCORE
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Getting Started
|
||
|
||
* We may have a fingerprint that looks for Begin
|
||
e'
|
||
|
||
```
|
||
لوحة التحكم
|
||
الإعدادات واغيرت
|
||
البريد الالكتروني / كلمه امرو
|
||
ناث المشرف
|
||
تعديل الملي الشخصي
|
||
عديل الخيارات
|
||
الرسائل الغربية
|
||
قائمة الرسائل
|
||
ارسال رسالة جديدة
|
||
تتبع الريال
|
||
alvall
|
||
سابيع بيشتر بيها
|
||
قائمة الاشتراكات
|
||
تعديل المجلدت
|
||
الرقابة
|
||
اليه تبع المعروفة
|
||
المشاركات المحذوفة
|
||
الحوضيع المراقة
|
||
الحشاركت المراقبة
|
||
lcgitaال
|
||
التذكير بالجيب
|
||
قبائر الأسدقاء / التعامل
|
||
العمليات المرفقة
|
||
|
||
تاريخ التسجيل : 2007 Mar
|
||
1,199 احشارکت
|
||
|
||
رسالة خاصة تفضل
|
||
PM 08:04 2008-18-02
|
||
غير متواجد حالياً larvad tawab
|
||
افراصي تفضل .
|
||
|
||
### Begin ASRAR El Mojahedeen v2.0 Encrypted Message ###
|
||
r/RgTzT/ATRhN2E1Zjg1OWQyNWRjMmE2ZTdINzZmZDhI0DUxZWZhMDQ1MjYwMjVIZGUO
|
||
ZGYwMjdkMmJmNTA4ZDY2Yjk0MGU2NGNIYjg6MzNjZTc5MThjY2 Y1ZmY6MTgzZDIkYjhjMTE
|
||
XOGYZYJc1ZDdIMDAxNTQzZmVINDVIY2YYMGJJYJU20DkyYJdmYJFJYJAzMWM5ZDQ20WFIMzg
|
||
4NThhM211Mjc5ODkzZGNh0GRmNWJmNjVIZjQ0MjMxNDM4MDIyOTg1MmRjMGJINGNKYTN
|
||
KYTQ4MzMxZIRIN2FiNjI3MjE1NG|3MTA3ZDQ4NWRmYzMyOTUzZjZIMjg3NjQ10GQ4MTA3N
|
||
TU2N2ZkN2ZjYzUzYzYyMjFIODAwN2VkM2U5MTZINDY2MmM2ZTY IYJQ2Yz100GQ20DUxNW
|
||
VKMjI2MWVINDAyOGI0MThkMTdhNTY1YzIxMDgyOGZIM2wZWZjMDgwM2U4MzNINDg1OD
|
||
UxZTc4ODc1MTY2M210NjU5ZjBhZjVhNjk0OTIhNGEx0ThmYWVINmFIZjlyNmMwZDA3MDM0
|
||
NjJkZDhhMml4ZmRhYjc3NmZINDFkODkyYjBhYjY3MDQ10GVIMjdhYmUwZTlyNGlxYmQyZDIz
|
||
ZjliM2E5ZGQ5NmNhZDQxOTM4NTI0Mjc3MzBIOWEwZWE1Njk3Yj gxY2ViNTQ10WULnoi/VD
|
||
ULIJTEuDJqne0GMRHesi/8PTnZj02yqbmKbFklPjwMhe7FUhFAOw74S+i+PokOREo5XhdP+ y9
|
||
GuBjuYTvrE0xGx20sSfNS5kfRXXH1DaTnb70yufe9r6mMIQ6
|
||
1
|
||
e6E0SRUIdU6YVupz0hhgd4D of
|
||
SBbFR30vg0S+pUxDYgmEOr/RA+fYi47tuHQMh+dynZqQspNdmRUmkjEpFqF03sPHS/10injqo
|
||
1Gsf8+xn52XE2q/WdnU+4XjWnilisVNAjv2nsL+s2TG11HbgocmpQoxy0B0SXPcRv+2JekV37
|
||
k1XyONZk9YH+DV3aWYPXt+ym+wG0XNTqPHIU1JWAZqI2NK/cSXt9DMtCtcb8czRj6G91XvJ9
|
||
Eny7t06xPd9BGio9M+3QuUkZHLEmJiAvgvB6R/X/3whBqk6zMHQLfo+VJcX9umW5mRtgCjzS
|
||
PW6lzzFCGt/B4SK4PxT52ZC0B2kWD8VMyNffrlsTG4XUesgx47Nd5xML8w5pj/fZwKNK+EfKIP
|
||
==Z1 ow29A9N3uLIXBX62Lh0yj/1iqfJ2FNR7AIONSEjwKoggVmkxDiuGaQi+TurpxBgat1g
|
||
### End ASRAR El Mojahedeen v2.0 Encrypted Message ###
|
||
|
||
Displaying 1 items
|
||
Hidden fields
|
||
```
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Boolean Equations
|
||
|
||
* Basic fingerprints can also use Boolean equations:
|
||
|
||
```
|
||
appid('voip/sip/IMS', 6.0, wireshark='sip') =
|
||
('via: sip' or 'v: sip') and 'cseq:' and (
|
||
'p-access-network-info:' or
|
||
'p-called-party-id:' or
|
||
'p-charging-vector:' or
|
||
'p-charging-vector-addresses:' or
|
||
'p-media-authorization:' or
|
||
'security-verify:' or
|
||
'proxy-authorization:' and 'scscf or
|
||
'path:' and 'pcscf or
|
||
'path:' and 'scscf
|
||
);
|
||
```
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Regular Expressions
|
||
|
||
* And Regular Expressions
|
||
|
||
```
|
||
fingerprint('encryption/mojahedeen2')=
|
||
/(?:Begin|End).ASRAR.El.Mojahedeen.v2\..{0,5}Encrypted.Message/ or
|
||
/Mojahedeen.v2\..{0,5}Encrypted.Message/ or
|
||
/(?:Begin|End).Al-Ekhlaas.Network.ASRAR.El.Moujahedeen.V2/ or
|
||
```
|
||
|
||
* Regular expressions must include a fixed "anchor" meeting the minimum keyword length.
|
||
|
||
```
|
||
Bad: /[A-Z]{3}-[0-9]{3,5}/
|
||
OK: /ABC-[0-9]{3,5}/
|
||
```
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Binary Patterns
|
||
|
||
* And Binary Patterns
|
||
|
||
```
|
||
fingerprint('botnet/IO/XXPW0023') =
|
||
$http and
|
||
'\x53\x53\x48\x00\x00\x00\x00\x00'c and
|
||
'\x00\x00\x00\x00\x03\x00\x53\x4D\x52'c;
|
||
|
||
fingerprint('botnet/IO/XXPW0023') =
|
||
$http and
|
||
hex('5353480000000000') and
|
||
hex('00000000300534D52');
|
||
```
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Positional Logic
|
||
|
||
```
|
||
fingerprint('botnet/IO/XXPW0023') =
|
||
pos('\x53\x53\x48\x00\x00\x00\x00\x00') == 4
|
||
and pos('\x00\x00\x00\x00\x03\x00\x53\x4D\x52') == 24;
|
||
|
||
fingerprint('botnet/IO/XXPW0023') =
|
||
$http and
|
||
(pos('\x53\x53\x48\x00\x00\x00\x00\x00') >= 144 and
|
||
pos('\x53\x53\x48\x00\x00\x00\x00\x00') <= 184) and
|
||
(pos('\x00\x00\x00\x00\x03\x00\x53\x4D\x52') >= 164 and
|
||
pos('\x00\x00\x00\x00\x03\x00\x53\x4D\x52') <= 204);
|
||
```
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## When that's not enough...
|
||
|
||
* For example, take the first scenario:
|
||
"I want to look for documents from Iran that mention a banned item"
|
||
* Just using keywords with Boolean equations, how could we restrict the term to only a document body and only coming from Iran?
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Context Sensitive Scanning
|
||
|
||
* X-KEYSCORE's context sensitive scanning engine allows you to explicitly say where you want a term to hit.
|
||
* As an early example, the Tech Strings in Documents capability allowed analysts to restrict terms to only Email, Chat or Documents Bodies
|
||
* The full XKS Context Sensitive Scanning engine allows for over 70 unique contexts to be used as part of an fingerprint
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Context Sensitive Scanning
|
||
|
||
* For example, take the first scenario:
|
||
"I want to look for documents from Iran that mention a banned item"
|
||
* Using the XKS context for Country Code (based on NKB information) and the XKS context for Document Bodies, this easily becomes:
|
||
|
||
```
|
||
fingerprint('demo/scenario1') =
|
||
cc('ir') and doc_body('banned item')
|
||
```
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Context Sensitive Scanning
|
||
|
||
* As another example, let's say we want to tag all Iphone usage
|
||
* Using the XKS context for User Agent this easily becomes
|
||
|
||
```
|
||
fingerprint('demo/scenario2') =
|
||
user_agent('iphone');
|
||
```
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## USSID18/HRA Considerations
|
||
|
||
* XKS Fingerprints may not be USSID18 or HRA compliant if they are queried on by themselves
|
||
* For example, we may want to fingerprint the use of mobile web devices like the IPhone, so that attribute could be used as part of a more complex query.
|
||
* But querying for the IPhone fingerprint itself would be a USSID18 and HRA violation.
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## USSID18/HRA Considerations
|
||
|
||
* But if you want to look for an IPhone user from an Iranian Proxy accessing his Mail.ru account:
|
||
|
||
```
|
||
IP Address: 78.[REDACTED] Either
|
||
|
||
AppID (+Fingerprints) [fulltext]:
|
||
|
||
Field Builder
|
||
AppID (+Fingerprints)
|
||
browser/cellphone/iphone
|
||
Add to Field Close
|
||
|
||
Field Builder
|
||
AppID (+Fingerprints)
|
||
mail/webmail/mailru
|
||
mail/webmail/mailru
|
||
mail/webmail/mailru/attachment
|
||
mail/webmail/mailru/post
|
||
```
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Context Sensitive Scanning
|
||
|
||
What contexts are available for use in XKS Fingerprints?
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## HTTP Activity Contexts (1 of 2)
|
||
|
||
| | |
|
||
|---|---|
|
||
| html_title(expr) | The normalized extracted text web page titles<br/>`html_title('how to' and 'bomb')` |
|
||
| http_host(expr) | The "Host:" name given in the http header.<br/>`http_host('yahoo.com')` |
|
||
| http_url(expr) | Every URL from HTTP GET and POST commands.<br/>`http_url('/mail/inbox?action=delete')` |
|
||
| http_url_args(expr) | All arguments given as part of a URL (ie. all text following the '?' in a URL string)<br/>`http_url('action=delete')` |
|
||
| http_referer(expr) | The "Referer:" URL given in the HTTP header<br/>`http_referer('http://badwebsite/cp?action=show')` |
|
||
| http_language(expr) | The normalized two letter iso-6393 language code as inferred from any http and or html header info<br/>`http_language('fa' or 'de')` |
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## HTTP Activity Contexts (2 of 2)
|
||
|
||
| | |
|
||
|---|---|
|
||
| http_cookie(expr) | The "Cookie:" field given in the http header.<br/>`http_cookie(/PREF=\d\d[a-z]/)` |
|
||
| http_server(expr) | The "Server:" type name in the http header.<br/>`http_server('GWS/2.1' or 'Apache')` |
|
||
| http_user_agent(expr) | The "User-Agent:" field given in the http header.<br/>`http_user_agent(/Mozilla\/[45]/ or 'Chrome')` |
|
||
| web_search(expr) | The normalized extracted text from web searches<br/>`web_search('ricin' or 'plague')` |
|
||
| x_forwarded_for(expr) | The X-Forwarded For IP address from the HTTP Header<br/>`x_forwarded_for('1.2.3.4')` |
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Protocol Contexts 1 of 2
|
||
|
||
| | |
|
||
|---|---|
|
||
| ip(expr) | The source or destination IP address of the session<br/>`ip('127.0.0.1')` |
|
||
| from_ip(expr) | The source IP address of the session<br/>`from_ip('127.0.0.1')` |
|
||
| to_ip(expr) | Every URL from HTTP GET and POST commands.<br/>`to_ip('127.0.0.1')` |
|
||
| ip_subnet(expr) | IP subnet in CIDR notation.<br/>`ip_subnet('7.211.143.148/24')` |
|
||
| port(expr) | The source or destination TCP or UDP port number.<br/>`port('22')` |
|
||
| from_port(expr) | The source TCP or UDP port number.<br/>`from_port('22')` |
|
||
| to_port(expr) | The destination TCP or UDP port number.<br/>`to_port('22')` |
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Protocol Contexts 1 of 2
|
||
|
||
| | |
|
||
|---|---|
|
||
| cc(expr) | The country (either to OR from) based on IP address<br/>`cc('ir' or 'pk')` |
|
||
| from_cc(expr) | The source country based on IP address<br/>`from_cc('ir' or 'pk')` |
|
||
| to_cc(expr) | The destination country based on IP address<br/>`to_cc('ir' or 'pk')` |
|
||
| protocol(expr) | The textual form of the IP next protocol.<br/>`protocol('TCP')` |
|
||
| next_protocol(expr) | The textual form of the IP next protocol.<br/>`ip_next_protocol('17')` |
|
||
| mac_address(expr) | The MAC address of the target network device.<br/>`mac_address('00:16:3E:3F:BD:EF')` |
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Communication Based Contexts
|
||
|
||
| | |
|
||
|---|---|
|
||
| email_body(expr) | The UTF-8 normalized text of all email bodies.<br/>`email_body('how to' and 'build' and ('bomb' or 'weapon'))` |
|
||
| chat_body(expr) | The UTF-8 normalized text of all chat bodies.<br/>`chat_body('how to' and 'build' and ('bomb' or 'weapon'))` |
|
||
| document_body(expr) | The UTF-8 normalized text of the Office document. – <br/>Office documents include (but are not limited to) Microsoft Office, Open Office, Google Docs and Spreadsheets.<br/>`document_body('how to' and 'build' and ('bomb' or 'weapon'))` |
|
||
| calendar_body(expr) | The UTF-8 normalized text of all calendars. An example is Google Calendar.<br/>`calendar_body('wedding')` |
|
||
| archive_files(expr) | Matches a list of files from within an archive. For example is a ZIP file is transmitted, all names of files within are passed to this context.<br/>`archive_files('bad.dll' or 'virus.doc')` |
|
||
| http_post_body(expr) | The UTF-8 normalized text HTTP url-encoded POSTS.<br/>`http_post_body('action=send' and 'badguy@yahoo')` |
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Communication Based Contexts
|
||
## Aliases
|
||
|
||
| | |
|
||
|---|---|
|
||
| doc_email_body(expr) | This covers the email_body and document_body contexts<br/>`doc_email_body('how to' and 'build' and ('bomb' or 'weapon'))` |
|
||
| communication_body(expr) | This covers the email_body, document_body and chat_body contexts<br/>`chat_body('how to' and 'build' and ('bomb' or 'weapon'))` |
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Context sensitivity
|
||
|
||
Why use context-sensitive scanning?
|
||
|
||
* More intuitive - you can say what you mean
|
||
* More accurate - if 'maps.google.com' is mentioned in a blog post, you don't want to try processing it as a Google Maps session
|
||
* Better performance for XKEYSCORE
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Examples
|
||
|
||
* "I want to look for people doing web searches on Jihad from Kabul"
|
||
* Using the from_city() and web_search() context this becomes
|
||
|
||
```
|
||
fingerprint('demo/scenario3') =
|
||
from_city('kabul') and web_search('jihad');
|
||
```
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Examples
|
||
|
||
* "I want to look for people using Mojahedeen Secrets encryption from an IPhone"
|
||
* You can even use existing fingerprints in a fingerprint definition! So this becomes:
|
||
|
||
```
|
||
fingerprint('demo/scenario4') =
|
||
fingerprint('encryption/mojahdeen2' and
|
||
fingerprint('browser/cellphone/iphone')
|
||
```
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Examples
|
||
|
||
* "I want to look for documents containing this regular expression"
|
||
* Using doc_body this becomes:
|
||
|
||
```
|
||
fingerprint('demo/scenario5') =
|
||
doc_body(/blah[a-z]{3-5}something/)
|
||
```
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Example 4
|
||
|
||
* "I want to look for E-mails that mention words from various categories of interest to CP"
|
||
* You can use multiple variables in an equation like this:
|
||
|
||
```
|
||
topic('wmd/acw/govtorgs') =
|
||
email_body($acwitems and $acwpositions and
|
||
($acwcountries or $acwbrokers or $acwports));
|
||
```
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Example 4
|
||
|
||
* $acwitems = 'machine gun' or 'grenade' or 'AK 47'
|
||
* $acwpositions = 'minister of defence' or 'defense minister'
|
||
* $acwcountries = 'somalia' or 'liberia' or 'sudan'
|
||
* $acwbrokers = 'south africa' or 'serbia' or 'bulgaria'
|
||
* $acwports = 'rangood' or 'albasra' or 'dar es salam'
|
||
|
||
```
|
||
topic('wmd/acw/govtorgs') =
|
||
email_body($acwitems and $acwpositions and
|
||
($acwcountries or $acwbrokers or $acwports));
|
||
```
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Advanced Code-Based Fingerprints
|
||
|
||
* What happens when there are no keywords or regular expressions that will help identify the traffic of interest to you?
|
||
* As enough example, many of the CT Targets are now smart enough to not leave the Mojahedeen Secrets header in the E-mails they send. How can we detect that the E-mail (which looks like junk) is in fact Mojahedeen Secrets encrypted text
|
||
* A C++ code fingerprint can help evaluate that data
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Code Based Fingerprint
|
||
|
||
```
|
||
لوحة التحكم
|
||
الإعدادات والشيرت
|
||
البريد الالكتروني / كلمه امرو
|
||
ناث المشرف
|
||
تعديل الملي الشخصي
|
||
عديل الخيارات
|
||
الرسائل الغنية
|
||
وائمة الرسائل
|
||
ارسال رسالة جديدة
|
||
تتبع الريال
|
||
alvall
|
||
سابيع بيشتر بيها
|
||
قائمة الاشتراكات
|
||
تعديل المجلدت
|
||
الرقابة
|
||
اليه تبع المعروفة
|
||
المشاركات المحذوفة
|
||
الحوضيع المراقة
|
||
الحشاركت المراقبة
|
||
alcgitaال
|
||
التذكير بالجيب
|
||
قبائر الأسدقاء / التعامل
|
||
العمليات المرفقة
|
||
|
||
تاريخ التسجيل : 2007 Mar
|
||
1,199 احشارکت
|
||
|
||
رسالة خاصة نفضا
|
||
PM 08:04 2008-18-02
|
||
غير متواجد حالياً larvad tawab
|
||
افراصي في تفضل .
|
||
|
||
|
||
r/RgTzT/ATRhN2E1Zjg1OWQyNWRjMmE2ZTdINzZmZDhI0DUxZWZhMDQ1MjYwMjVIZGUO
|
||
ZGYwMjdkMmJmNTA4ZDY2Yjk0MGU2NGNIYjg6MzNjZTc6MThjY2 Y1ZmY6MTgzZDIkYjhjMTE
|
||
XOGYZYJc1ZDdIMDAxNTQzZmVINDVIY2YYMGJJYJU20DkyYJdmYJFJYJAzMWM5ZDQ20WFIMzg
|
||
4NThhM211Mjc5ODkzZGNh0GRmNWJmNjVIZjQ0MjMxNDM4MDIyOTg1MmRjMGJINGNKYTN
|
||
KYTQ4MzMxZIRIN2FiNjI3MjE1NG|3MTA3ZDQ4NWRmYzMyOTUzZjZIMjg3NjQ10GQ4MTA3N
|
||
TU2N2ZkN2ZjYzUzYzYyMjFIODAwN2VkM2U5MTZINDY2MmM2ZTY IYJQ2Yz100GQ20DUxNW
|
||
VKMjI2MWVINDAyOGI0MThkMTdhNTY1YzIxMDgyOGZIM2wZWZjMDgwM2U4MzNINDg1OD
|
||
UxZTc4ODc1MTY2M210NjU5ZjBhZjVhNjk0OTIhNGEx0ThmYWVINmFIZjlyNmMwZDA3MDM0
|
||
NjJkZDhhMml4ZmRhYjc3NmZINDFkODkyYjBhYjY3MDQ10GVIMjdhYmUwZTlyNGlxYmQyZDIz
|
||
ZjliM2E5ZGQ5NmNhZDQxOTM4NTI0Mjc3MzBIOWEwZWE1Njk3Yj gxY2ViNTQ10WULnoi/VD
|
||
ULIJTEuDJqne0GMRHesi/8PTnZj02yqbmKbFklPjwMhe7FUhFAOw74S+i+PokOREo5XhdP+ y9
|
||
GuBjuYTvrE0xGx20sSfNS5kfRXXH1DaTnb70yufe9r6mMIQ6
|
||
1
|
||
e6E0SRUIdU6YVupz0hhgd4D of
|
||
SBbFR30vg0S+pUxDYgmEOr/RA+fYi47tuHQMh+dynZqQspNdmRUmkjEpFqF03sPHS/10injqo
|
||
1Gsf8+xn52XE2q/WdnU+4XjWnifisVNAjv2nsL+s2TG11HbgocmpQoxy0B0SXPcRv+2JekV37
|
||
k1XyONZk9YH+DV3aWYPXt+ym+wG0XNTqPHIU1JWAZqI2NK/cSXt9DMtCtcb8czRj6G91XvJ9
|
||
Eny7t06xPd9BGio9M+3QuUkZHLEmJiAvgvB6R/X/3whBqk6zMHQLfo+VJcX9umW5mRtgCjzS
|
||
PW6lzzFCGt/B4SK4PxT52ZC0B2kWD8VMyNffrlsTG4XUesgx47Nd5xML8w5pj/fZwKNK+EfKIP
|
||
==Z1 ow29A9N3uLIXBX62Lh0yj/1iqfJ2FNR7AIONSEjwKoggVmkxDiuGaQi+TurpxBgat1g
|
||
|
||
Displaying 1 items
|
||
Hidden fields
|
||
```
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Advanced Code-Based Fingerprints
|
||
|
||
```
|
||
fingerprint('encryption/mojaheden2/hidden44') =
|
||
$mol_cipher_first_test
|
||
: c-+ extractors: {{
|
||
msg1 = /((a-zA-Z0-9\+\/](6) [DHLPTXofjnrvs37\/] [4-9\+\/]) AT ([ABEFIJMNQFUVYZcdghkl] [hijklmwxyz012345] [ΜΝΟΥ]] [[GTWjnz2](12))/c
|
||
msg2 /((a-zA-Z0-9\+\/](6) [DIILPTXofjnrvz37\/] [4-3\+\/]) AU ([ADEFIJMNQFUVYZcdchkl] [hijklmwxyz012045] [ΜΝΟΥZ] [DGTWjwz2](12))/c;
|
||
}}
|
||
main : {{
|
||
std::string msg;
|
||
if (msc1
|
||
msg = msg1[0];
|
||
else if (msg2)
|
||
msg = msg2[0];
|
||
else
|
||
return false;
|
||
|
||
char buf[16]:
|
||
char chunk1 [16];
|
||
char chunk2 [16];
|
||
char chunk3 [16];
|
||
if (crue):
|
||
snprintf(chunk1, 16, "%02x%02x%02x%02x",
|
||
msg[10] & 0xff,
|
||
msg[11] & 0xff,
|
||
msg[12] & 0xff,
|
||
msg[13] & 0xff);
|
||
snprintf(chunk2, 16, "%02x02x02xC2x",
|
||
msg[14] & 0xff,
|
||
msg[15] & 0xff,
|
||
msg[16] & 0xff,
|
||
msg[17] & 0xff);
|
||
snprintf(chunk3, 16, "%02x%02x%02x%02x",
|
||
msg[18] & 0xff,
|
||
msg[19] & 0xff,
|
||
mas[20] & 0xff,
|
||
msg[21] & 0xff);
|
||
if(!((strcmp(chunk, chunk2) == 0) ||
|
||
(strcmp(chunk2, chunk3; == 0) ||
|
||
(strcmp(chunk1, chunk3; == 0)){
|
||
std::string msc_decoded = xks::base64dec imsg);
|
||
for(size_t i = 7; i < msg_decoded.size(); i++){
|
||
if (msg_decoded[i] < '0' || (msg_decoded[i] > '9' && msg_decoded[i] < 'a') || msg_decoded[i] > 'f'){
|
||
return false;
|
||
}
|
||
}
|
||
snprintf(buf, 16, "%02x02x02x%02x",
|
||
msg_decoded[1] & 0xff,
|
||
msg_decoded[2] & 0xff,
|
||
msg_decoded[3] & 0xff,
|
||
msg_decoded[4] & 0xff);
|
||
std::string keyid hex = buf;
|
||
```
|
||
|
||
```
|
||
Field Builder
|
||
AppID (+Fingerprints)
|
||
encryption/mojaheden2/hidden44
|
||
Add to Field Close
|
||
```
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Advanced Code-Based Fingerprints
|
||
|
||
* As another example, some of the activity from the Conficker botnet simply can't be detected with keywords or regular expressions
|
||
* In cases like this, C++ code can be used inside a fingerprint to test the data further
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Advanced Code-Based Fingerprints
|
||
|
||
```
|
||
fingerprint('botnet/conficker_p2p_udp_data', 7.5) =
|
||
udp and not 'to ap 00' and not 'fm ap 00
|
||
: c++;;
|
||
// Classification: CONFIDENTIAL//REL TO USA, FVEY
|
||
// NOT releasable to third-parties
|
||
uint8_t key8;
|
||
uint8_t key9;
|
||
uint8_t pkt type;
|
||
uint0_t decrypted_bytes[4];
|
||
uint32_t running_hash - 0;
|
||
uint32_t K_h1ck;
|
||
uint32_t K_lct;
|
||
uint32_t store_hashes[4] = {11,11,1,11};
|
||
uint32_t min_pkt_len;
|
||
uint32_t max_pkt_len;
|
||
uint32_t t;
|
||
packet_t pkt;
|
||
|
||
while (pkt - get packet (11
|
||
{
|
||
if (pkt.size < 10)
|
||
return false;
|
||
|
||
key8 = (uint8_t)pkt.data[7]<<1 | pkt.data[7]>>7)&1));
|
||
key9 = (uint8_t)key8<<2 | ((pkt.data[7]>>5|&1));
|
||
|
||
if ((key9 ^ pkt.data[0] &0xfc) != 0x80;
|
||
return false; // Not Conficker, so abort
|
||
if ((key0 ^ pkt.data[0] & 0x02)
|
||
return false; // bi not set for UDP packets
|
||
|
||
if (pkt.size<23)
|
||
continue;
|
||
|
||
if ((key9 ^ pkt.data[9] !=0x80)
|
||
continue;
|
||
|
||
pkt type - (key8 ^ pkt.data[8])>>3;
|
||
|
||
if (pkt_type & 0x10) // summary
|
||
continue;
|
||
|
||
if (!(pkt_type & 0x08:: // not a data packet
|
||
continue;
|
||
|
||
min_pkt_len = 23;
|
||
max_pkt_len = (uint32_t pkt.size;
|
||
|
||
K high = uint32_t:pkt.data[7])<<24u | uint32_t(pkt.data[6])<<15 | uint32_t pkt.data[5])<<Cu | uint32 (pkt.data[4]);
|
||
K_low - uint32_t:pkt.data[3])<<24u | uint32_t(pkt.data[2])<<15u | uint32_t pkt.data[1])<<8u | uint32_ (pkt.data[0]);
|
||
|
||
running_hash = 0;
|
||
|
||
for(i=1; t<max_pkt_len; i++)
|
||
{
|
||
if(t>=8) // decrypt data
|
||
{
|
||
```
|
||
|
||
```
|
||
Field Builder
|
||
|
||
AppID (+Fingerprints)
|
||
botnet/conficker_p2p_udp_data
|
||
Add to Field Close
|
||
```
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Meta-data Extracting Fingerprints
|
||
|
||
* What happens when you find data and want some pieces of meta-data extracted?
|
||
* XKS Fingerprints can be used to extract meta-data to select XKS database tables.
|
||
* Or if no existing database is applicable, you can define your own database schema for the meta-data
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Free File Upload Sites
|
||
|
||
* As a real life example, think of all of various Free File Upload (FFU) sites of interest
|
||
* When a user uploads a document they get a response page that looks like this:
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Free File Upload Sites
|
||
|
||
```
|
||
Welcome to zSHARE
|
||
|
||
With zSHARE you can upload files, images, videos, audio and flash for free. Simply use the upload form below and start sharing! You
|
||
can also use zSHARE as your personal file storage: backup your data and protect your files. First Time? Read our FAQ!
|
||
|
||
* Upload now
|
||
* Login
|
||
* Create Free Account
|
||
* Premium
|
||
* FAQ
|
||
|
||
File Uploaded
|
||
|
||
The file khi pics.zip was successfully uploaded! (4.04MB). You're now ready to share it with unlimited people or keep it as a backup.
|
||
|
||
Download Link
|
||
|
||
http://www.zshare.net/download/637199570b174c9f/
|
||
|
||
Link for forums: [URL=http://www.zshare.net/download/637199570b174
|
||
Direct Link: http://www.zshare.net/download/637199570b174c9f/
|
||
Delete Link: http://www.zshare.net/delete.html?63719957-7c8893b1k
|
||
|
||
E-mail Me This Info
|
||
|
||
To receive all the info on the file you uploaded, such as removal instructions and download link, enter your e-mail address on the
|
||
field below:
|
||
Your e-mail:
|
||
```
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Free File Upload Sites
|
||
|
||
* Look at all the great information on that page:
|
||
|
||
```
|
||
File Uploaded
|
||
|
||
The file khi pics.zip was successfully uploaded! (4.04MB). You're now ready to share it with unlimited people or keep it as a backup.
|
||
|
||
Download Link
|
||
|
||
http://www.zshare.net/download/637199570b174c9f
|
||
|
||
Link for forums: [URL=http://www.zshare.net/download/637199570b174
|
||
Direct Link: http://www.zshare.net/download/637199570b174c9f/
|
||
Delete Link: http://www.zshare.net/delete.html?63719957-7c8893b1b
|
||
```
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Free File Upload Sites
|
||
|
||
* How can we quickly get that information extracted as Meta-data and be agile enough to respond to each FFU site which may have its own format
|
||
* XKS "v4" Fingerprints allow you to use the XKS Fingerprint Language to extract meta-data into the XKS database
|
||
* Fingerprints are deployed within an hour of being accepted meaning you no longer need to wait for all 130+ XKS sites to be upgraded to have the latest and greatest capabilities.
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Free File Upload Sites
|
||
|
||
```
|
||
appid('filetransfer/web/zshare_net/upload/response', 5.0) =
|
||
http_title('zSHARE') and 'zshare.net/delete.html'
|
||
: c++
|
||
extractors: {{
|
||
wft_file_name = /The sfile\s<strong><font\scolor=\"#333333">([^<](1,300})\s</;
|
||
wft_delete_url = /zshare.net\/delete.html\?([0-9]+)-([0-9a-zA-Z]{32})\"/"
|
||
wft_upload_id = /<font color=\"#666666\"><a href=\"http:\/\/www\.zshare\.net\/[^\/]+\/([0-9]+)[0-9a-f]{8}/;
|
||
wft_url = /<font color=\"#666666\"><a href=\"(http:\/\/www\.zshare\.net\/[^\/]+\/[^\/]+)/;
|
||
wft_uploader_username = /<small>Logged in as: ([^<]+)<\/small>/;
|
||
);
|
||
main = {{
|
||
if (wft_delete_url ) {
|
||
DB["web_file_transfer"]["wft_upload_id"] = wft_upload_id[0];
|
||
DB["web_file_transfer"]["wft_delete"] = wft_delete_url[0]+"-"+wft_delete_url[1];
|
||
|
||
DB["web_file_transfer"]["wft_site_name"] = "zshare.net";
|
||
DB["web_file_transfer"]["transfer_type"] = "upload";
|
||
|
||
if (wft_file_name) {
|
||
DB["web_file_transfer"]["wft_filename"] = wft_file_name[0];
|
||
}
|
||
|
||
if (wft_url) {
|
||
DB["web_file_transfer"]["wft_url"] = wft_url[0]
|
||
}
|
||
if (wft_uploader_username) {
|
||
DB["web_file_transfer"]["uploader_username"] = wft_uploader_username[0];
|
||
}
|
||
DB.apply();
|
||
} else {
|
||
logger.debug("filetransfer/web/zshare.net/upload/response: Host regexs didn't match");
|
||
}
|
||
return true;
|
||
}};
|
||
```
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Meta-data Extracting Fingerprints
|
||
|
||
* All you do is tell XKS when to start extracting meta-data
|
||
|
||
```
|
||
appid('filetransfer/web/zshare_net/upload/response', 5.0)=
|
||
http_title('zSHARE') and 'zshare.net/delete.html'
|
||
: c++
|
||
```
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
# Meta-data Extracting Fingerprints
|
||
|
||
* Use Regular Expressions to tell it what to extract:
|
||
|
||
```
|
||
extractors: {{
|
||
wft_file_name = /The\sfile\s<strong><font\scolor=\"#333333">([^<]{1,300})\s</;
|
||
wft_delete_url = /zshare.net\/delete.html\?([0-9]+)-([0-9a-zA-Z]{32})\"/;
|
||
wft_upload_id = /<font color=\"#666666\"><a href=\"http:\/\/www\.zshare\.net\/[^\/]+\/([0-9]+) [0-9a-f]{8}/;
|
||
wft_url = /<font color=\"#666666\"><a href=\"(http:\/\/www.zshare\.net\/[^\/]+\/[^\/]+)/;
|
||
wft_uploader_username = /<small>Logged in as: ([^<]+)<\/small>/;
|
||
}}
|
||
```
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Meta-data Extracting Fingerprints
|
||
|
||
* Finally tell it which database tables you want to store the information:
|
||
|
||
```
|
||
main = {{
|
||
if (wft_delete_url) {
|
||
DB["web_file_transfer"]["wft_upload_id"] = wft_upload_id[0];
|
||
DB["web_file_transfer"]["wft_delete"] = wft_delete_url[0]+"-"+wft_delete_url[1];
|
||
|
||
DB["web_file_transfer"]["wft_site_name"] = "zshare.net";
|
||
DB["web_file_transfer"]["transfer_type"] = "upload";
|
||
```
|
||
|
||
| File URL | Filename |
|
||
|:---|:---|
|
||
| http://www.zshare.net/download/637199570b174c9f | khi pics.zip |
|
||
|
||
| Transfer Type | Upload ID | Delete ID | Site Name |
|
||
|:---|:---|:---|:---|
|
||
| upload | 63719957 | 7c8893b1bf04170771dca3e7f0756a26 | zshare.net |
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Meta-data Extracting Fingerprints
|
||
|
||
* What if the meta-data you want to extract doesn't fit nicely into any of the existing XKS meta-data tables?
|
||
|
||
```
|
||
Classic A-M
|
||
* ASF and WMV Metadata
|
||
* Alert
|
||
* BlackBerry
|
||
* CNE
|
||
* Call Logs
|
||
* Category DNI
|
||
* Cellular DNI
|
||
* Cisco Passwords
|
||
* Document Metadata
|
||
* Document Tagging
|
||
* Email Addresses
|
||
* Extracted Files
|
||
* Full Log DNI
|
||
* HTTP Activity
|
||
* IRC Cafe Geolocation
|
||
* Logins and Passwords
|
||
|
||
Classic N-Z
|
||
* Network Logs
|
||
* PDF Metadata
|
||
* PILBEAM
|
||
* Phone Number Extractor
|
||
* RBGAN
|
||
* REGISTRY
|
||
* RTP
|
||
* Radius Logs
|
||
* RealMedia Metadata
|
||
* SIP
|
||
* TOR Log
|
||
* Tech Strings in Documents
|
||
* User Activity
|
||
* WLAN
|
||
* Web Proxy
|
||
* Wireshark
|
||
```
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Meta-data Extracting Fingerprints
|
||
|
||
* Define your own with the "Microplugin" query forms:
|
||
|
||
```
|
||
Microplugins
|
||
* Beb Flood
|
||
* Cone Blindmatesmen Beacon
|
||
* Cone Byzantine Raptor Trojan3
|
||
* Cone Traffic
|
||
* Cone Victim Id
|
||
* Encryption Steg JSTEG
|
||
* Exif Metadata
|
||
* Ipv6 Addresses
|
||
* Mailer Accounts
|
||
* Ms2 Extract Keyids
|
||
* Munged Traffic
|
||
* NetStrings
|
||
* QUANTUMBOT Table
|
||
* Saudi Mfa Visa
|
||
* Udp Xxpj9010 Lzo Networkname
|
||
* VPN External IP Addresses
|
||
* Vpn Users
|
||
* Web Geo Cell Towers
|
||
* Web Geo Results
|
||
* Web Geo Wifi Towers
|
||
* _sub Dictionary Code Snippet
|
||
```
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Meta-data Extracting Fingerprints
|
||
|
||
* Example MS2 KeyIDs
|
||
|
||
```
|
||
Search: Ms2 Extract Keyids
|
||
Query Name: [REDACTED]
|
||
Justification: Recent Justifications
|
||
Additional Justification:
|
||
Miranda Number:
|
||
|
||
Datetime: 1 Day Start: 2010-05-03 00:00 Stop: 2010-05-04 23:59
|
||
|
||
ejKeyID:
|
||
Username<realm>:
|
||
|
||
IP Address: From [IP Address Field Builder]
|
||
IP Address: To [IP Address Field Builder]
|
||
Port: From
|
||
Port: To
|
||
```
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Meta-data Extracting Fingerprints
|
||
|
||
```
|
||
Search: Ccne Byzantine Raptor Trojan3
|
||
Query Name: [REDACTED]
|
||
Justification: Recent Justifications
|
||
Additional Justification:
|
||
Miranda Number:
|
||
|
||
Datetime: 1 Day Start: 2010-05-03 00:00 Stop: 2010-05-04 23:59
|
||
|
||
brt_decrypt:
|
||
brt_hostname:
|
||
brt_ipaddress:
|
||
brt_length:
|
||
brt_osversion:
|
||
DB["
|
||
brt_packet_type:
|
||
DB[
|
||
brt_sequence_num:
|
||
brt_username:
|
||
|
||
Username <realm>:
|
||
IP Address: From [IP Address Field Builder]
|
||
IP Address: To [IP Address Field Builder]
|
||
```
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## New Fingerprint GUI
|
||
|
||
* New XKS Fingerprint GUI allows analysts to directly test, submit and manage fingerprints through the web
|
||
|
||
```
|
||
Navigation Menu
|
||
* Fingerprints
|
||
* Validate / Submit
|
||
* Approved
|
||
* Pending
|
||
* My Signatures
|
||
|
||
Fingerprint Validation / Submittal
|
||
Step #1 Compile
|
||
Step #2 Test Against Session Data
|
||
Step #3 Save
|
||
Help
|
||
|
||
Global Variable Declarations
|
||
Type or paste any global VARIABLE DECLARATIONS here.
|
||
|
||
Signature
|
||
Type or paste a FINGERPRINT definition here.
|
||
|
||
Press Compile when done editing
|
||
```
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## New Fingerprint GUI
|
||
|
||
* New XKS Fingerprint GUI allows analysts to directly test, submit and manage fingerprints through the web
|
||
|
||
```
|
||
Fingerprint Validation / Submittal
|
||
Step #1 Compile
|
||
Step #2 Test Against Session Data
|
||
Step #3 Save
|
||
|
||
Global Variable Declarations
|
||
Stest = 'bomb' or 'missle' or 'ied';
|
||
|
||
Signature
|
||
fingerprin= ('test/testl') = email_body($test);
|
||
|
||
Results
|
||
SUCCESS!
|
||
Congratulations, your fingerprint was successfully compiled!
|
||
Now use the Test button to run it against the cesignated session data.
|
||
```
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
--
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Questions?
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Syntax Rules
|
||
|
||
* The definition of the fingerprint will look like this:
|
||
|
||
`fingerprint('test/blah/something', owner = [REDACTED]) =`
|
||
|
||
Note the single quotes needed for the fingerprint name and owner
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Syntax Rules
|
||
|
||
* Secondly every fingerprint definition must be completed by a semi-colon.
|
||
|
||
`fingerprint('test/blah/something', owner = [REDACTED]) = 'badguy' ;`
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Syntax Rules
|
||
|
||
* Variables also must be completed by a semi-colon.
|
||
|
||
```
|
||
$badguy =
|
||
'bomb' or 'gun' or 'weapon' ;
|
||
fingerprint('test/blah/something', owner = [REDACTED]) =
|
||
$badguy;
|
||
```
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
---
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
|
||
|
||
## Syntax Rules
|
||
|
||
* Definitions and Variables can span multiple lines
|
||
|
||
```
|
||
$badguy =
|
||
'bomb' or
|
||
'gun' or
|
||
'weapon' ;
|
||
fingerprint('test/blah/something', owner = '[REDACTED]') =
|
||
$badguy;
|
||
```
|
||
|
||
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL |