TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL # Introduction to Context Sensitive Scanning with X-KEYSCORE Fingerprints [REDACTED] May 2010 Derived From: NSA/CSSM [REDACTED] Dated 200 [REDACTED] Declassify On [REDACTED] TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Opening Question: How do you find your target's activity in DNI traffic? TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Opening Question: What if you don't know your targets E-mail address? Or you're trying to find new ones they may be using? What if the traffic you're interested in doesn't even contain an E-mail address? What do you do then? TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Opening Question: You may try to look for keywords or patterns to help find your target. But how do we scan for keywords in the large volumes of data we see in DNI collection? Won't we get too many false hits? TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Context Sensitive Scanning Context sensitive scanning gives analysts a powerful way to surgically target the traffic you're interested in, by only applying the keywords in the manner in which the analyst intended them to be applied TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Context Sensitive Scanning * For example, think about these scenarios: * "I want to look for documents from Iran that mention a banned item" * "I want to look for people doing web searches on Jihad from Kabul" * "I want to look for people using Mojahedeen Secrets encryption from an IPhone" * "I want to look for documents containing this regular expression" * "I want to look for E-mails that mention words from various categories of interest to CP" * How would you go about targeting those in passive DNI? TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## XKS Fingerprints can help! * Fingerprints are an extremely flexible way to target DNI traffic without the foreknowledge of a strong selector * They take advantage of X-KEYSCORE's context sensitive scanning engine that has over 70 unique contexts that can be targeted. * An XKS Fingerprint is simply a meta-data tag that gets applied to a session when a certain criteria is met * Think of fingerprints as analyst-defined "attributes" of a session TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## "There's an App for that!" * There are currently almost 10,000 AppIDs and Fingerprints in X-KEYSCORE – the full list is available from the NSA XKS Home Page * Odds are there may already be a fingerprint for the traffic you're interested in. * If not you can easily create your own! TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## For example * I'm an analyst in CT – I want to find anytime Mojahadeen Secrets 2 is seen in DNI Traffic. * I'm an analyst in CP – I want to find E-mails or Documents relating to the Iranian Nuclear Procurement network * I'm an analyst in NDIST/NTOC – I want to find traffic from a known botnet TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Use Fingerprints! ``` Field Builder AppID (+Fingerprints) topic/wmd/iran/irisl topic/wmd/iran/irisl/edi1/chat_body topic/wmd/iran/irisl/edi1/document_body topic/wmd/iran/irisl/edi1/email_body topic/wmd/iran/irisl/edi1/filename topic/wmd/iran/irisl/edi1/url_path topic/wmd/iran/irisl/edi2 topic/wmd/iran/irisl/edi3 Field Builder AppID (+Fingerprints) mojahe encryption/mojaheden2 encryption/mojaheden2/encodedheader encryption/mojaheden2/hidden encryption/mojaheden2/hidden2 encryption/mojaheden2/hidden44 encryption/mojaheden2/secure_file_encoded encryption/mojaheden2/securefile Field Builder AppID (+Fingerprints) botnet/black botnet/blackenergybot/command/die botnet/blackenergybot/command/flood botnet/blackenergybot/command/icmp botnet/blackenergybot/command/stop botnet/blackenergybot/command/syn botnet/blackenergybot/command/wait ``` TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Who is writing fingerprints? * Ext * F32/GCHQ * F5 * F6 * F77 * FGS * FHS * FHV * FTV * I73 * I75 * MISCELLANEOUS * R1 * R1, R4 * R1,IDA_CCS * R22 * R4 * S * S2B * S2C * S2D * S2E * S2G * S2H * S2I * S31 * S32 * S33 * SSG * ST TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Getting Started * What are the basics of XKS Fingerprints? * Simple XKS fingerprints are keyword or regular expression based signatures that are evaluated across the data collected and processed by X-KEYSCORE TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Getting Started * We may have a fingerprint that looks for Begin e' ``` لوحة التحكم الإعدادات واغيرت البريد الالكتروني / كلمه امرو ناث المشرف تعديل الملي الشخصي عديل الخيارات الرسائل الغربية قائمة الرسائل ارسال رسالة جديدة تتبع الريال alvall سابيع بيشتر بيها قائمة الاشتراكات تعديل المجلدت الرقابة اليه تبع المعروفة المشاركات المحذوفة الحوضيع المراقة الحشاركت المراقبة lcgitaال التذكير بالجيب قبائر الأسدقاء / التعامل العمليات المرفقة تاريخ التسجيل : 2007 Mar 1,199 احشارکت رسالة خاصة تفضل PM 08:04 2008-18-02 غير متواجد حالياً larvad tawab افراصي تفضل . ### Begin ASRAR El Mojahedeen v2.0 Encrypted Message ### r/RgTzT/ATRhN2E1Zjg1OWQyNWRjMmE2ZTdINzZmZDhI0DUxZWZhMDQ1MjYwMjVIZGUO ZGYwMjdkMmJmNTA4ZDY2Yjk0MGU2NGNIYjg6MzNjZTc5MThjY2 Y1ZmY6MTgzZDIkYjhjMTE XOGYZYJc1ZDdIMDAxNTQzZmVINDVIY2YYMGJJYJU20DkyYJdmYJFJYJAzMWM5ZDQ20WFIMzg 4NThhM211Mjc5ODkzZGNh0GRmNWJmNjVIZjQ0MjMxNDM4MDIyOTg1MmRjMGJINGNKYTN KYTQ4MzMxZIRIN2FiNjI3MjE1NG|3MTA3ZDQ4NWRmYzMyOTUzZjZIMjg3NjQ10GQ4MTA3N TU2N2ZkN2ZjYzUzYzYyMjFIODAwN2VkM2U5MTZINDY2MmM2ZTY IYJQ2Yz100GQ20DUxNW VKMjI2MWVINDAyOGI0MThkMTdhNTY1YzIxMDgyOGZIM2wZWZjMDgwM2U4MzNINDg1OD UxZTc4ODc1MTY2M210NjU5ZjBhZjVhNjk0OTIhNGEx0ThmYWVINmFIZjlyNmMwZDA3MDM0 NjJkZDhhMml4ZmRhYjc3NmZINDFkODkyYjBhYjY3MDQ10GVIMjdhYmUwZTlyNGlxYmQyZDIz ZjliM2E5ZGQ5NmNhZDQxOTM4NTI0Mjc3MzBIOWEwZWE1Njk3Yj gxY2ViNTQ10WULnoi/VD ULIJTEuDJqne0GMRHesi/8PTnZj02yqbmKbFklPjwMhe7FUhFAOw74S+i+PokOREo5XhdP+ y9 GuBjuYTvrE0xGx20sSfNS5kfRXXH1DaTnb70yufe9r6mMIQ6 1 e6E0SRUIdU6YVupz0hhgd4D of SBbFR30vg0S+pUxDYgmEOr/RA+fYi47tuHQMh+dynZqQspNdmRUmkjEpFqF03sPHS/10injqo 1Gsf8+xn52XE2q/WdnU+4XjWnilisVNAjv2nsL+s2TG11HbgocmpQoxy0B0SXPcRv+2JekV37 k1XyONZk9YH+DV3aWYPXt+ym+wG0XNTqPHIU1JWAZqI2NK/cSXt9DMtCtcb8czRj6G91XvJ9 Eny7t06xPd9BGio9M+3QuUkZHLEmJiAvgvB6R/X/3whBqk6zMHQLfo+VJcX9umW5mRtgCjzS PW6lzzFCGt/B4SK4PxT52ZC0B2kWD8VMyNffrlsTG4XUesgx47Nd5xML8w5pj/fZwKNK+EfKIP ==Z1 ow29A9N3uLIXBX62Lh0yj/1iqfJ2FNR7AIONSEjwKoggVmkxDiuGaQi+TurpxBgat1g ### End ASRAR El Mojahedeen v2.0 Encrypted Message ### Displaying 1 items Hidden fields ``` TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Boolean Equations * Basic fingerprints can also use Boolean equations: ``` appid('voip/sip/IMS', 6.0, wireshark='sip') = ('via: sip' or 'v: sip') and 'cseq:' and ( 'p-access-network-info:' or 'p-called-party-id:' or 'p-charging-vector:' or 'p-charging-vector-addresses:' or 'p-media-authorization:' or 'security-verify:' or 'proxy-authorization:' and 'scscf or 'path:' and 'pcscf or 'path:' and 'scscf ); ``` TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Regular Expressions * And Regular Expressions ``` fingerprint('encryption/mojahedeen2')= /(?:Begin|End).ASRAR.El.Mojahedeen.v2\..{0,5}Encrypted.Message/ or /Mojahedeen.v2\..{0,5}Encrypted.Message/ or /(?:Begin|End).Al-Ekhlaas.Network.ASRAR.El.Moujahedeen.V2/ or ``` * Regular expressions must include a fixed "anchor" meeting the minimum keyword length. ``` Bad: /[A-Z]{3}-[0-9]{3,5}/ OK: /ABC-[0-9]{3,5}/ ``` TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Binary Patterns * And Binary Patterns ``` fingerprint('botnet/IO/XXPW0023') = $http and '\x53\x53\x48\x00\x00\x00\x00\x00'c and '\x00\x00\x00\x00\x03\x00\x53\x4D\x52'c; fingerprint('botnet/IO/XXPW0023') = $http and hex('5353480000000000') and hex('00000000300534D52'); ``` TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Positional Logic ``` fingerprint('botnet/IO/XXPW0023') = pos('\x53\x53\x48\x00\x00\x00\x00\x00') == 4 and pos('\x00\x00\x00\x00\x03\x00\x53\x4D\x52') == 24; fingerprint('botnet/IO/XXPW0023') = $http and (pos('\x53\x53\x48\x00\x00\x00\x00\x00') >= 144 and pos('\x53\x53\x48\x00\x00\x00\x00\x00') <= 184) and (pos('\x00\x00\x00\x00\x03\x00\x53\x4D\x52') >= 164 and pos('\x00\x00\x00\x00\x03\x00\x53\x4D\x52') <= 204); ``` TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## When that's not enough... * For example, take the first scenario: "I want to look for documents from Iran that mention a banned item" * Just using keywords with Boolean equations, how could we restrict the term to only a document body and only coming from Iran? TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Context Sensitive Scanning * X-KEYSCORE's context sensitive scanning engine allows you to explicitly say where you want a term to hit. * As an early example, the Tech Strings in Documents capability allowed analysts to restrict terms to only Email, Chat or Documents Bodies * The full XKS Context Sensitive Scanning engine allows for over 70 unique contexts to be used as part of an fingerprint TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Context Sensitive Scanning * For example, take the first scenario: "I want to look for documents from Iran that mention a banned item" * Using the XKS context for Country Code (based on NKB information) and the XKS context for Document Bodies, this easily becomes: ``` fingerprint('demo/scenario1') = cc('ir') and doc_body('banned item') ``` TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Context Sensitive Scanning * As another example, let's say we want to tag all Iphone usage * Using the XKS context for User Agent this easily becomes ``` fingerprint('demo/scenario2') = user_agent('iphone'); ``` TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## USSID18/HRA Considerations * XKS Fingerprints may not be USSID18 or HRA compliant if they are queried on by themselves * For example, we may want to fingerprint the use of mobile web devices like the IPhone, so that attribute could be used as part of a more complex query. * But querying for the IPhone fingerprint itself would be a USSID18 and HRA violation. TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## USSID18/HRA Considerations * But if you want to look for an IPhone user from an Iranian Proxy accessing his Mail.ru account: ``` IP Address: 78.[REDACTED] Either AppID (+Fingerprints) [fulltext]: Field Builder AppID (+Fingerprints) browser/cellphone/iphone Add to Field Close Field Builder AppID (+Fingerprints) mail/webmail/mailru mail/webmail/mailru mail/webmail/mailru/attachment mail/webmail/mailru/post ``` TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Context Sensitive Scanning What contexts are available for use in XKS Fingerprints? TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## HTTP Activity Contexts (1 of 2) | | | |---|---| | html_title(expr) | The normalized extracted text web page titles
`html_title('how to' and 'bomb')` | | http_host(expr) | The "Host:" name given in the http header.
`http_host('yahoo.com')` | | http_url(expr) | Every URL from HTTP GET and POST commands.
`http_url('/mail/inbox?action=delete')` | | http_url_args(expr) | All arguments given as part of a URL (ie. all text following the '?' in a URL string)
`http_url('action=delete')` | | http_referer(expr) | The "Referer:" URL given in the HTTP header
`http_referer('http://badwebsite/cp?action=show')` | | http_language(expr) | The normalized two letter iso-6393 language code as inferred from any http and or html header info
`http_language('fa' or 'de')` | TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## HTTP Activity Contexts (2 of 2) | | | |---|---| | http_cookie(expr) | The "Cookie:" field given in the http header.
`http_cookie(/PREF=\d\d[a-z]/)` | | http_server(expr) | The "Server:" type name in the http header.
`http_server('GWS/2.1' or 'Apache')` | | http_user_agent(expr) | The "User-Agent:" field given in the http header.
`http_user_agent(/Mozilla\/[45]/ or 'Chrome')` | | web_search(expr) | The normalized extracted text from web searches
`web_search('ricin' or 'plague')` | | x_forwarded_for(expr) | The X-Forwarded For IP address from the HTTP Header
`x_forwarded_for('1.2.3.4')` | TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Protocol Contexts 1 of 2 | | | |---|---| | ip(expr) | The source or destination IP address of the session
`ip('127.0.0.1')` | | from_ip(expr) | The source IP address of the session
`from_ip('127.0.0.1')` | | to_ip(expr) | Every URL from HTTP GET and POST commands.
`to_ip('127.0.0.1')` | | ip_subnet(expr) | IP subnet in CIDR notation.
`ip_subnet('7.211.143.148/24')` | | port(expr) | The source or destination TCP or UDP port number.
`port('22')` | | from_port(expr) | The source TCP or UDP port number.
`from_port('22')` | | to_port(expr) | The destination TCP or UDP port number.
`to_port('22')` | TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Protocol Contexts 1 of 2 | | | |---|---| | cc(expr) | The country (either to OR from) based on IP address
`cc('ir' or 'pk')` | | from_cc(expr) | The source country based on IP address
`from_cc('ir' or 'pk')` | | to_cc(expr) | The destination country based on IP address
`to_cc('ir' or 'pk')` | | protocol(expr) | The textual form of the IP next protocol.
`protocol('TCP')` | | next_protocol(expr) | The textual form of the IP next protocol.
`ip_next_protocol('17')` | | mac_address(expr) | The MAC address of the target network device.
`mac_address('00:16:3E:3F:BD:EF')` | TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Communication Based Contexts | | | |---|---| | email_body(expr) | The UTF-8 normalized text of all email bodies.
`email_body('how to' and 'build' and ('bomb' or 'weapon'))` | | chat_body(expr) | The UTF-8 normalized text of all chat bodies.
`chat_body('how to' and 'build' and ('bomb' or 'weapon'))` | | document_body(expr) | The UTF-8 normalized text of the Office document. –
Office documents include (but are not limited to) Microsoft Office, Open Office, Google Docs and Spreadsheets.
`document_body('how to' and 'build' and ('bomb' or 'weapon'))` | | calendar_body(expr) | The UTF-8 normalized text of all calendars. An example is Google Calendar.
`calendar_body('wedding')` | | archive_files(expr) | Matches a list of files from within an archive. For example is a ZIP file is transmitted, all names of files within are passed to this context.
`archive_files('bad.dll' or 'virus.doc')` | | http_post_body(expr) | The UTF-8 normalized text HTTP url-encoded POSTS.
`http_post_body('action=send' and 'badguy@yahoo')` | TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Communication Based Contexts ## Aliases | | | |---|---| | doc_email_body(expr) | This covers the email_body and document_body contexts
`doc_email_body('how to' and 'build' and ('bomb' or 'weapon'))` | | communication_body(expr) | This covers the email_body, document_body and chat_body contexts
`chat_body('how to' and 'build' and ('bomb' or 'weapon'))` | TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Context sensitivity Why use context-sensitive scanning? * More intuitive - you can say what you mean * More accurate - if 'maps.google.com' is mentioned in a blog post, you don't want to try processing it as a Google Maps session * Better performance for XKEYSCORE TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Examples * "I want to look for people doing web searches on Jihad from Kabul" * Using the from_city() and web_search() context this becomes ``` fingerprint('demo/scenario3') = from_city('kabul') and web_search('jihad'); ``` TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Examples * "I want to look for people using Mojahedeen Secrets encryption from an IPhone" * You can even use existing fingerprints in a fingerprint definition! So this becomes: ``` fingerprint('demo/scenario4') = fingerprint('encryption/mojahdeen2' and fingerprint('browser/cellphone/iphone') ``` TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Examples * "I want to look for documents containing this regular expression" * Using doc_body this becomes: ``` fingerprint('demo/scenario5') = doc_body(/blah[a-z]{3-5}something/) ``` TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Example 4 * "I want to look for E-mails that mention words from various categories of interest to CP" * You can use multiple variables in an equation like this: ``` topic('wmd/acw/govtorgs') = email_body($acwitems and $acwpositions and ($acwcountries or $acwbrokers or $acwports)); ``` TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Example 4 * $acwitems = 'machine gun' or 'grenade' or 'AK 47' * $acwpositions = 'minister of defence' or 'defense minister' * $acwcountries = 'somalia' or 'liberia' or 'sudan' * $acwbrokers = 'south africa' or 'serbia' or 'bulgaria' * $acwports = 'rangood' or 'albasra' or 'dar es salam' ``` topic('wmd/acw/govtorgs') = email_body($acwitems and $acwpositions and ($acwcountries or $acwbrokers or $acwports)); ``` TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Advanced Code-Based Fingerprints * What happens when there are no keywords or regular expressions that will help identify the traffic of interest to you? * As enough example, many of the CT Targets are now smart enough to not leave the Mojahedeen Secrets header in the E-mails they send. How can we detect that the E-mail (which looks like junk) is in fact Mojahedeen Secrets encrypted text * A C++ code fingerprint can help evaluate that data TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Code Based Fingerprint ``` لوحة التحكم الإعدادات والشيرت البريد الالكتروني / كلمه امرو ناث المشرف تعديل الملي الشخصي عديل الخيارات الرسائل الغنية وائمة الرسائل ارسال رسالة جديدة تتبع الريال alvall سابيع بيشتر بيها قائمة الاشتراكات تعديل المجلدت الرقابة اليه تبع المعروفة المشاركات المحذوفة الحوضيع المراقة الحشاركت المراقبة alcgitaال التذكير بالجيب قبائر الأسدقاء / التعامل العمليات المرفقة تاريخ التسجيل : 2007 Mar 1,199 احشارکت رسالة خاصة نفضا PM 08:04 2008-18-02 غير متواجد حالياً larvad tawab افراصي في تفضل . r/RgTzT/ATRhN2E1Zjg1OWQyNWRjMmE2ZTdINzZmZDhI0DUxZWZhMDQ1MjYwMjVIZGUO ZGYwMjdkMmJmNTA4ZDY2Yjk0MGU2NGNIYjg6MzNjZTc6MThjY2 Y1ZmY6MTgzZDIkYjhjMTE XOGYZYJc1ZDdIMDAxNTQzZmVINDVIY2YYMGJJYJU20DkyYJdmYJFJYJAzMWM5ZDQ20WFIMzg 4NThhM211Mjc5ODkzZGNh0GRmNWJmNjVIZjQ0MjMxNDM4MDIyOTg1MmRjMGJINGNKYTN KYTQ4MzMxZIRIN2FiNjI3MjE1NG|3MTA3ZDQ4NWRmYzMyOTUzZjZIMjg3NjQ10GQ4MTA3N TU2N2ZkN2ZjYzUzYzYyMjFIODAwN2VkM2U5MTZINDY2MmM2ZTY IYJQ2Yz100GQ20DUxNW VKMjI2MWVINDAyOGI0MThkMTdhNTY1YzIxMDgyOGZIM2wZWZjMDgwM2U4MzNINDg1OD UxZTc4ODc1MTY2M210NjU5ZjBhZjVhNjk0OTIhNGEx0ThmYWVINmFIZjlyNmMwZDA3MDM0 NjJkZDhhMml4ZmRhYjc3NmZINDFkODkyYjBhYjY3MDQ10GVIMjdhYmUwZTlyNGlxYmQyZDIz ZjliM2E5ZGQ5NmNhZDQxOTM4NTI0Mjc3MzBIOWEwZWE1Njk3Yj gxY2ViNTQ10WULnoi/VD ULIJTEuDJqne0GMRHesi/8PTnZj02yqbmKbFklPjwMhe7FUhFAOw74S+i+PokOREo5XhdP+ y9 GuBjuYTvrE0xGx20sSfNS5kfRXXH1DaTnb70yufe9r6mMIQ6 1 e6E0SRUIdU6YVupz0hhgd4D of SBbFR30vg0S+pUxDYgmEOr/RA+fYi47tuHQMh+dynZqQspNdmRUmkjEpFqF03sPHS/10injqo 1Gsf8+xn52XE2q/WdnU+4XjWnifisVNAjv2nsL+s2TG11HbgocmpQoxy0B0SXPcRv+2JekV37 k1XyONZk9YH+DV3aWYPXt+ym+wG0XNTqPHIU1JWAZqI2NK/cSXt9DMtCtcb8czRj6G91XvJ9 Eny7t06xPd9BGio9M+3QuUkZHLEmJiAvgvB6R/X/3whBqk6zMHQLfo+VJcX9umW5mRtgCjzS PW6lzzFCGt/B4SK4PxT52ZC0B2kWD8VMyNffrlsTG4XUesgx47Nd5xML8w5pj/fZwKNK+EfKIP ==Z1 ow29A9N3uLIXBX62Lh0yj/1iqfJ2FNR7AIONSEjwKoggVmkxDiuGaQi+TurpxBgat1g Displaying 1 items Hidden fields ``` TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Advanced Code-Based Fingerprints ``` fingerprint('encryption/mojaheden2/hidden44') = $mol_cipher_first_test : c-+ extractors: {{ msg1 = /((a-zA-Z0-9\+\/](6) [DHLPTXofjnrvs37\/] [4-9\+\/]) AT ([ABEFIJMNQFUVYZcdghkl] [hijklmwxyz012345] [ΜΝΟΥ]] [[GTWjnz2](12))/c msg2 /((a-zA-Z0-9\+\/](6) [DIILPTXofjnrvz37\/] [4-3\+\/]) AU ([ADEFIJMNQFUVYZcdchkl] [hijklmwxyz012045] [ΜΝΟΥZ] [DGTWjwz2](12))/c; }} main : {{ std::string msg; if (msc1 msg = msg1[0]; else if (msg2) msg = msg2[0]; else return false; char buf[16]: char chunk1 [16]; char chunk2 [16]; char chunk3 [16]; if (crue): snprintf(chunk1, 16, "%02x%02x%02x%02x", msg[10] & 0xff, msg[11] & 0xff, msg[12] & 0xff, msg[13] & 0xff); snprintf(chunk2, 16, "%02x02x02xC2x", msg[14] & 0xff, msg[15] & 0xff, msg[16] & 0xff, msg[17] & 0xff); snprintf(chunk3, 16, "%02x%02x%02x%02x", msg[18] & 0xff, msg[19] & 0xff, mas[20] & 0xff, msg[21] & 0xff); if(!((strcmp(chunk, chunk2) == 0) || (strcmp(chunk2, chunk3; == 0) || (strcmp(chunk1, chunk3; == 0)){ std::string msc_decoded = xks::base64dec imsg); for(size_t i = 7; i < msg_decoded.size(); i++){ if (msg_decoded[i] < '0' || (msg_decoded[i] > '9' && msg_decoded[i] < 'a') || msg_decoded[i] > 'f'){ return false; } } snprintf(buf, 16, "%02x02x02x%02x", msg_decoded[1] & 0xff, msg_decoded[2] & 0xff, msg_decoded[3] & 0xff, msg_decoded[4] & 0xff); std::string keyid hex = buf; ``` ``` Field Builder AppID (+Fingerprints) encryption/mojaheden2/hidden44 Add to Field Close ``` TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Advanced Code-Based Fingerprints * As another example, some of the activity from the Conficker botnet simply can't be detected with keywords or regular expressions * In cases like this, C++ code can be used inside a fingerprint to test the data further TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Advanced Code-Based Fingerprints ``` fingerprint('botnet/conficker_p2p_udp_data', 7.5) = udp and not 'to ap 00' and not 'fm ap 00 : c++;; // Classification: CONFIDENTIAL//REL TO USA, FVEY // NOT releasable to third-parties uint8_t key8; uint8_t key9; uint8_t pkt type; uint0_t decrypted_bytes[4]; uint32_t running_hash - 0; uint32_t K_h1ck; uint32_t K_lct; uint32_t store_hashes[4] = {11,11,1,11}; uint32_t min_pkt_len; uint32_t max_pkt_len; uint32_t t; packet_t pkt; while (pkt - get packet (11 { if (pkt.size < 10) return false; key8 = (uint8_t)pkt.data[7]<<1 | pkt.data[7]>>7)&1)); key9 = (uint8_t)key8<<2 | ((pkt.data[7]>>5|&1)); if ((key9 ^ pkt.data[0] &0xfc) != 0x80; return false; // Not Conficker, so abort if ((key0 ^ pkt.data[0] & 0x02) return false; // bi not set for UDP packets if (pkt.size<23) continue; if ((key9 ^ pkt.data[9] !=0x80) continue; pkt type - (key8 ^ pkt.data[8])>>3; if (pkt_type & 0x10) // summary continue; if (!(pkt_type & 0x08:: // not a data packet continue; min_pkt_len = 23; max_pkt_len = (uint32_t pkt.size; K high = uint32_t:pkt.data[7])<<24u | uint32_t(pkt.data[6])<<15 | uint32_t pkt.data[5])<=8) // decrypt data { ``` ``` Field Builder AppID (+Fingerprints) botnet/conficker_p2p_udp_data Add to Field Close ``` TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Meta-data Extracting Fingerprints * What happens when you find data and want some pieces of meta-data extracted? * XKS Fingerprints can be used to extract meta-data to select XKS database tables. * Or if no existing database is applicable, you can define your own database schema for the meta-data TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Free File Upload Sites * As a real life example, think of all of various Free File Upload (FFU) sites of interest * When a user uploads a document they get a response page that looks like this: TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Free File Upload Sites ``` Welcome to zSHARE With zSHARE you can upload files, images, videos, audio and flash for free. Simply use the upload form below and start sharing! You can also use zSHARE as your personal file storage: backup your data and protect your files. First Time? Read our FAQ! * Upload now * Login * Create Free Account * Premium * FAQ File Uploaded The file khi pics.zip was successfully uploaded! (4.04MB). You're now ready to share it with unlimited people or keep it as a backup. Download Link http://www.zshare.net/download/637199570b174c9f/ Link for forums: [URL=http://www.zshare.net/download/637199570b174 Direct Link: http://www.zshare.net/download/637199570b174c9f/ Delete Link: http://www.zshare.net/delete.html?63719957-7c8893b1k E-mail Me This Info To receive all the info on the file you uploaded, such as removal instructions and download link, enter your e-mail address on the field below: Your e-mail: ``` TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Free File Upload Sites * Look at all the great information on that page: ``` File Uploaded The file khi pics.zip was successfully uploaded! (4.04MB). You're now ready to share it with unlimited people or keep it as a backup. Download Link http://www.zshare.net/download/637199570b174c9f Link for forums: [URL=http://www.zshare.net/download/637199570b174 Direct Link: http://www.zshare.net/download/637199570b174c9f/ Delete Link: http://www.zshare.net/delete.html?63719957-7c8893b1b ``` TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Free File Upload Sites * How can we quickly get that information extracted as Meta-data and be agile enough to respond to each FFU site which may have its own format * XKS "v4" Fingerprints allow you to use the XKS Fingerprint Language to extract meta-data into the XKS database * Fingerprints are deployed within an hour of being accepted meaning you no longer need to wait for all 130+ XKS sites to be upgraded to have the latest and greatest capabilities. TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Free File Upload Sites ``` appid('filetransfer/web/zshare_net/upload/response', 5.0) = http_title('zSHARE') and 'zshare.net/delete.html' : c++ extractors: {{ wft_file_name = /The sfile\s([^<](1,300})\sLogged in as: ([^<]+)<\/small>/; ); main = {{ if (wft_delete_url ) { DB["web_file_transfer"]["wft_upload_id"] = wft_upload_id[0]; DB["web_file_transfer"]["wft_delete"] = wft_delete_url[0]+"-"+wft_delete_url[1]; DB["web_file_transfer"]["wft_site_name"] = "zshare.net"; DB["web_file_transfer"]["transfer_type"] = "upload"; if (wft_file_name) { DB["web_file_transfer"]["wft_filename"] = wft_file_name[0]; } if (wft_url) { DB["web_file_transfer"]["wft_url"] = wft_url[0] } if (wft_uploader_username) { DB["web_file_transfer"]["uploader_username"] = wft_uploader_username[0]; } DB.apply(); } else { logger.debug("filetransfer/web/zshare.net/upload/response: Host regexs didn't match"); } return true; }}; ``` TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Meta-data Extracting Fingerprints * All you do is tell XKS when to start extracting meta-data ``` appid('filetransfer/web/zshare_net/upload/response', 5.0)= http_title('zSHARE') and 'zshare.net/delete.html' : c++ ``` TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL # Meta-data Extracting Fingerprints * Use Regular Expressions to tell it what to extract: ``` extractors: {{ wft_file_name = /The\sfile\s([^<]{1,300})\sLogged in as: ([^<]+)<\/small>/; }} ``` TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Meta-data Extracting Fingerprints * Finally tell it which database tables you want to store the information: ``` main = {{ if (wft_delete_url) { DB["web_file_transfer"]["wft_upload_id"] = wft_upload_id[0]; DB["web_file_transfer"]["wft_delete"] = wft_delete_url[0]+"-"+wft_delete_url[1]; DB["web_file_transfer"]["wft_site_name"] = "zshare.net"; DB["web_file_transfer"]["transfer_type"] = "upload"; ``` | File URL | Filename | |:---|:---| | http://www.zshare.net/download/637199570b174c9f | khi pics.zip | | Transfer Type | Upload ID | Delete ID | Site Name | |:---|:---|:---|:---| | upload | 63719957 | 7c8893b1bf04170771dca3e7f0756a26 | zshare.net | TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Meta-data Extracting Fingerprints * What if the meta-data you want to extract doesn't fit nicely into any of the existing XKS meta-data tables? ``` Classic A-M * ASF and WMV Metadata * Alert * BlackBerry * CNE * Call Logs * Category DNI * Cellular DNI * Cisco Passwords * Document Metadata * Document Tagging * Email Addresses * Extracted Files * Full Log DNI * HTTP Activity * IRC Cafe Geolocation * Logins and Passwords Classic N-Z * Network Logs * PDF Metadata * PILBEAM * Phone Number Extractor * RBGAN * REGISTRY * RTP * Radius Logs * RealMedia Metadata * SIP * TOR Log * Tech Strings in Documents * User Activity * WLAN * Web Proxy * Wireshark ``` TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Meta-data Extracting Fingerprints * Define your own with the "Microplugin" query forms: ``` Microplugins * Beb Flood * Cone Blindmatesmen Beacon * Cone Byzantine Raptor Trojan3 * Cone Traffic * Cone Victim Id * Encryption Steg JSTEG * Exif Metadata * Ipv6 Addresses * Mailer Accounts * Ms2 Extract Keyids * Munged Traffic * NetStrings * QUANTUMBOT Table * Saudi Mfa Visa * Udp Xxpj9010 Lzo Networkname * VPN External IP Addresses * Vpn Users * Web Geo Cell Towers * Web Geo Results * Web Geo Wifi Towers * _sub Dictionary Code Snippet ``` TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Meta-data Extracting Fingerprints * Example MS2 KeyIDs ``` Search: Ms2 Extract Keyids Query Name: [REDACTED] Justification: Recent Justifications Additional Justification: Miranda Number: Datetime: 1 Day Start: 2010-05-03 00:00 Stop: 2010-05-04 23:59 ejKeyID: Username: IP Address: From [IP Address Field Builder] IP Address: To [IP Address Field Builder] Port: From Port: To ``` TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Meta-data Extracting Fingerprints ``` Search: Ccne Byzantine Raptor Trojan3 Query Name: [REDACTED] Justification: Recent Justifications Additional Justification: Miranda Number: Datetime: 1 Day Start: 2010-05-03 00:00 Stop: 2010-05-04 23:59 brt_decrypt: brt_hostname: brt_ipaddress: brt_length: brt_osversion: DB[" brt_packet_type: DB[ brt_sequence_num: brt_username: Username : IP Address: From [IP Address Field Builder] IP Address: To [IP Address Field Builder] ``` TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## New Fingerprint GUI * New XKS Fingerprint GUI allows analysts to directly test, submit and manage fingerprints through the web ``` Navigation Menu * Fingerprints * Validate / Submit * Approved * Pending * My Signatures Fingerprint Validation / Submittal Step #1 Compile Step #2 Test Against Session Data Step #3 Save Help Global Variable Declarations Type or paste any global VARIABLE DECLARATIONS here. Signature Type or paste a FINGERPRINT definition here. Press Compile when done editing ``` TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## New Fingerprint GUI * New XKS Fingerprint GUI allows analysts to directly test, submit and manage fingerprints through the web ``` Fingerprint Validation / Submittal Step #1 Compile Step #2 Test Against Session Data Step #3 Save Global Variable Declarations Stest = 'bomb' or 'missle' or 'ied'; Signature fingerprin= ('test/testl') = email_body($test); Results SUCCESS! Congratulations, your fingerprint was successfully compiled! Now use the Test button to run it against the cesignated session data. ``` TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL -- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Questions? TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Syntax Rules * The definition of the fingerprint will look like this: `fingerprint('test/blah/something', owner = [REDACTED]) =` Note the single quotes needed for the fingerprint name and owner TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Syntax Rules * Secondly every fingerprint definition must be completed by a semi-colon. `fingerprint('test/blah/something', owner = [REDACTED]) = 'badguy' ;` TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Syntax Rules * Variables also must be completed by a semi-colon. ``` $badguy = 'bomb' or 'gun' or 'weapon' ; fingerprint('test/blah/something', owner = [REDACTED]) = $badguy; ``` TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL --- TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL ## Syntax Rules * Definitions and Variables can span multiple lines ``` $badguy = 'bomb' or 'gun' or 'weapon' ; fingerprint('test/blah/something', owner = '[REDACTED]') = $badguy; ``` TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL