TOP SECRET//COMINT//REL TO USA, FVEY # (S//SI//REL) User-agents: Why and How and How to NATIONAL SECURITY AGENCY UNITED STATES OF AMERICA The overall classification of this briefing to TOP SECRET//COMINT//REL TO USA, FVEY [REDACTED] – S2I61 July 2010 Derived From NSA/CSSM 1-52 Dated 20070108 Declassify on: 20320108 TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY ## Agenda * WHY: Learn why we care about user agents (UAs) * HOW: Learn how to read a user-agent * HOW TO: (get it?) Learn how to use user-agents in our tools TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY ## WHY? TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY ## What is a User-Agent? A user-agent is a string which lets websites know your: * type of web-browser or application * Operating System * Security settings or permissions * Versions of relevant programs (media, java, etc.) * Etc. (Language settings, ad-ware) TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY ## Why would I want to give a website that? * Compatibility * Specific Website Features * Security permissions TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY ## User-Agents Can * Link a target's “selected” activity to their unselected web-browsing * Create a tentative link between targets that have the same user-agent * Identify CNE opportunities TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY ## BUT... TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY ## User-Agents Also * Can vary from very unique to extremely common * Change with software updates * Only identify the web-browser * 2 web-browsers = 2 user-agents * Can't be trusted... TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY ## Who started this nonsense? A History Lesson TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY ## The Great Browser Wars Back...in the 20th Century! Before Now but After What is Below TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY ## Remember Frames? * Netscape's new fancy web-browser support them! * The original “web-browser” Mosaic did not * And so began browser “sniffing” TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY ## Worthless Trivia! * Mosaic was the first web-browser to embed images with text * It supported FTP, Usenet, and Gopher! * Its web-browser competitors at the time were Erwise and ViolaWWW TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY ## The Great Internet Explorer Hoax Ever wonder why so many user agents start with “Mozilla” but aren't Firefox? TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY ## Internet Explorer fools us all * When Internet Explorer was released it did frames too! * But since its user-agent didn't say so, no websites would send their super cool frames version to the IE users TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY ## What to do? TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY ## LIE, of course! TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY ## Internet Explorer starts to spoof * Internet Explorer changed its user agent starting their user agent with Mozilla/1.22 * Mozilla/1.22 (compatible; MSIE 2.0; Windows 95) TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY ## And it continues to this very day... TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY ## The How... TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY ## Three Basic Pieces of the UA TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY Mozilla/1.22 (compatible; MSIE 2.0; Windows 95) Mozilla/1.22 Part 1: The Netscape Historical Token * Appears in primarily Mozilla Firefox, Google Chrome, and MSIE browsers * Modern Version: Mozilla 4.0 or Mozilla 5.0 * Does not indicate a target uses “Mozilla Firefox” TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY Mozilla/1.22 (compatible; MSIE 2.0; Windows 95) MSIE 2.0 Part 2: The Web Browser Identifier * Appears in generally all user-agents * Not always in the same place, but usually self explanatory * Opera X.X = Opera * Firefox X.X = Firefox * Safari X.X= Safari * Chrome X.X Safari X.X = Google Chrome TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY Mozilla/1.22 (compatible; MSIE 2.0; Windows 95) Windows 95 Part 3: The Operating System Token * Appears in basically all HTTP user-agent strings * Examples: * Windows NT 6.1 = Windows Version 7 * Windows NT 6.0 = Windows Vista * Windows NT 5.2 = Windows XP 64bit * Windows NT 5.1 = Windows XP * Windows NT 4.0 actually equals Windows NT 4.0 TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY ## Other operating systems UAs * Mobile * MAC OS X * Linux * Linux i686 * Free BSD TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY ## Game Consoles * Opera/9.30 (Nintendo Wii; U; 2047-7; en) * Mozilla/5.0 (Playstation 3; 2.00) * PSP (PlayStation Portable); 2.00 TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY ## Mobile User Agents * Usually self-explanatory * Iphone * Ipad * Blackberry * Android * Mobile user agents also usually give you the phone model (Read: IMEI correlation opportunities) TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY ## Ever wonder what that was? * Gecko: a rendering engine used by Firefox and others * AppleWebkit: Apple's version of KHTML rendering engine used in Safari and Chrome most commonly * Presto : the “core” of the Opera platform suite TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY ## Ever Wonder Contd. * .NET CLR is the .NET Framework version * SV1 is an artifact created by MSIE 6.0 to make its security better * Win64 can indicate that the system is running a 64 bit processor TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY ## Ever Wonder... One more Many web browsers will also have an “encryption strength” marker U = USA (128 bit encryption) I = International (40 bit encryption) N = No encryption (Woo!) Most Browsers nowadays come with a U Since the USG no longer requires encryption changes for international usage. TOP SECRET//COMINT//REL TO USA, FVEY -- TOP SECRET//COMINT//REL TO USA, FVEY ## Your target's user-agents can shorten and lengthen! * Each website may require different information * Longer user-agents may have various rendering engine, java versions, and language settings * If you see a shortened or longer version of a UA close to your targets logins. Check it out! Carefully... TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY ## The How To... TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY ## Querying in Xkeyscore ``` Navigation Menu * Classic A-M * Alert * BlackBerry * Call Logs * Category DNI * Cellular DNI * Cisco Passwords * Clarent * Crypt Scanner * DNS * Document Metadata * Document Tagging * Email Addresses * Extracted Files * Full Log DNI * Gen Info * HTTP Activity * IKE Parser ``` ``` Browser: Mozilla 5.0* ``` Remember: Since Xkeyscore no longer supports leading wildcards you need to be specific with your User-Agent A User-agent alone is not a strong query Time Frame, Active IP, Country, etc. all will help make your query compliant ``` Browser Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.99 Safari/533.4 ``` TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY ## Uniqueness – When to use a User-Agent ``` Reports VIEW FILTERS Browser Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1) Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1) Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1) Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1) Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1) Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1) Geo Info Sort Ascending Sort Descending Filters Color By Group By Histogram Pivot Data Histogram Grid Show/Hide... AutoFit Column Width ``` Results of an IP-based search for a target ``` Histogram Grid Page 1 of 1 Clear Selection Export Filter Browser Count Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) 6 Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1) 6 ``` Low Number of users and traffic volume as well as only 2 User-agents Verdict: Probably reliable TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY ## More User = More Problems | Browser | Count | |-------- |-------| | Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) | 6 | | Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; | 1 | | Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; | 1 | | Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; GTB0.0; .NET CLR 2.0.50727; AskTB5.6) | 1 | | Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; GTB6.5) | 1 | | Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; GTB6.5; .NET CLR 2.0.50727) | 1 | | Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; GTB6.5; InfoPath.2) | 2 | | Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; GTB6.5; InfoPath.2; .NET CLR 2.0.50727; .NET | 1 | | Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; InfoPath.1; .NET CLR 2.0.50727; .NET CLR 3.0. | 1 | | Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322) | 1 | | Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1) | 6 | | Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322) | 1 | | Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; FunWebProducts; AskTbPTV/5.8.0.12304) | 1 | | Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; GTB6; .NET CLR 1.1.4322) | 1 | | Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; IEMB3; IEMB3) | 1 | | Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727 | 1 | | Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; InfoPath.2; .NET CLF | 2 | Reliability of correlating your target's browser to his web-activity plummets due to User-agent variations and the threat of multiple users. At this point, cookie and TDI correlation are necessary. User-agents can not be used by themselves. TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY ## Common Sense Helps Never assume because a User-agent is complicated that it is unique * Example: Mozilla 5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.3(KHTML, like Gecko) Chrome/6.0.464.0 Safari/534.3 This is the standard user agent for EVERYONE with an updated Chrome browser using Windows 7. TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY ## Xkeyscore Storage Push to Pinwale or Archive Results ``` Download Session Mode: S AGILITY-RealTime Agility DNI Presenter PINWALE PINWALE and Decrypt Download SOTF Session Download Session Download D124 Session finished Archive Results finished ``` TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY ## Pinwale Fields to add to Metadata View ``` AppProc Active User User Agent Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) ... ``` Generally speaking, the User Agent listed belongs to the selector in the Active User column (if populated) TOP SECRET//COMINT//REL TO USA, FVEY --- TOP SECRET//COMINT//REL TO USA, FVEY ## Add a User-Agent to a compliant query Fielded Search Form ``` 1. Put in the User Agent that is being search for into the CONTENT\_META field under the fielded search form. KEYWORDS and cs=ISO8859_1 ( ( ("Opera Mini") ) ) CONTENT_META Opera Mini between tags UserAgent and ``` Smart Form/Native Query ``` Native, cs=ISO8859_1 ( (content_meta=(((("Opera Mini")) between \