15 KiB
TOP SECRET//COMINT//REL TO USA, FVEY
(S//SI//REL) User-agents: Why and How and How to
NATIONAL SECURITY AGENCY UNITED STATES OF AMERICA
The overall classification of this briefing to TOP SECRET//COMINT//REL TO USA, FVEY
[REDACTED] – S2I61 July 2010
Derived From NSA/CSSM 1-52 Dated 20070108 Declassify on: 20320108
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
Agenda
- WHY: Learn why we care about user agents (UAs)
- HOW: Learn how to read a user-agent
- HOW TO: (get it?) Learn how to use user-agents in our tools
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
WHY?
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
What is a User-Agent?
A user-agent is a string which lets websites know your:
- type of web-browser or application
- Operating System
- Security settings or permissions
- Versions of relevant programs (media, java, etc.)
- Etc. (Language settings, ad-ware)
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
Why would I want to give a website that?
- Compatibility
- Specific Website Features
- Security permissions
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
User-Agents Can
- Link a target's “selected” activity to their unselected web-browsing
- Create a tentative link between targets that have the same user-agent
- Identify CNE opportunities
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
BUT...
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
User-Agents Also
- Can vary from very unique to extremely common
- Change with software updates
- Only identify the web-browser
- 2 web-browsers = 2 user-agents
- Can't be trusted...
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
Who started this nonsense?
A History Lesson
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
The Great Browser Wars
Back...in the 20th Century!
Before Now but After What is Below
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
Remember Frames?
- Netscape's new fancy web-browser support them!
- The original “web-browser” Mosaic did not
- And so began browser “sniffing”
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
Worthless Trivia!
- Mosaic was the first web-browser to embed images with text
- It supported FTP, Usenet, and Gopher!
- Its web-browser competitors at the time were Erwise and ViolaWWW
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
The Great Internet Explorer Hoax
Ever wonder why so many user agents start with “Mozilla” but aren't Firefox?
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
Internet Explorer fools us all
- When Internet Explorer was released it did frames too!
- But since its user-agent didn't say so, no websites would send their super cool frames version to the IE users
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
What to do?
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
LIE, of course!
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
Internet Explorer starts to spoof
- Internet Explorer changed its user agent starting their user agent with Mozilla/1.22
- Mozilla/1.22 (compatible; MSIE 2.0; Windows 95)
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
And it continues to this very day...
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
The How...
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
Three Basic Pieces of the UA
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
Mozilla/1.22 (compatible; MSIE 2.0; Windows 95) Mozilla/1.22
Part 1: The Netscape Historical Token
- Appears in primarily Mozilla Firefox, Google Chrome, and MSIE browsers
- Modern Version: Mozilla 4.0 or Mozilla 5.0
- Does not indicate a target uses “Mozilla Firefox”
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
Mozilla/1.22 (compatible; MSIE 2.0; Windows 95) MSIE 2.0
Part 2: The Web Browser Identifier
- Appears in generally all user-agents
- Not always in the same place, but usually self explanatory
- Opera X.X = Opera
- Firefox X.X = Firefox
- Safari X.X= Safari
- Chrome X.X Safari X.X = Google Chrome
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
Mozilla/1.22 (compatible; MSIE 2.0; Windows 95) Windows 95
Part 3: The Operating System Token
- Appears in basically all HTTP user-agent strings
- Examples:
- Windows NT 6.1 = Windows Version 7
- Windows NT 6.0 = Windows Vista
- Windows NT 5.2 = Windows XP 64bit
- Windows NT 5.1 = Windows XP
- Windows NT 4.0 actually equals Windows NT 4.0
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
Other operating systems UAs
- Mobile
- MAC OS X
- Linux
- Linux i686
- Free BSD
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
Game Consoles
- Opera/9.30 (Nintendo Wii; U; 2047-7; en)
- Mozilla/5.0 (Playstation 3; 2.00)
- PSP (PlayStation Portable); 2.00
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
Mobile User Agents
- Usually self-explanatory
- Iphone
- Ipad
- Blackberry
- Android
- Mobile user agents also usually give you the phone model (Read: IMEI correlation opportunities)
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
Ever wonder what that was?
- Gecko: a rendering engine used by Firefox and others
- AppleWebkit: Apple's version of KHTML rendering engine used in Safari and Chrome most commonly
- Presto : the “core” of the Opera platform suite
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
Ever Wonder Contd.
- .NET CLR is the .NET Framework version
- SV1 is an artifact created by MSIE 6.0 to make its security better
- Win64 can indicate that the system is running a 64 bit processor
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
Ever Wonder... One more
Many web browsers will also have an “encryption strength” marker
U = USA (128 bit encryption) I = International (40 bit encryption) N = No encryption (Woo!)
Most Browsers nowadays come with a U Since the USG no longer requires encryption changes for international usage.
TOP SECRET//COMINT//REL TO USA, FVEY
--
TOP SECRET//COMINT//REL TO USA, FVEY
Your target's user-agents can shorten and lengthen!
- Each website may require different information
- Longer user-agents may have various rendering engine, java versions, and language settings
- If you see a shortened or longer version of a UA close to your targets logins. Check it out! Carefully...
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
The How To...
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
Querying in Xkeyscore
Navigation Menu
* Classic A-M
* Alert
* BlackBerry
* Call Logs
* Category DNI
* Cellular DNI
* Cisco Passwords
* Clarent
* Crypt Scanner
* DNS
* Document Metadata
* Document Tagging
* Email Addresses
* Extracted Files
* Full Log DNI
* Gen Info
* HTTP Activity
* IKE Parser
Browser: Mozilla 5.0*
Remember: Since Xkeyscore no longer supports leading wildcards you need to be specific with your User-Agent
A User-agent alone is not a strong query Time Frame, Active IP, Country, etc. all will help make your query compliant
Browser
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.99 Safari/533.4
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
Uniqueness – When to use a User-Agent
Reports
VIEW
FILTERS
Browser
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)
Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)
Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)
Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)
Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)
Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)
Geo Info
Sort Ascending
Sort Descending
Filters
Color By
Group By
Histogram
Pivot Data
Histogram Grid
Show/Hide...
AutoFit Column Width
Results of an IP-based search for a target
Histogram Grid
Page 1 of 1
Clear Selection Export
Filter Browser
Count
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
6
Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)
6
Low Number of users and traffic volume as well as only 2 User-agents Verdict: Probably reliable
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
More User = More Problems
| Browser | Count |
|---|---|
| Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) | 6 |
| Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; | 1 |
| Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; | 1 |
| Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; GTB0.0; .NET CLR 2.0.50727; AskTB5.6) | 1 |
| Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; GTB6.5) | 1 |
| Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; GTB6.5; .NET CLR 2.0.50727) | 1 |
| Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; GTB6.5; InfoPath.2) | 2 |
| Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; GTB6.5; InfoPath.2; .NET CLR 2.0.50727; .NET | 1 |
| Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; InfoPath.1; .NET CLR 2.0.50727; .NET CLR 3.0. | 1 |
| Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322) | 1 |
| Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1) | 6 |
| Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322) | 1 |
| Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; FunWebProducts; AskTbPTV/5.8.0.12304) | 1 |
| Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; GTB6; .NET CLR 1.1.4322) | 1 |
| Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; IEMB3; IEMB3) | 1 |
| Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727 | 1 |
| Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; InfoPath.2; .NET CLF | 2 |
Reliability of correlating your target's browser to his web-activity plummets due to User-agent variations and the threat of multiple users.
At this point, cookie and TDI correlation are necessary. User-agents can not be used by themselves.
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
Common Sense Helps
Never assume because a User-agent is complicated that it is unique
- Example: Mozilla 5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.3(KHTML, like Gecko) Chrome/6.0.464.0 Safari/534.3
This is the standard user agent for EVERYONE with an updated Chrome browser using Windows 7.
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
Xkeyscore Storage
Push to Pinwale or Archive Results
Download Session Mode: S
AGILITY-RealTime
Agility
DNI Presenter
PINWALE
PINWALE and Decrypt
Download SOTF Session
Download Session
Download D124 Session
finished
Archive Results
finished
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
Pinwale
Fields to add to Metadata View
AppProc Active User
User Agent
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) ...
Generally speaking, the User Agent listed belongs to the selector in the Active User column (if populated)
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
Add a User-Agent to a compliant query
Fielded Search Form
1. Put in the User Agent that is being search for into the CONTENT\_META field under the fielded search form.
KEYWORDS and
cs=ISO8859_1 ( ( ("Opera Mini") ) )
CONTENT_META
Opera Mini
between tags UserAgent and
Smart Form/Native Query
Native, cs=ISO8859_1 ( (content_meta=(((("Opera Mini")) between \<UserAgent and \/UserAgent))))
(content_meta=(((("Opera Mini")) between \<UserAgent and \/UserAgent )))
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
Not sure the specific User-Agent?
1. Add the following syntax to your query:
\TERM \<yourfield here (E.G. \TERM \<useragent )
2. Apply Native to the field
3. Apply the content_meta document zone to the field.
Free Form Query Smart Form
cs=ISO8859_1(((["selectorl"]))
selector1.
or
cs=ISO8859_1( ("selector2")))
selector2
or
) and
CONTENT_META:Native, cs=ISO8859_1 [[TERM \<useragent ]]
TERM \<useragent
or or and
(TS//SI//REL) This query basically ensures that a certain field exists in each result thus removing all the content not relevant to your query.
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
User-Agent Manipulation
- The best for last
- User-agents can completely be changed or not included by the user!
- By Firefox Plugin
- By Browser settings (Opera)
- Outside programs (TOR Button)
- These programs allow users to have a different user agent for each session!
- User-agents can completely be changed or not included by the user!
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
Questions??
-
Contact Info: [REDACTED]@nsa.ic.gov
-
Website on the High-Side:
-
[REDACTED]
-
[REDACTED]
-
[REDACTED]
-
[REDACTED]
-
Lots of great stuff in open source as well! http://www-archive.mozilla.org/build/user-agents-strings.html
TOP SECRET//COMINT//REL TO USA, FVEY
TOP SECRET//COMINT//REL TO USA, FVEY
Got a Tech Problem?
Have a random SIGDEV question?
Need help with a target using new tech to communicate?
Need help developing an accurate collateral description of a technology?
Want help developing Xkeyscore fingerprints for a weird target behavior?
Let us know : DL S2I61_all
TOP SECRET//COMINT//REL TO USA, FVEY