first commit

This commit is contained in:
libroot 2026-11-19 04:39:11 +02:00
commit 0d90363116
Signed by: libroot
GPG key ID: 2AC68C029EBE9ACE
6510 changed files with 679146 additions and 0 deletions

View file

@ -0,0 +1,676 @@
TOP SECRET//COMINT//REL TO USA, FVEY
# (S//SI//REL) User-agents: Why and How and How to
NATIONAL SECURITY AGENCY
UNITED STATES OF AMERICA
The overall classification of this briefing to TOP SECRET//COMINT//REL TO USA, FVEY
[REDACTED] – S2I61
July 2010
Derived From
NSA/CSSM 1-52
Dated 20070108
Declassify on: 20320108
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## Agenda
* WHY: Learn why we care about user agents (UAs)
* HOW: Learn how to read a user-agent
* HOW TO: (get it?) Learn how to use user-agents in our tools
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## WHY?
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## What is a User-Agent?
A user-agent is a string which lets websites
know your:
* type of web-browser or application
* Operating System
* Security settings or permissions
* Versions of relevant programs (media, java, etc.)
* Etc. (Language settings, ad-ware)
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## Why would I want to give a website that?
* Compatibility
* Specific Website Features
* Security permissions
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## User-Agents Can
* Link a target's “selected” activity to their unselected web-browsing
* Create a tentative link between targets that have the same user-agent
* Identify CNE opportunities
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## BUT...
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## User-Agents Also
* Can vary from very unique to extremely common
* Change with software updates
* Only identify the web-browser
* 2 web-browsers = 2 user-agents
* Can't be trusted...
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## Who started this nonsense?
A History Lesson
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## The Great Browser Wars
Back...in the 20th Century!
Before Now but After What is Below
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## Remember Frames?
* Netscape's new fancy web-browser support them!
* The original “web-browser” Mosaic did not
* And so began browser “sniffing”
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## Worthless Trivia!
* Mosaic was the first web-browser to embed images with text
* It supported FTP, Usenet, and Gopher!
* Its web-browser competitors at the time were Erwise and ViolaWWW
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## The Great Internet Explorer Hoax
Ever wonder why so many user agents start with “Mozilla” but aren't Firefox?
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## Internet Explorer fools us all
* When Internet Explorer was released it did frames too!
* But since its user-agent didn't say so, no websites would send their super cool frames version to the IE users
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## What to do?
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## LIE, of course!
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## Internet Explorer starts to spoof
* Internet Explorer changed its user agent starting their user agent with Mozilla/1.22
* Mozilla/1.22 (compatible; MSIE 2.0; Windows 95)
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## And it continues to this very day...
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## The How...
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## Three Basic Pieces of the UA
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
Mozilla/1.22 (compatible; MSIE 2.0; Windows 95)
Mozilla/1.22
Part 1: The Netscape Historical Token
* Appears in primarily Mozilla Firefox, Google Chrome, and MSIE browsers
* Modern Version: Mozilla 4.0 or Mozilla 5.0
* Does not indicate a target uses “Mozilla Firefox”
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
Mozilla/1.22 (compatible; MSIE 2.0; Windows 95)
MSIE 2.0
Part 2: The Web Browser Identifier
* Appears in generally all user-agents
* Not always in the same place, but usually self explanatory
* Opera X.X = Opera
* Firefox X.X = Firefox
* Safari X.X= Safari
* Chrome X.X Safari X.X = Google Chrome
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
Mozilla/1.22 (compatible; MSIE 2.0; Windows 95)
Windows 95
Part 3: The Operating System Token
* Appears in basically all HTTP user-agent strings
* Examples:
* Windows NT 6.1 = Windows Version 7
* Windows NT 6.0 = Windows Vista
* Windows NT 5.2 = Windows XP 64bit
* Windows NT 5.1 = Windows XP
* Windows NT 4.0 actually equals Windows NT 4.0
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## Other operating systems UAs
* Mobile
* MAC OS X
* Linux
* Linux i686
* Free BSD
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## Game Consoles
* Opera/9.30 (Nintendo Wii; U; 2047-7; en)
* Mozilla/5.0 (Playstation 3; 2.00)
* PSP (PlayStation Portable); 2.00
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## Mobile User Agents
* Usually self-explanatory
* Iphone
* Ipad
* Blackberry
* Android
* Mobile user agents also usually give you the phone model (Read: IMEI correlation opportunities)
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## Ever wonder what that was?
* Gecko: a rendering engine used by Firefox and others
* AppleWebkit: Apple's version of KHTML rendering engine used in Safari and Chrome most commonly
* Presto : the “core” of the Opera platform suite
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## Ever Wonder Contd.
* .NET CLR is the .NET Framework version
* SV1 is an artifact created by MSIE 6.0 to make its security better
* Win64 can indicate that the system is running a 64 bit processor
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## Ever Wonder... One more
Many web browsers will also have an “encryption strength” marker
U = USA (128 bit encryption)
I = International (40 bit encryption)
N = No encryption (Woo!)
Most Browsers nowadays come with a U
Since the USG no longer requires encryption changes for international usage.
TOP SECRET//COMINT//REL TO USA, FVEY
--
TOP SECRET//COMINT//REL TO USA, FVEY
## Your target's user-agents can shorten and lengthen!
* Each website may require different information
* Longer user-agents may have various rendering engine, java versions, and language settings
* If you see a shortened or longer version of a UA close to your targets logins. Check it out! Carefully...
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## The How To...
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## Querying in Xkeyscore
```
Navigation Menu
* Classic A-M
* Alert
* BlackBerry
* Call Logs
* Category DNI
* Cellular DNI
* Cisco Passwords
* Clarent
* Crypt Scanner
* DNS
* Document Metadata
* Document Tagging
* Email Addresses
* Extracted Files
* Full Log DNI
* Gen Info
* HTTP Activity
* IKE Parser
```
```
Browser: Mozilla 5.0*
```
Remember:
Since Xkeyscore no longer supports leading wildcards you need to be specific with your User-Agent
A User-agent alone is not a strong query
Time Frame, Active IP,
Country, etc. all will help make
your query compliant
```
Browser
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.99 Safari/533.4
```
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## Uniqueness – When to use a User-Agent
```
Reports
VIEW
FILTERS
Browser
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)
Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)
Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)
Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)
Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)
Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)
Geo Info
Sort Ascending
Sort Descending
Filters
Color By
Group By
Histogram
Pivot Data
Histogram Grid
Show/Hide...
AutoFit Column Width
```
Results of an IP-based search for a target
```
Histogram Grid
Page 1 of 1
Clear Selection Export
Filter Browser
Count
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
6
Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)
6
```
Low Number of users and traffic volume as well as only 2 User-agents
Verdict: Probably reliable
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## More User = More Problems
| Browser | Count |
|-------- |-------|
| Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) | 6 |
| Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; | 1 |
| Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; | 1 |
| Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; GTB0.0; .NET CLR 2.0.50727; AskTB5.6) | 1 |
| Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; GTB6.5) | 1 |
| Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; GTB6.5; .NET CLR 2.0.50727) | 1 |
| Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; GTB6.5; InfoPath.2) | 2 |
| Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; GTB6.5; InfoPath.2; .NET CLR 2.0.50727; .NET | 1 |
| Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; InfoPath.1; .NET CLR 2.0.50727; .NET CLR 3.0. | 1 |
| Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322) | 1 |
| Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1) | 6 |
| Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322) | 1 |
| Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; FunWebProducts; AskTbPTV/5.8.0.12304) | 1 |
| Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; GTB6; .NET CLR 1.1.4322) | 1 |
| Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; IEMB3; IEMB3) | 1 |
| Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727 | 1 |
| Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; InfoPath.2; .NET CLF | 2 |
Reliability of correlating your target's browser to his web-activity plummets due to User-agent variations and the threat of multiple users.
At this point, cookie and TDI correlation are necessary. User-agents can not be used by themselves.
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## Common Sense Helps
Never assume because a User-agent is complicated that it is unique
* Example:
Mozilla 5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.3(KHTML, like Gecko) Chrome/6.0.464.0 Safari/534.3
This is the standard user agent for EVERYONE with an updated Chrome browser using Windows 7.
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## Xkeyscore Storage
Push to Pinwale or Archive Results
```
Download Session Mode: S
AGILITY-RealTime
Agility
DNI Presenter
PINWALE
PINWALE and Decrypt
Download SOTF Session
Download Session
Download D124 Session
finished
Archive Results
finished
```
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## Pinwale
Fields to add to Metadata View
```
AppProc Active User
User Agent
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) ...
```
Generally speaking, the User Agent listed belongs to the selector in the Active User column (if populated)
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## Add a User-Agent to a compliant query
Fielded Search Form
```
1. Put in the User Agent that is being search for into the CONTENT\_META field under the fielded search form.
KEYWORDS and
cs=ISO8859_1 ( ( ("Opera Mini") ) )
CONTENT_META
Opera Mini
between tags UserAgent and
```
Smart Form/Native Query
```
Native, cs=ISO8859_1 ( (content_meta=(((("Opera Mini")) between \<UserAgent and \/UserAgent))))
(content_meta=(((("Opera Mini")) between \<UserAgent and \/UserAgent )))
```
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## Not sure the specific User-Agent?
```
1. Add the following syntax to your query:
\TERM \<yourfield here (E.G. \TERM \<useragent )
2. Apply Native to the field
3. Apply the content_meta document zone to the field.
Free Form Query Smart Form
cs=ISO8859_1(((["selectorl"]))
selector1.
or
cs=ISO8859_1( ("selector2")))
selector2
or
) and
CONTENT_META:Native, cs=ISO8859_1 [[TERM \<useragent ]]
TERM \<useragent
or or and
(TS//SI//REL) This query basically ensures that a certain field exists in each result thus removing all the content not relevant to your query.
```
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## User-Agent Manipulation
* The best for last
* User-agents can completely be changed or not included by the user!
* By Firefox Plugin
* By Browser settings (Opera)
* Outside programs (TOR Button)
* These programs allow users to have a different user agent for each session!
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## Questions??
* Contact Info: [REDACTED]@nsa.ic.gov
* Website on the High-Side:
* [REDACTED]
* [REDACTED]
* [REDACTED]
* [REDACTED]
* Lots of great stuff in open source as well!
http://www-archive.mozilla.org/build/user-agents-strings.html
TOP SECRET//COMINT//REL TO USA, FVEY
---
TOP SECRET//COMINT//REL TO USA, FVEY
## Got a Tech Problem?
Have a random SIGDEV question?
Need help with a target using new tech to communicate?
Need help developing an accurate collateral description of a technology?
Want help developing Xkeyscore fingerprints for a weird target behavior?
Let us know : DL S2I61_all
TOP SECRET//COMINT//REL TO USA, FVEY