snowchive-meta/data/finding-and-querying-document-metadata/finding-and-querying-document-metadata.md
2026-11-19 04:39:11 +02:00

724 lines
No EOL
16 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20291123
# KEYSCORE
# Finding and Querying on Document Metadata
Booz Allen Hamilton
Sigint Development Support / SIGINT Technical Analysis (SDS/STA)
April 2009
Derived From: NSA/CSSM 1-52
Dated: 20070108
Declassify On: 20291123
DERIVED FROM: NSA/CSSM 1-52
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20291123
08
---
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
## Document Metadata Agenda
* Why to Query on Document Metadata
* How to Find Document Metadata
* e.g. File -> Properties
* Google
* How to Create Queries in XKS
* XKEYSCORE Document Metadata and PDF Metadata
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
---
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
## Document Metadata Analysis
* What?: Use non-traditional selectors to find and track targets sending/receiving documents of interest
* How? It targets documents by Author, Organization, or embedded images (logos)
* Why? We don't always know WHO is sending the documents, but they are "guilty-by-association" if they send/receive the document. So, who are THEY?
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
---
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
## Finding Document Metadata
* We find "Document Metadata" in File Properties
```
XKEYSCORE_Terms.doc - Microsoft Word TO
File Edit View Insert Format Tools Table Window Help SecretAgent Classif
New...
Open...
Close
Save
Save As...
Save as Web Page...
Remove Hidden Data...
File Search...
Versions...
Web Page Preview
Page Setup...
Print Preview
Print...
Send to
Properties
1 U:\private\andrew.doc
2 U:\...\XKEYSCORE Tips and Tricks 4 ap...
3 U:\private\Présentation.doc
4 U:\private\...\NIA Cross Training.doc
5 U:\private\NIA/MSRs\April 2009.doc
6 C:\Documents and Settings\...\IMEI.doc
7 U:\...\NIA\RITCHIE_DNI_IWPS(revised).doc
8 U:\...\XKS_kmkeith_tips7April.doc
9 U:\...\Zwakenberg, Garrit Trey IPWS.doc
Exit
STM4, STM16, STM64, multip
s.pdf
XKEYSCORE_Terms.doc Properties
General | Summary | Statistics | Contents | Custom
Title: XKeyscore Terms
Subject:
Author: Joe BaggaDonuts
Manager:
Company: ZMFA Zendian MFA
Category:
Keywords:
Comments:
Hyperlink base:
Template: Normal.dot
Save preview picture
OK Cancel
If unique, these Document Properties can be targeted
```
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
---
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
## Document Metadata Analysis
* How do you find document metadata?
* Passive Collection: Collected Documents already contain data
* Active Collection: CNE "Categorized Collection" from TUNINGFORK Data or Pinwale Queries on "US-3101"
* Open Source: Google Hacking
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
---
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
## Finding Document Metadata
| ID | To | From | CO BC Date | Subject | Size(K) | Type |
|---|---|---|---|---|---|---|
| 48 | [REDACTED]o.com" <[REDACTED]com> | "DESTOCKPRO" <des[REDACTED]> | 5/15/2008 6:31:35 PM | ARRIVAGE G STAR DOLCE&GABBANA DIESEL | 16 | text/html |
| 49 | [REDACTED]yahoo.com" [REDACTED]noo.com> | "Target Center" <[REDACTED]> | 5/16/2037 9:36:13 PM | Confirmation: Target Card | 2 | text/html |
| 51 | | [REDACTED]75[REDACTED]> | 5/15/2008 11:14:32 PM | | 1 | text/html |
| 41 | [REDACTED]<azedi[REDACTED]il.com>, "abu zubeer alyamani" | "khalid" <o[REDACTED]> | 5/15/2008 3:05:25 PM | Las Villas de Dubai | 3898 | application/octet-stre |
| 32 | [REDACTED]<ke[REDACTED] | [REDACTED] | 5/15/2008 11:15:21 PM | skriv ut | 452 | application/msword |
| 51 | | [REDACTED] | 5/15/2008 11:14:32 PM | | 452 | application/msword |
| | | | | | 50 | application/msword |
```
3898 application/octet-strear bas
452 application/msword bas
50 application/msword bas
| Display original | Raw | SMTP header | Properties | Control | C2C | Trailer | Collected Doc | Search Kwd |
Dictionary Kwd | SRI |
Document Properties
Category:
Company:
HiddenSlideCount: 0
LineCount: 29
LinksUpToDate: False
Manager:
MMClipCount: 29
NoteCount: 29
ParagraphCount: 8
PresentationTarget:
ScaleCrop: False
SlideCount: 8
AppName: Microsoft Word 10.0
Author: GoGo
CharacterCount: 3582
Comments:
DateCreated: 5/12/2008 3:13:00 AM
SecurityLevel: none
Keywords:
LastAuthor: [REDACTED]
```
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
---
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
## Finding Document Metadata
* Active Collection: CNE “Categorized Collection” from TUNINGFORK Data
```
TUNINGFORK
Collection
No EP user information found.
Raw Project Details[s3115 only]
Mailbox Collection
Last Collection [limit 3 dates listed]: 2008-08-29 2008-08-27 2008-07-19
List All
Categorized Collection
Cipher (8) | MicroSoft (277) | Multimedia (17) | Mail (35) | Inst Msgr (9) | VOIP (1642) | HTM
Cipher
Access (0)
Excel (2)
Execs (4)
Ini files (2)
Other Office (5)
Powerpoint (0)
Thumbs.db (12)
Word (252)
```
| Show Pat | Collected | Size |
|----------------------------|------------|------|
| 81e0bd 21-4a68af648ec5 | 2008-07-19 | 388 |
| b850ea 1d-3dffb4d38926 | 2008-07-19 | 388 |
| 0c6527 80-c1ed1756266f | 2008-07-13 | 388 |
To find Document Metadata in TUNINGFORK, you must view each Document in Categorized Collection (manual intensive)
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
---
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
## Finding Document Metadata
* Using XKEYSCORE to query on CNE data
```
Fields | Advanced Features | Show Hidden Search Fields | Clear Search Values | Reload Last Search Values
Search: Document Metadata
Input Source: FOXACID\* or FOXBASE*
[x] Selected implant exfils from active collection (xks-cne.corp.nsa.ic.gov:xs_web_db)
```
This query in XKS
Produced these results
| Filename | Extension | Author | Title | Input Source |
|---------------------------------------------|-----------|-----------------|-------|----------------|
| \System Admin CV.doc | doc | Authorised User | | [REDACTED]E4 |
| C:\Downloads\Physical_Layer_in_RPR_0402.pdf | pdf | | | [REDACTED]ER12 |
| C:\Documents and Settings\Guest\Desktop\05070807_excelbook.pdf | pdf | Center For Excellence | (Microsoft Word - \307\341\343\336\3) | |
| C:\Documents and Settings\user\Desktop\desktop icons\yementrack.doc | doc | user | | [REDACTED]DER12 |
| C:\Downloads\ins-3-overview.pdf | pdf | | | [REDACTED]DER12 |
| C:\Documents and Settings\user\Desktop\desktop icons\servers_exp | doc | results | الأخ مدير عام المشتريات والمخازن المحترم | [REDACTED]R12 |
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
---
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
## Finding Document Metadata
* Open Source: Google Hacking
```
Google
Advanced Search
Search Tip
Help
site:comsats.net.pk filetype:doc Google Search
Or....
filetype:xls filetype:pdf
```
* Search by domains
* "site:comsats.net.pk"
* Search by file types
* "filetype:pdf" or "filetype:doc"
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
---
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
## Document Metadata Analysis
How to find Document Metadata when you have NEVER collected a document
```
Employee List
```
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
---
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
## Document Metadata Analysis
Take Client’s (Active User) IP address and query on it in XKEYSCORE
```
Active User:
[REDACTED]@yahoo.com
```
| ACTIVE USER | ACTIVE_USER_IP |
| :-----------------| :--------------|
| [REDACTED]<yahoo> | 89[REDACTED] |
| [REDACTED]<yahoo> | 89[REDACTED] |
```
Search: Document Metadata
Extension: ppt or doc or pdf or xls
IP Address: 89[REDACTED] Either
```
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
---
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
## Targeting Document Metadata
* Use XKEYSCORE to Find Who Else is sending the files?
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
---
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
## Document Metadata Analysis
Take "File Properties" information and fill-in query
```
XKEYSCORE_Terms.doc Properties
General | Summary | Statistics | Contents | Custom |
Title: XKeyscore Terms
Subject:
Author: Joe BaggaDonuts
Manager:
Company: ZMFA Zendian MFA
Category:
Keywords:
Comments:
Hyperlink base:
Template: Normal.dot
Save preview picture
OK Cancel
Search: Document Metadata
Document Type:
Encrypted?:
Corrupted?:
Filename:
Extension:
*Subject*:
*Creation Time*:
*Last Modified Time*:
*Unique ID* [fulltext]:
Author: Joe BaggaDonuts
Last Author:
Organization: ZMFA Zendian MFA
Title:
Language:
*Comment* [fulltext]:
File/Embedded Image Hash [fulltext]:
Metadata Name:
Metadata Value [fulltext]:
```
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
---
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
## Document Metadata Analysis
Sample Query
```
Sample Query:
Organization = PTCL
To/From Country = Pakistan
Search: Document Metadata
Organization: PTCL
Title:
Language:
Comment [fulltext]:
File/Embedded Image Hash [fulltext]:
Metadata Name:
Metadata Value [fulltext]:
IP Address: From
IP Address: To
Port: From
Port: To
Country: PK Either
```
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
---
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
## Document Metadata Analysis
Sample Query (Results)
Previous Slide produces these results
| Filename | Organization |
|---|---|
| Instructions to Kunar province bidders community midwifery.doc | PTCL |
| Instructions to Kunar province bidders community midwifery.doc | PTCL |
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
---
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
## Embedded Images
* Turn a logo into a selector
```
YemenNet
The Gateway of Yemen
= SIGINT VALUE
```
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
---
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
## Embedded Images
* XKEYSCORE parses out logos from within documents (PDFs, DOCs, Outlook Emails, etc) embedded as images
```
Logo/Image 32-character hash can be parsed out and queried.
YemenNet
The Gateway of Yemen
image-6b71557c23ce00a03a662d6dd1eb1c1b.jpeg
1 / 2 80.6% Find
المحترم
الخازن
الموضوع : طلب مراسلة للشركات المقدمة في مناقصة توسعة سرفرات خدمات الانترنت
بالاشارة الى الموضوع أعلاه نرجوا مخاطبة الشركات المذكورة بالملاحظات الخاصة بكل منها
الصنف |اسم الشركة الملاحظات
MDS لم تذكر : MB L2 On Chip Cache per Processor 2+ System Controller Card 1+ Solaris 10 03/05 HW1 Operating System Preinstalled.
Sun Fire V490 Server
```
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
---
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
## Embedded Images
Files often contain embedded images, such as company logos.
```
Step 1: Identify if a document HAS an image in it
تطورات خدمات الانترنت بالاشاره الى الموضوع اعلاه برده
YemenNet
النواحي التوجيهيه إلى من يلزم تفعيل خدمة ADSL الأخ
بال ١٥٢٥٦ الادارية و آآ را ت ا ا و دارات ١٥٢٥
VPN Clients Configuration Example I
لسنوي لادارة الانترنت للعام 2007 12/1/2008 8
YemenNet
by ITE. 316) & Abdullah Mohammed
```
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
---
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
## Embedded Images
| Datetime | Case Notation | From IP |
| :------------------ | :-------------- | :------ |
| 2009-03-26 15:54:50 | YM.PGQXXXABDDTC | |
```
Step 2: Open Document and click on "Full Session"
Session | Header (3) | Attachments (6) | Meta (3)
Formatter: AUTO
Send to: Download Session | Mode: Full Session | Options |
Quick Clicks
* Session
* Attachments
* sigint
* image_summary_mont
* image_summary_m
* document_meta
* c:_documents and s
* unknown
* text
* document_body.رسید
* document_body.ی (ره
* image
* jpeg
* b3d7853e4bfde7087
* office
* pdf
* C:\Documents and S
* One-Click Searches
* Find opposite side of sessi
* :0->
* :0
* Find More Docs with Same
* 635ed0657cfe25b7790f
* b3d7853e4bfde70874cf
Retrieving Attachment...
```
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
---
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
## Embedded Images
```
Session | Header (3) | Attachments (6) | Meta (3)
Formatter: AUTO
Send to: Download Session | Mode: Full Session | Options |
Quick Clicks
* Session
* Attachments
* sigint
* image_summary_mont
* image_summary_m
* document_meta
* c:_documents and s
* unknown
* text
* document_body.رسید
* document_body.ی (ره
* image
* jpeg
* b3d7853e4bfde7087
* office
* pdf
* C:\Documents and S
* One-Click Searches
* Find opposite side of sessi
* :0->
* :0
* Find More Docs with Same
* 635ed0657cfe25b77901
* b3d7853e4bfde70874cf
b3d7853e4bfde70874cf402a3d6cfe10.jpg
Virus scan results
Using IMAGE format
b3d7853e4bfde70874cf402a3d6cfe10.jpg
G.T.I
General Telecommunication Institute" and المعهد العام للاتصالات
```
Step 3: In left-side menu bar, select an image and copy/paste the 32-character name (without the extension)
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
---
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
## Embedded Images
Step 4: Paste the 32-character name into the "File/Embedded Image Hash" Field in the Document Metadata query
```
Fields | Advanced Features | Show Hidden Search Fields | Clear Search Values | Reload Last Search Values
Search: Document Metadata
File/Embedded Image Hash [fulltext]: b3d7853e4bfde70874cf402a3d6cfe10
Step 5: Select all of your good collection sites + SUBMIT!
* Classic A-M
* ASF and WMV Metadata
* Alert
* BlackBerry
* CNE
* Call Logs
* Category DNI
* Cellular DNI
* Cisco Passwords
* Document Metadata
* Document Tagging
Search Databases
Clear Checks
Reset Checks
* [x] (xks-central.corp.nsa.ic.gov:qsummary)
* [x] Australian sites (xkcentral2.dsd:xs_web_db)
* [x] CARBOY (carboy-proxy.r1.r.nsa:carboy_web_db)
* [x] CARDAMON (xkey-dsd.r1.r.nsa:xs_web_db)
Submit Cancel
```
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
---
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
## Embedded Images
Or... You can one-click query to create a new query
```
Session | Header (3) | Attachments (6) | Meta (4)
Formatter: AUTO
Send to: Down
Quick Clicks
* image_summary_montage.jpeg
* document_meta
* c:_documents and settings_usuari
* unknown
* text
* ?document_body.SOLICITANTE .txt
* office
* word
* C:\Documents and Settings\usuari
* One-Click Searches
* Find opposite side of session
* :0->
* :0
* Find More Docs with Same hash
* a97d82d06aaa9017cacbe5fe4b12f15c
* f4c6353ebd01ba02b7c087a91bdf29c4
Banco Nacional de Vivienda y Hábitat
BANAVIH
a533e3a6e83d50ea629ba899f9b9091
a99d3f1acf45e5f466d6ed99e484d877
* Find email address
* zakimoussa@hotmail.com
Search: Document Metadata
Query Name: One-click search on document hash: f4c6353
Justification: One-click search to find more documents with
Additional Justification:
Miranda Number:
Language:
\*Comment* [fulltext]:
File/Embedded Image Hash [fulltext]: f4c6353ebd01ba02b7c087a91bdf29c4
```
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
---
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
## Embedded Images
* Stand-alone files can be uploaded into XKS and images parsed out
* Useful for TAO collection that didn't get into XKS (non United Rake)
* https://xks-central.corp.nsa.ic.gov/general/view_file.php
Employee List
```
This system is aud
CLASSIFICATION: TO
XKEYS
You can upload SOTF and D-124 files, as well as just random files (.doc, .ppt, etc.)
Upload File Browse...
Upload
This system is aud
CLASSIFICATION: TO
```
* To task the hex values for images in CADENCE or Query in PINWALE, contact The Xtreme Target Pursuit Team [REDACTED] S2I7 and [REDACTED] S3114
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
---
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
## Embedded Images
* Questions on any of these tools or techniques, contact:
* [REDACTED]
* [REDACTED]
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL