first commit
This commit is contained in:
commit
0d90363116
6510 changed files with 679146 additions and 0 deletions
68
data/pull-through-steering-group-minutes/metadata.json
Normal file
68
data/pull-through-steering-group-minutes/metadata.json
Normal file
|
|
@ -0,0 +1,68 @@
|
|||
{
|
||||
"title": "Pull Steering Group Minutes",
|
||||
"type": "REPORTED",
|
||||
"description": "A top secret STRAP1 GCHQ PullThrough Steering Group meeting record from February 2008 documenting the approval process for five surveillance and intelligence prototypes including KARMA POLICE — the mass internet browsing profiling system — alongside ORB FINDER for CNE operational relay box identification, SALTY OTTER for cross-media timing analysis, MOOSE MILK for telephone kiosk surveillance, and LIGHTWOOD for email address extraction used by JTRIG.\n",
|
||||
"long_description": "This is the minutes of the 16th PullThrough Steering Group meeting at GCHQ, dated 29 February 2008. The PTSG was the internal body responsible for evaluating analytical prototypes and progressing them through Technology Readiness Levels toward operational deployment. The document is significant primarily because it contains the earliest known internal record of KARMA POLICE being formally considered for development — years before it became the operational mass profiling system described in the September 2015 Intercept reporting.\n\nThe KARMA POLICE entry describes it in its own words as aiming \"to correlate every user visible to passive SIGINT with every website they visit,\" producing either a browsing profile for every visible internet user or a user profile for every visible website. The PTSG approved it for further development, but with a notable caveat: the legalities around collecting content rather than metadata still needed to be resolved. This is an early indication that GCHQ was aware of the legal uncertainties surrounding the programme before it was built and deployed, undercutting later characterisations of the agency's surveillance activities as operating within a clear legal framework.\n\nThe other prototypes considered at the meeting illustrate the breadth of GCHQ's analytical development agenda at the time. A cross-media timing correlation tool called SALTY OTTER was designed to link events across different communication channels — detecting, for example, that a phone call preceded a chat session — a technique with obvious applications for connecting pseudonymous online identities to known targets. A tool called ORB FINDER aimed to reduce the active footprint of computer network exploitation by passively identifying compromised third-party systems suitable for use as routing hops before any active engagement with them. MOOSE MILK applied data mining to detect suspicious patterns in UK public telephone kiosk usage. LIGHTWOOD, already operational within JTRIG, extracted email addresses from intercepted traffic with a sophistication beyond standard pattern matching, with its more sensitive capabilities left undescribed in the minutes.\n\nTaken together the document offers a rare view into the internal bureaucratic process by which GCHQ's most consequential surveillance capabilities moved from prototype to deployment — and into the moment, long before public awareness, when the agency was deciding whether and how to build a system designed to profile the entire visible internet.",
|
||||
"tags": [
|
||||
"PullThrough Steering Group",
|
||||
"PTSG",
|
||||
"JTRIG",
|
||||
"Computer Network Exploitation",
|
||||
"Technology Readiness Level",
|
||||
"TRL",
|
||||
"cross-media timing",
|
||||
"telephone surveillance",
|
||||
"emails",
|
||||
"Operational Relay Box",
|
||||
"internet profiling",
|
||||
"prototype approval",
|
||||
"analytical development",
|
||||
"passive SIGINT",
|
||||
"legal compliance",
|
||||
"content collection",
|
||||
"every website they visit",
|
||||
"every visible user on the internet"
|
||||
],
|
||||
"document": {
|
||||
"creator": "GCHQ",
|
||||
"date_created": "2008-02-29",
|
||||
"classification": [
|
||||
"TOP SECRET",
|
||||
"STRAP 1"
|
||||
],
|
||||
"dissemination": [],
|
||||
"banner": "TOP SECRET STRAP1",
|
||||
"genre": "report",
|
||||
"programs": [
|
||||
"MYOFIBRIL",
|
||||
"SALTYOTTER",
|
||||
"PRIMETIME",
|
||||
"KARMAPOLICE",
|
||||
"ORBFINDER",
|
||||
"MOOSEMILK",
|
||||
"LIGHTWOOD",
|
||||
"JTRIG",
|
||||
"MERAPEAK",
|
||||
"MONTEVISTA",
|
||||
"KED"
|
||||
],
|
||||
"countries": [],
|
||||
"people": []
|
||||
},
|
||||
"publication": [
|
||||
{
|
||||
"publisher": "The Intercept",
|
||||
"date_published": "2015-09-25",
|
||||
"title": "Profiled: From Radio to Porn, British Spies Track Web Users' Online Identities",
|
||||
"link": "https://theintercept.com/2015/09/25/gchq-radio-porn-spies-track-web-users-online-identities/",
|
||||
"reporters": [
|
||||
"Gallagher, Ryan"
|
||||
],
|
||||
"contributors": [],
|
||||
"doc_source": "https://theintercept.com/document/2015/09/25/pull-steering-group-minutes/",
|
||||
"doc_url": "https://s3.documentcloud.org/documents/2432296/pull-through-steering-group-minutes.pdf"
|
||||
}
|
||||
],
|
||||
"pdf_sha256": "1e97b4b72139c05abe25dd1588a1119b91e47e93d25fe3f103118cc158848614",
|
||||
"pages": 3
|
||||
}
|
||||
|
|
@ -0,0 +1,116 @@
|
|||
TOP SECRET STRAP1
|
||||
|
||||
THQ/1202THQ/1900/0058
|
||||
29 February 2008
|
||||
|
||||
PullThrough Steering Group Meeting #16
|
||||
|
||||
29 February 2008.
|
||||
|
||||
Distribute to: Invitees, GCWeb, SIGMOD-PU's-GCHQ-dl, T-Domain-Heads-GCHQ-dl, SigMod-BCM-Support-Team-GCHQ-dl
|
||||
|
||||
Present:
|
||||
* [REDACTED] OPD-SDHQ
|
||||
* [REDACTED] TPS
|
||||
* [REDACTED] OPD-ATS
|
||||
* [REDACTED] AR
|
||||
* [REDACTED] Head OPD-SD
|
||||
* [REDACTED] OPDTAT
|
||||
|
||||
Apologies:
|
||||
* None
|
||||
|
||||
Summary of prototypes in the process:
|
||||
|
||||
Briefly considered – MYOFIBRIL, addressed in Better Analysis bid.
|
||||
|
||||
Discussion of PTSG future
|
||||
|
||||
The future of ‘pull through' and the PTSG was discussed in the context of programmes, themes, product centres, the four box model, evolutionary development, stripy teams etc.
|
||||
|
||||
The role of evaluation and championing of AR prototypes for further development within the business was recognised by all as important for continuity, however no specific conclusions were reached with respect to the relationship between the PTSG and other organisational structures arising from business change, e.g. the CCBs that have been proposed in the context of the 4 box model transition processes.
|
||||
|
||||
(Ed note: Worthwhile for the PTSG to consider this relationship further more formally)
|
||||
|
||||
New Prototypes seeking PTSG Approval:
|
||||
|
||||
* SALTY OTTER [REDACTED]/SALTY_OTTER- submitted at TRL
|
||||
4.
|
||||
|
||||
B17 technique for finding cross-media timing patterns and incorporates the CLASP algorithm, which is considered to be similar, yet more general than the
|
||||
|
||||
1 of 3
|
||||
This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to GCHQ on [REDACTED]
|
||||
|
||||
TOP SECRET STRAP1
|
||||
|
||||
TOP SECRET STRAP1
|
||||
|
||||
THQ/1202THQ/1900/0058
|
||||
29 February 2008
|
||||
|
||||
popular PRIME TIME. looks for cross-media timing chains, e.g. a telephone call triggers a chat event.
|
||||
|
||||
ACTION: Ran out of time for discussion – to be brought back for next PSTG session.
|
||||
|
||||
* KARMA POLICE - submitted at TRL 4.
|
||||
|
||||
KARMA POLICE aims to correlate every user visible to passive SIGINT with every website they visit, hence providing either (a) a web browsing profile for every visible user on the internet, or (b) a user profile for every visible website on the internet.
|
||||
|
||||
For more details, refer to high-level overview document B/6760BA/5001/1.
|
||||
|
||||
Initial work with INOC shows high analyst usage of technique.
|
||||
|
||||
ACTION: To be progressed as a Better Analysis theme based requirement for explore.
|
||||
- noted that the legalities with respect to 'content' need to be cleared.
|
||||
|
||||
* ORB FINDER submitted at TRL 4.
|
||||
|
||||
Enables the identification of potential candidate Operational Relay Boxes (ORBs) for use in CNE's active exploitation network through passive logging, providing a richer picture of the end-system, and consequently reducing CNE's active footprint by focussing attention to those candidates that meet a stricter set of criteria.
|
||||
|
||||
ACTION : To be progressed as a CNE theme based requirement for explore.
|
||||
|
||||
* MOOSE MILK [REDACTED]/MOOSE_MILK - submitted at TRL 4.
|
||||
|
||||
A data mining algorithm devised by CTOR to detect suspicious use of telephone kiosks in the UK. MOOSE MILK is currently being developed further by the Operational Data Mining team in OPD-TAT.
|
||||
|
||||
ACTION: To be progressed as a Better Analysis Agility requirement for explore.
|
||||
- noted that the timeliness of the tool depends on feeds in near-real time, which have not been available so far.
|
||||
|
||||
* LIGHTWOOD [REDACTED]/LIGHTWOOD - submitted to allocate a TRL rating given operational usage within JTRIG.
|
||||
|
||||
Extracts act email addresses from any character stream. What differentiates LIGHTWOOD is its ability to distinguish valid email formats from those that appear to be email addresses but are not for open Internet (e.g. better@management).
|
||||
|
||||
2 of 3
|
||||
This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to GCHQ on [REDACTED]
|
||||
|
||||
TOP SECRET STRAP1
|
||||
|
||||
TOP SECRET STRAP1
|
||||
|
||||
THQ/1202THQ/1900/0058
|
||||
29 February 2008
|
||||
|
||||
The program also has a number of other email and URL detection capabilities that standard regular expression extraction rules do not have. Some of these are sensitive. Used operationally in JTRIG and to be used in MERA PEAK and MONTEVISTA.
|
||||
|
||||
Accepted by PTSG as having attained TRL7. No further PTSG action.
|
||||
|
||||
Other Business
|
||||
|
||||
KED
|
||||
|
||||
KED needs to be on the SE radar – it was not clear where the final ‘product centre' lay – [REDACTED] to talk to [REDACTED] and [REDACTED] together with [REDACTED], [REDACTED], [REDACTED] and [REDACTED].
|
||||
|
||||
There was some concern over programme allocation of resources to Mission Management tasks more generally. .
|
||||
|
||||
Date Of Next Meeting
|
||||
|
||||
9 May 2008 10:30 AB3-M2
|
||||
|
||||
[REDACTED]
|
||||
TPS IP-Activity Board SE
|
||||
|
||||
3 of 3
|
||||
This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to GCHQ on [REDACTED]
|
||||
|
||||
TOP SECRET STRAP1
|
||||
Loading…
Add table
Add a link
Reference in a new issue