first commit
This commit is contained in:
commit
0d90363116
6510 changed files with 679146 additions and 0 deletions
57
data/Milkwhite/Milkwhite.md
Normal file
57
data/Milkwhite/Milkwhite.md
Normal file
|
|
@ -0,0 +1,57 @@
|
|||
TOP SECRET STRAP1
|
||||
|
||||
9 March 2011
|
||||
DISCOVER ID 5100181
|
||||
|
||||
E. MILKWHITE Enrichment Service (MES) Programme
|
||||
|
||||
[REDACTED]
|
||||
MILKWHITE
|
||||
|
||||
In FY11/12 the MES Programme will continue to support the Home Office Communications Capabilities Directorate (CCD), whose programme is tasked with modernising the UK's domestic interception capabilities. The CCD SR10 settlement includes funding to sustain CCD mitigation projects that deliver early benefits during years 1-2 of the funding period. MES sustainment is one key component of this portfolio:
|
||||
|
||||
* External funding deferred from 2010/11 will enable GCHQ to deliver advanced analytics, mobile convergence and to scale the MES Advanced Service
|
||||
* External sustainment funding will enable GCHQ to deliver post and continuing design services (PDS/CDS); to maintain the MES-funded access and bulk events storage; and to tailor the advanced analytics capability to keep pace with short term changes in the communications environment.
|
||||
|
||||
GCHQ's separate SR10 sustainment settlement does not contain parallel funding to sustain MES access and processing.
|
||||
|
||||
Customer Diversity (collaborative working)
|
||||
The MES Standard Service has been TTO’d to the five intercepting law enforcement agencies² via the SOCA-hosted, multi-agency Internet Data Unit (iDU) and to BSS via the A8 Digint Identification Team (DIT). SOCA and HMRC also receive the MES Advanced Service via established ‘business as usual' channels. During 2011/12 the standard service will be extended to SIS and the advanced service will be tailored to meet BSS and SIS requirements.
|
||||
|
||||
Service Capacity
|
||||
During 2011/12, service capacity will grow to keep pace with increasing customer demand, up to the agreed service level agreements for each agency. The BSS Standard and SOCA/HMRC Advanced services are expected to have reached their planned capacity by 31st March 2011 and this will be sustained through SR10. As new services are established their capacity will be grown to match planned SLA's in line with actual customer demand.
|
||||
|
||||
² SOCA, HMRC, MPS (CTC), PSNI, Scottish Recording Centre.
|
||||
|
||||
This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to GCHQ on [REDACTED]
|
||||
TOP SECRET STRAP 1
|
||||
|
||||
TOP SECRET STRAP1
|
||||
9 March 2011
|
||||
DISCOVER ID 5100181
|
||||
|
||||
Service Productivity
|
||||
GCHQ will continue to monitor the productivity per request (% of identifiers enriched and diversity of enrichment) to ensure that the underlying analytic capabilities and tools keep pace with the changing communications environment.
|
||||
|
||||
Enabling Business Change
|
||||
Rapid changes in technology and target communications behaviours are driving our customers to adopt new ways of working with Internet data. Implementing these new services is enabling the community to identify and overcome many of the operational, cultural and policy challenges associated with this transformation, and GCHQ will continue to be forward leaning in supporting the CCD change agenda.
|
||||
|
||||
| Programme - Capability/RMI | Finance - post CAP (£m) | | | Outcomes | | | | |
|
||||
| :--------------------------- | :---------------------- | :-- | :-- | :-------------- | :-------------------- | :------------------- | :----------------------- | :--------------------------------- |
|
||||
| | CAP | OCE | External Funding | Improved customer reach | Increased service capacity | Better productivity of outputs | More analysts trained for core missions | Sustain |
|
||||
| MES | | | | | | | | |
|
||||
| MES - DEFERRED | 0.0 | 0.0 | 2.4 | • | • | • | • | |
|
||||
| MES - PDS RISK INFLATION | • 0.0 | 0.0 | 3.2 | | | | | • |
|
||||
| MES-POWER | 0.0 | 0.0 | 2.1 | | | | | • |
|
||||
| MES-OPS TEAM | 0.0 | 0.0 | 1.1 | | | | | • |
|
||||
| MES-SINO | 0.0 | 0.0 | 3.5 | | | | | • |
|
||||
| MES-SMO MOBILE | 0.0 | 0.0 | 0.3 | | | | | • |
|
||||
| MES-TA | 0.0 | 0.0 | 6.6 | | | | | • |
|
||||
| MILKWHITE - Governance | 0.0 | 0.0 | 1.6 | | | | | • |
|
||||
| Current Demand | 0.0 | 0.0 | 20.8 | | | | | |
|
||||
| Total CTIP Demand | 0.0 | | | | | | | |
|
||||
| Total Delegation | 0.0 | | | | | | | |
|
||||
| Manage In-year | 0.0 | | | | | | | |
|
||||
|
||||
This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to GCHQ on [REDACTED]
|
||||
TOP SECRET STRAP 1
|
||||
88
data/Milkwhite/metadata.json
Normal file
88
data/Milkwhite/metadata.json
Normal file
|
|
@ -0,0 +1,88 @@
|
|||
{
|
||||
"title": "MILKWHITE Enrichment Service (MES) Programme",
|
||||
"type": "REPORTED",
|
||||
"description": "This March 2011 GCHQ document details the MILKWHITE program, which is enhancing domestic interception and data analysis capabilities. It outlines expanded funding for advanced analytics, bulk data storage, and service scaling to meet growing demand from multiple law enforcement and intelligence agencies, including SOCA, HMRC, MI5, and SIS, thereby broadening access to large volumes of internet-derived metadata for operational use.",
|
||||
"long_description": "The March 2011 GCHQ document titled “MILKWHITE Enrichment Service (MES) Programme” provides an overview of how the United Kingdom’s intelligence and law enforcement agencies were expanding their domestic data interception and analysis capabilities through the MILKWHITE program. The programme was part of a broader initiative led by the Home Office’s Communications Capabilities Directorate (CCD), which aimed to modernize and strengthen the U.K.’s ability to monitor internet and communications data. The document indicates that the CCD’s settlement for the 2010–2011 financial year included funding specifically to sustain early projects that yielded “mitigation” benefits—essentially, capabilities that helped fill gaps in digital surveillance infrastructure. Through this funding, GCHQ was tasked with delivering enhanced analytic tools, scaling its bulk data storage and access systems, and maintaining the agility to adapt to rapid technological changes in global communications networks.\n\nA major focus of the report is collaborative access to MES services by various government entities. The document notes that GCHQ had extended the “standard service” of MES to the five major U.K. intercepting law enforcement agencies—including SOCA (Serious Organised Crime Agency), HMRC (Her Majesty’s Revenue and Customs), the Metropolitan Police Counter-Terrorism Command (CTC), the Police Service of Northern Ireland (PSNI), and the Scottish Recording Centre—through a multi-agency Internet Data Unit (iDU) hosted by SOCA. The “advanced service,” which offered deeper analytic capabilities and broader metadata enrichment functions, was also being provided to HMRC and SOCA through established “business as usual” channels. During 2011–2012, the standard service, already provided to the BSS (British Security Service, or MI5), was to be extended to SIS (Secret Intelligence Service), with the advanced service tailored to BSS and SIS requirements, demonstrating how GCHQ’s centralized data services were becoming a shared intelligence backbone across the U.K.’s surveillance ecosystem.\n\nFinally, the document details GCHQ’s ongoing efforts to increase service capacity, productivity, and efficiency. It includes a financial and outcomes table showing millions of pounds allocated to specific MES subprograms, such as advanced analytics, mobile data convergence, and bulk event storage. GCHQ emphasized the importance of monitoring productivity per data request—measured by the percentage of identifiers successfully enriched—to ensure the system kept pace with the evolving communications landscape. It also acknowledged that rapid shifts in technology and target behavior required “new ways of working with internet data” and noted that the MES framework was helping the intelligence community confront the operational, cultural, and policy challenges of the digital era. Overall, the report portrays MILKWHITE as a critical, large-scale metadata enrichment and analytics system underpinning the U.K.’s surveillance architecture, designed to expand inter-agency intelligence sharing and sustain the nation’s digital interception capabilities well into the following decade.",
|
||||
"tags": [
|
||||
"metadata",
|
||||
"metadata enrichment",
|
||||
"digital surveillance",
|
||||
"bulk collection",
|
||||
"internet interception",
|
||||
"SR10 settlement",
|
||||
"intelligence modernization",
|
||||
"UK surveillance",
|
||||
"domestic surveillance",
|
||||
"metadata analysis",
|
||||
"data analysis",
|
||||
"advanced analytics",
|
||||
"data storage",
|
||||
"metadata storage",
|
||||
"internet surveillance",
|
||||
"metadata exploitation",
|
||||
"law enforcement cooperation",
|
||||
"SOCA",
|
||||
"HMRC",
|
||||
"Her Majesty’s Revenue and Customs",
|
||||
"Metropolitan Police",
|
||||
"Counter Terrorism Command",
|
||||
"CTC",
|
||||
"PSNI",
|
||||
"Police Service of Northern Ireland",
|
||||
"Scottish Recording Centre",
|
||||
"SIS",
|
||||
"MI5",
|
||||
"BSS",
|
||||
"British Security Service",
|
||||
"intelligence sharing",
|
||||
"interagency collaboration",
|
||||
"surveillance infrastructure",
|
||||
"UK intelligence community",
|
||||
"metadata processing",
|
||||
"surveillance network",
|
||||
"communications monitoring",
|
||||
"domestic intelligence",
|
||||
"operational planning",
|
||||
"enrichment",
|
||||
"mass surveillance",
|
||||
"intelligence funding",
|
||||
"government oversight",
|
||||
"digital communications environment",
|
||||
"signal processing",
|
||||
"target identification"
|
||||
],
|
||||
"document": {
|
||||
"creator": "GCHQ",
|
||||
"date_created": "2011-03-09",
|
||||
"classification": [
|
||||
"TOP SECRET",
|
||||
"STRAP 1"
|
||||
],
|
||||
"dissemination": [],
|
||||
"banner": "TOP SECRET STRAP1",
|
||||
"genre": "report",
|
||||
"programs": [
|
||||
"MILKWHITE",
|
||||
"MES",
|
||||
"SINO"
|
||||
],
|
||||
"countries": [],
|
||||
"people": []
|
||||
},
|
||||
"publication": [
|
||||
{
|
||||
"publisher": "The Intercept",
|
||||
"date_published": "2016-06-07",
|
||||
"title": "Facing Data Deluge, Secret U.K. Spying Report Warned of Intelligence Failure",
|
||||
"link": "https://theintercept.com/2016/06/07/mi5-gchq-digint-surveillance-data-deluge/",
|
||||
"reporters": [
|
||||
"Gallagher, Ryan"
|
||||
],
|
||||
"contributors": [],
|
||||
"doc_source": "https://theintercept.com/document/2016/06/07/milkwhite/",
|
||||
"doc_url": "https://s3.documentcloud.org/documents/2852971/Milkwhite.pdf"
|
||||
}
|
||||
],
|
||||
"pdf_sha256": "860c99a5b599542beee10bbf89316f9e253908f05c1f9a32235b04e650dc2d71",
|
||||
"pages": 2
|
||||
}
|
||||
Loading…
Add table
Add a link
Reference in a new issue