{ "title": "SKYNET: Courier Detection via Machine Learning", "type": "REPORTED", "description": "A June 2012 NSA research presentation detailing the machine learning methodology behind SKYNET's courier detection system, which applies a Random Forest classifier to 55 million Pakistani phone records to identify individuals exhibiting courier-like behavioral patterns — with one of the highest-scoring results being Ahmed Zaidan, Al Jazeera journalist flagged as a suspected al-Qaeda courier.", "long_description": "This top secret NSA research presentation, dated June 5, 2012 and produced collaboratively by analysts from S2I5, R66F, and T1211, describes the technical methodology underlying SKYNET's automated courier detection capability. The core research question is whether a machine learning classifier trained on a small set of known al-Qaeda Senior Leadership (AQSL) courier selectors can identify behaviorally similar individuals from a population of approximately 55 million Pakistani phone subscribers — without relying on call chaining from known targets. The system extracts up to 80 behavioral features from GSM metadata including unique cell tower visit counts, travel frequency, incoming-to-outgoing call ratios, SIM and handset swapping behavior, and social network characteristics, then uses these features to distinguish couriers from the general Pakistani population.\n\nThe presentation documents a progression of classifier performance improvements. An initial centroid-based detector achieved an equal error rate of 10-20% in a cross-validation experiment on seven known courier selector pairs. Switching to a Random Forest classifier reduced the false alarm rate to 0.18% at 50% miss rate — a sevenfold improvement over random performance. Adding training data scraped from Anchory intelligence reports containing the keyword \"courier\" further reduced the false alarm rate to 0.008% (at a 50% miss rate) and gave a 46-fold improvement over random in tasked precision, with 21 of the top 500 scoring selectors being already-tasked known targets. The system was then run via MapReduce against the full 55 million selector dataset, with results assessed both quantitatively and through manual SIGINT review.\n\nThe presentation's most significant and controversial disclosure is the identification of Ahmed Zaidan — described in the document as a member of al-Qaeda, a member of the Muslim Brotherhood, and a journalist working for Al Jazeera — as the highest-scoring selector among those who traveled between Peshawar and Lahore. Zaidan was at the time Al Jazeera's Islamabad bureau chief, and his travel patterns to conflict zones as a journalist were apparently sufficient for SKYNET's algorithm to flag him as a probable courier. The document does not indicate any corroborating evidence beyond behavioral metadata, raising significant questions about the reliability of algorithmic targeting against a population of 55 million people at a 0.008% false alarm rate — which would still produce thousands of false positives at population scale.", "tags": [ "journalism", "Al Jazeera", "Al-Qaeda", "machine learning", "courier tracking", "behavioral profiling", "pattern-of-life", "GSM", "call detail records", "AQSL", "courier network", "journalist surveillance", "IMSI", "MSISDN", "cell towers", "travel analytics", "social network analysis", "Anchory", "targeting" ], "document": { "creator": "NSA", "date_created": "2012-06-02", "classification": [ "TOP SECRET", "COMINT" ], "dissemination": [ "REL TO USA, FVEY" ], "banner": "TOP SECRET//COMINT//REL TO USA, FVEY", "genre": "slides", "programs": [ "SMARTTRACKER", "MAPREDUCE", "SKYNET" ], "countries": [ "Pakistan", "Afghanistan" ], "people": [ "Zaidan, Ahmad Muaffaq" ] }, "publication": [ { "publisher": "The Intercept", "date_published": "2015-05-08", "title": "U.S. Government Designated Prominent Al Jazeera Journalist as \"Member of Al Qaeda\"", "link": "https://theintercept.com/2015/05/08/u-s-government-designated-prominent-al-jazeera-journalist-al-qaeda-member-put-watch-list/", "reporters": [ "Currier, Cora", "Greenwald, Glenn", "Fishman, Andrew" ], "contributors": [], "doc_source": "https://theintercept.com/document/2015/05/08/skynet-courier/", "doc_url": "https://s3.documentcloud.org/documents/2074620/skynet-courier-detection-via-machine-learning.pdf" } ], "pdf_sha256": "d64b2a68973e894fb1564d7ee8b691362a97f8812508341464d3d2ded20fa34c", "pages": 20 }