{ "title": "CSEC ITS/N2E Cyber Threat Discovery", "type": "REPORTED", "description": "A CSEC IT Security presentation to the DISCOCON 2010 conference, given with speakers' notes, on how the agency's cyber threat discovery team hunts for intrusions in the traffic it collects from Government of Canada networks. It describes the PonyExpress email processing system, which the notes record as handling 400,000 emails a day, the 8Ball attachment scanner and the Stripsearch binary analyser, and reports experiments in reducing the volume of metadata analysts see — including a machine learning model that discards 85% of email before deep scanning.", "long_description": "CSEC's IT Security branch ran its computer network defence operations out of a group called N2, with incident response at the core. N2E was split off from it to do discovery rather than firefighting — “hunting vs. firefighting: new threats, techniques, tradecraft” — and this deck is that team reporting on its first year to DISCOCON 2010, subtitled “A CND Perspective”. The team was growing to seven people and reported “excellent access to full take data”, with policy work under way on the use of intercepted private communications.\n\nThe haystack is described in the deck and quantified in the notes. CSEC had three client departments producing tens to low thousands of signature-based alerts a day, mostly false positives, and was retaining single to tens of terabytes of passively tapped traffic daily plus tens to hundreds of gigabytes of metadata. An “official” Internet aggregation point for gc.ca was coming: the notes give its capacity as 400TB a month, starting at 150TB across 68 departments and expected to reach more than a hundred.\n\nThe tooling is named throughout. PonyExpress rebuilds SMTP sessions from full packet captures, extracts attachments and feeds them to a cluster of 8Ball deep-scan services; the notes record it processing 400,000 emails a day — all of which had already passed commercial anti-spam and anti-virus filtering — promoting 400 to the alert system, of which about 1% were judged worth reporting to the client. Stripsearch runs binaries recursively through an analysis chain, clearing a repository of roughly 1,500 malware samples in ninety minutes. Popquiz and Slipstream produce the network metadata, and a tool called findmask hunts XORed strings: the slide is entirely blacked out, but its notes survive and explain that it looks for Windows API calls and registry-key fragments, with seven to ten unique known strings in one flow as the threshold worth investigating.\n\nAmong the catches: a WMI-based implant that persists through ActiveScriptEventConsumer rather than the registry or boot sector, first seen in September 2009 and observed growing more sophisticated — initially creating its WMI event through a script run by CSCRIPT.EXE, later directly from the binary. Malware in the TROPPUSNU family used Google IP addresses as a sleep command, so searching DNS metadata for Google IPs surfaced new domains and infected workstations. The team also found Tor running inside a government network, assessed as “most likely a breach of Internet usage policy”, and worked out how to detect it from the two-hour validity window on its self-signed certificates — the deck prints the working ngrep hex string and the notes record refining it down to a single false positive.\n\nThe second half reports results from SAWUNEH, explained on slide 17 as “Summer Analytic Workshop Up North EH”: an annual data analysis workshop hosted at CSEC by its crypt-math shop, with 2010's theme being computer network defence and representatives from all Five Eyes agencies plus cleared outside researchers, working on a Netezza appliance and a Cray XMT. Correlating email, web and flow metadata so that a URL is only shown to an analyst when the mail was inbound, the link was flagged suspicious and a recipient actually visited it produced results of which 50% were malicious. A Random Forest classifier trained on more than 10,000 previously scanned samples predicted which emails would score in the deep scan from metadata available early in processing, giving 85% data reduction for 1-3% loss of interesting email. The closing slide draws the lesson: “Our days of plain old grep as primary search tool are nearing an end.”\n\nThe file holds the deck twice: slides 1-30, then the same thirty slides again as presenter's notes pages, each with the slide reduced at the top and the speaker's notes printed beneath it.", "tags": [ "Computer Network Defence", "Government of Canada", "email interception", "metadata", "network metadata", "passive collection", "full take", "packet capture", "malware", "implants", "malware analysis", "threat discovery", "anomaly detection", "machine learning", "data reduction", "Tor", "DNS", "SMTP", "HTTP", "attachment scanning", "incident response", "Five Eyes", "retention", "SIGDEV", "email surveillance", "insider conduct" ], "document": { "creator": "CSE", "date_created": "2010-10-26", "classification": [ "TOP SECRET" ], "dissemination": [], "banner": "TOP SECRET", "genre": "slides", "programs": [ "PONYEXPRESS", "8BALL", "STRIPSEARCH", "POPQUIZ", "SLIPSTREAM", "FINDMASK", "SAWUNEH", "TROPPUSNU", "N2E", "DISCOCON" ], "countries": [ "Canada" ], "people": [] }, "publication": [ { "publisher": "CBC News", "date_published": "2015-02-25", "title": "CSE monitors millions of Canadian emails to government", "link": "http://www.cbc.ca/news/canada/cse-monitors-millions-of-canadian-emails-to-government-1.2969687", "reporters": [ "Seglins, Dave" ], "contributors": [ "Hildebrandt, Amber", "Pereira, Michael", "Greenwald, Glenn", "Gallagher, Ryan" ], "doc_url": "https://s3.documentcloud.org/documents/1676153/csec-its-dsco-2010-20101026-final.pdf" }, { "publisher": "The Intercept", "date_published": "2015-02-25", "title": "Canadian Spies Collect Domestic Emails in Secret Security Sweep", "link": "https://theintercept.com/2015/02/25/canada-cse-pony-express-email-surveillance/", "reporters": [ "Gallagher, Ryan", "Greenwald, Glenn" ], "contributors": [], "doc_url": "https://s3.documentcloud.org/documents/1676153/csec-its-dsco-2010-20101026-final.pdf" } ], "aka_id": [], "pdf_sha256": "c70502094c8635bf7679bccb35e937d0b05aab3d201f227b2c2ed8a876d8b946", "pages": 60 }