{ "title": "SSO GHOSTMACHINE Analytics", "type": "REPORTED", "description": "This document describes the NSA’s cloud analytics platform GHOSTMACHINE, used by Special Source Operations (SSO) to process and analyze massive volumes of data (e.g., from FASCIA) using Hadoop-based infrastructure. It’s essentially the backbone system for querying, correlating, and enriching surveillance data at scale — not just a single analytic, but the analytic/cloud environment powering many programs.", "long_description": "This top secret NSA Special Source Operations presentation introduces GHOSTMACHINE, the agency's cloud analytics platform built on Hadoop distributed computing infrastructure, designed to overcome the limitations of traditional corporate databases by enabling analysts to query and correlate terabytes of metadata at scale. The platform aggregates data from multiple SSO collection feeds including FASCIA (metadata repository), TUSKATTIRE (call data record normalization), and other sources identified by codenames including CERF, POPTOP, DRT, SIGDASYS, AST128, JUGGERNAUT, and SEADIVER. A chart titled \"Hitting The Wall\" displays TUSKATTIRE volume totals from 9 December 2010 to 5 January 2011, with total daily record counts approaching and potentially exceeding 5 billion — suggesting the platform was encountering infrastructure scaling limits at the time of the presentation.\n\nThe document's \"Data at a Larger Scale\" slide illustrates GHOSTMACHINE's Hadoop Distributed File System (HDFS) architecture, showing ASDF-format metadata and FASCIA data stored in distributed nodes, and emphasizes that the platform's real value lies in the ability to correlate and enrich data across multiple collection types simultaneously — a capability not available in traditional corporate database environments. A SIGAD summary table lists active collection sources feeding the platform; per other documents in the Snowden archive, US-3180 (SPINNERET, a RAMPART-A site) contributes the largest number of OPC/DPC pairs at 2,022, followed by US-3237 (SMOKYSINK, another RAMPART-A site) at 1,816 and US-990 (FAIRVIEW) at 860. The interface screenshot reveals associated SSO tools including BROKENRECORD, REDHAWK, a DNR Deduplicator, and Mission Support Hub — suggesting GHOSTMACHINE operates within a broader SSO analytics ecosystem rather than as a standalone platform.", "tags": [ "cloud analytics", "cloud computing", "big data", "data storage", "data deduplication", "data correlation", "telecommunications", "metadata processing", "Hadoop", "data infrastructure", "AST 128", "SMA Tools", "web operations center", "SSO Dashboard", "DNR Deduplicator", "mission support", "SSO", "HDFS", "distributed computing", "SIGDASYS", "metadata", "bulk collection", "OPC", "DPC", "call detail records", "infrastructure", "analytic tools" ], "document": { "creator": "NSA", "date_created": "", "classification": [ "TOP SECRET", "COMINT" ], "dissemination": [ "NOFORN" ], "banner": "TOP SECRET//COMINT//NOFORN", "genre": "slides", "programs": [ "GHOSTMACHINE", "FASCIA", "CERF", "POPTOP", "DRT", "SIGDASYS", "JUGGERNAUT", "SEADIVER", "AST128", "TUSKATTIRE", "ASDF", "BROKENRECORD", "REDHAWK", "SSO" ], "countries": [], "people": [] }, "publication": [ { "publisher": "The Washington Post", "date_published": "2013-12-10", "title": "New documents show how the NSA infers relationships based on mobile location data", "link": "https://www.washingtonpost.com/news/the-switch/wp/2013/12/10/new-documents-show-how-the-nsa-infers-relationships-based-on-mobile-location-data/", "reporters": [ "Soltani, Ashkan", "Gellman, Barton" ], "contributors": [] } ], "aka_id": [ { "id": "nsa-ghost-machine" }, { "id": "ACLU_-_2013-12-10_Ghost_Machine" }, { "id": "2013-12-10_Ghost_Machine" } ], "pdf_sha256": "d178b4df88d6aa2d9ed57283a89fb89e80858b3fd96af4733be1b2930ec1f2f2", "pages": 4 }