PSQR: Data Structures for Virtual Public Squares

Introduction

Virtual Public Squares operate on globally unique identities for participants, globally unique content addressing for articles, cryptographic signatures to ensure provenance and provide tamper-resistance for each individual article of content, and consistent data structures for articles of content across representations of that content in search indexes, feeds, lists, bittorrent metdata and any other form an article or articles might take. Each article contains provenance within it, and can be replicated from aggregator to aggregator, archived, served via https or another protocol, or transported as simple files.

The data structures are expressed in JSON. Single articles are stored in the same structure, and serialization for edge delivery is presently in JSON, with collections (feeds, lists) serialized in JSONL.

The data structure extends BitTorrent v1 metadata with social media content presentation elements, publisher identity, provenance via cryptographic signature, pointers to link articles of content to each other, geonum to specify a geographic location with arbitrary resolution, political subdivision identifiers to track appropriate races, and and optional body element for full length articles.

The simple structure is flexible enough to support content presentation in the fashion popularized by social media, long-form articles, or media-heavy content with images, videos, or documents distributed via BitTorrent. References to images or video in the description or body elements may be external via HTTPS or internal references to files within an attached torrent archive.

Data Model

      {
        "name": "",                          // recommended name of archive
        "infoHash": "",                      // SHA-1 of .info (v1)
        "created": 0,                        // timestamp
        "createdBy": "",                     // identity string
        "urlList": [ "" ],                   // array of webseed URLs
        "announce": [ "" ],                  // array of tracker URLs
        "files": [                           // array of files, if multiple
          { "name": "", "offset": 0, "length": "" }
        ],
        "provenance": {                      // PSQR Provenance
          "jwk": {                           // JSON Web Key
            "kty": "EC",                     // type: ellyptical curve
            "crv": "P-384",                  // curve
            "alg": "ES384",                  // signing algo
            "kid": "",                       // key ID - did#keyname
            "x": "",                         // key data
            "y": ""                          // key data
          },
          "signature": "",                   // JWS string
                                               // compact serialized, detached
                                               // no header, no payload (redundant)
          "publisher": {                     // PSQR publisher data
            "name": "",                      // name string, human readable
            "image": "",                     // image URL
            "url": "",                       // primary URL of publisher
            "tagline": ""                    // tagline string
          }
        },
        "info": {
          "publicSquare": {                  // PSQR Namespace
            "package": {                     // packaging / presentation data
              "geo": "",                     // geonum string (leading 0s)
              "politicalSubdivison": "",     // space-delimited identifiers
              "publishDate": 0,              // timestamp
              "lang": "en",                  // english
              "title": "",                   // og:title
              "description": "",             // og:description, twitter body, short
              "image": "",                   // og:image
              "canonicalUrl": "",            // source URL, if crawled
              "body": "",                    // article body, longer
              "references": {
                "content": {
                  "reply": "",               // .infoHash reference of parent
                  "amplify": "",             // .infoHash reference of child
                  "like": ""                 // .infoHash reference of parent
                                               // likes would not have other content
                }
              }
            }
          }
        }
      }

Data Viewability

The use of JSON and JSONL is convenient and readily accessible to software developers. JSON is plain text, and JSONL simply bundles multiple JSON objects into a single text file, one per line. Using this scheme, the data structures are readily visible from the command line with curl and jq.

For example, to see the structure of the most recent Tweet from a United States Senator or Member of Congress published to Ology Newswire, use the command below. (Curl fetches the data, head -1 gives us a single line, and jq filters to the core data structure and formats the JSON.) 

curl --silent https://feed.ology.com/feed/politics-senators/latest.jsonl | \
head -1 | jq ".metainfo"

Standards for Reference

Standard Description
BitTorrent v1 Peer to peer file distribution.
Webseed HTTP(S) as unchoked seed.
Open Graph Content presentation data standard (Facebook).
Twitter Cards Content presentation data standard (Twitter).
JSON Web Key JSON cryptographic key representation.
JSON Web Signature JSON cryptographic signature representation.
JWS Compact Serialization JWS compact serialization.
JWS Detached JWS detached signatures.