hermes-brain/projects/passepartout/social-protocol/requirements-03-infrastructure.org

#+title: Social Protocol Requirements - 03: Infrastructure
#+author: Amero Garcia
#+created: [2026-03-16 Mon 14:28]
#+DATE: 2026-03-14
#+ID: agora-requirements-03-infrastructure
#+STARTUP: content

:PROPERTIES:
:CREATED:  [2026-05-24 Sun]
:ID:       3b43a9b8-31d1-4479-a35f-22273b74f0c7
:END:
* The Sovereign Infrastructure: Your Digital Foundation

[[id:1d074690-a279-59cb-b91d-e9a22ae104ad][Social Protocol]]'s infrastructure is meticulously architected to deliver on the promise of true digital sovereignty. Unlike traditional platforms that centralize control, the protocol distributes power to the edges—directly into the hands of users. This section details the foundational infrastructure that makes self-sovereign identity, data ownership, decentralized communication, and global discovery not just possible, but practical and scalable.

** Personal Data Store (PDS): Your Digital Fortress

The Personal Data Store (PDS) is the cornerstone of the protocol's sovereignty model—your personal, encrypted vault where all your Notes, identity data, and digital assets reside. Unlike cloud services that claim ownership of your data, your PDS is unequivocally yours. You control it. You host it. You decide who accesses it. It is the physical manifestation of your digital self-sovereignty.

*** Requirements

- The system MUST use a hybrid network architecture with Personal Data Stores (PDS) and Relays.
- Every user MUST control their own PDS (hosted or self-run).
- The PDS MUST serve as the master archive for all the user's content (encrypted) and identity data.
- The PDS MUST act as a gatekeeper, issuing decryption keys upon valid payment or credential verification.
- Relays MUST NOT store data long-term (unless paid to).
- Relays MUST route ciphertext based on CID and persona subscriptions.
- The system MUST incentivize Relays to route high-traffic content or provide specific delivery guarantees.
- The system MUST allow users to publish their CIDs to multiple relays to ensure availability and bypass censorship.
- The system MUST use Double Ratchet for 1-on-1 private messaging.
- The system SHOULD use MLS (Messaging Layer Security) for group chats.
- The system MUST use symmetric encryption for paywalled content (individual keys per object).
- The system MUST support social recovery using Shamir's Secret Sharing, allowing users to split their Master Key into shards distributed to trusted guardians.

*** Technical Logic

**** Personal Data Store (PDS)
- *Home Base:* Every user controls their own PDS (hosted or self-run).
- *Master Archive:* All the user's content (encrypted) and identity data live here.
- *Key-Server Separation:* The PDS MUST include a distinct Key-Management Module that handles the automated sale and distribution of decryption keys/LSATs. This MUST be logically separate from the Data-server hosting the encrypted blobs, ensuring that the entity holding the keys does not necessarily host the content payload.
- *Access Control:* PDS acts as a gatekeeper, issuing decryption keys upon valid payment or credential verification.

**** Encryption Model (E2EE)
- *Double Ratchet:* Used for 1-on-1 private messaging.
- *MLS (Messaging Layer Security):* Proposed for group chats.
- *Symmetric Encryption:* Used for paywalled content (individual keys per object).
- *Envelope Encryption (Data-at-Rest):* To protect against stolen devices, PDS storage utilizes Envelope Encryption. Large files are encrypted with a random Data Encryption Key (DEK), which is itself encrypted (wrapped) with the Persona Public Key.
- *Automated Re-Keying Service:* The PDS MUST include a background worker that triggers upon a `KEY_ROTATION_EVENT`. The worker iterates through KeyHeader objects and uses a Proxy Re-Encryption (PRE) scheme to securely re-wrap the DEKs with the new key, without ever exposing the raw Master Seed to the PDS.

*** Developer Implementation

To send a private message:
1. Encrypt message for the recipient's Persona Encryption Key (X25519).
2. Upload ciphertext to the user's PDS.
3. Notify the recipient's subscribed Relays of the new CID.
4. Recipient's client fetches the CID from the Relay/PDS and decrypts locally.

*** PDS Migration: Seamless Sovereignty Transfer

PDS Migration represents a fundamental capability of the protocol's architecture—the seamless, user-initiated transfer of one's entire digital corpus from one Personal Data Store to another. Unlike traditional platforms where data migration is often complex, permission-based, or impossible, the protocol treats PDS Migration as a first-class operation. This is not an edge case, but a core feature that ensures users retain ultimate sovereignty over their data throughout its lifecycle. Whether changing hosting providers, upgrading [[id:84a537b4-4256-50c8-91f5-dd5b4538418f][hardware]], or responding to security incidents, PDS Migration ensures users are never trapped by infrastructure choices.

**** Concept

PDS Migration is the comprehensive process of transferring a user's entire encrypted content repository and identity data from one PDS to another while rigorously maintaining Content Identifier (CID) integrity, subscription continuity, and access control mechanisms. This process ensures that all cryptographic relationships between Notes remain intact, and that no data is lost or corrupted during the transfer.

Key principles of PDS Migration:

-   *User Sovereignty Absolute:* Users retain complete autonomy to migrate their data without requiring permission, intervention, or cooperation from any third party. This is a fundamental right within the protocol ecosystem.
-   *Zero-Downtime Operation:* Migration SHOULD occur without interrupting the user's ongoing presence or activities on the network. This ensures continuous availability of services and interactions.
-   *Rollback Safety:* Users MUST have the capability to revert to the original PDS if the new PDS fails to perform adequately or if any issues arise during the migration process. This provides a safety net for critical data transfers.
-   *Atomic Cutover:* There is a clearly defined, atomic moment when the new PDS becomes the authoritative source of truth, and the old PDS transitions to a backup role, ensuring data consistency.

Migration scenarios include a comprehensive range of use cases:
-   Self-hosted PDS → Cloud-hosted PDS (moving from personal infrastructure to professional hosting)
-   Cloud provider A → Cloud provider B (e.g., AWS → GCP, avoiding vendor lock-in)
-   Old hardware → New hardware (self-hosted upgrade for improved performance or capacity)
-   Compromised PDS → Clean PDS (security incident response and remediation)
-   Cost optimization (migrating to more economical hosting solutions)
-   Performance enhancement (migrating to geographically closer or faster infrastructure)

**** Requirements

- The system MUST support full PDS migration without service interruption.
- The system MUST preserve all Content Object CIDs during migration (content-addressed storage guarantees this).
- The migration process MUST update the Persona's DID Document to point to the new PDS service endpoint.
- The system MUST notify all subscribed Relays about the PDS endpoint change.
- The system MUST support parallel operation (old and new PDS active simultaneously) during migration testing.
- The system MUST provide migration progress tracking and verification.
- The system MUST support incremental pre-migration sync to minimize final cutover time.
- The system MUST handle in-flight messages during cutover (queue and forward).
- The system MUST provide a rollback mechanism if migration fails.

**** Migration Phases

***** Phase 0: Preparation

- *Prerequisites:* Ensure new PDS meets minimum specs (storage, bandwidth, availability).
- *Provisioning:* Configure new PDS with Persona's DID but initially in "standby" mode.
- *Authorization:* New PDS MUST prove ownership via Persona signature challenge.

***** Phase 1: Initial Sync

- *Full Replication:* Transfer all Content Objects from old PDS to new PDS.
- *CID Verification:* Block-by-block verification that all content transferred correctly.
- *Metadata Sync:* Sync Persona profiles, access control lists, and configuration.
- *Duration:* May take hours/days depending on data volume.
- *Old PDS Remains Authoritative:* Writes still go to old PDS during this phase.

***** Phase 2: Incremental Catch-up

- *Delta Sync:* Catch up changes made since Phase 1 started.
- *Repeat:* [[id:22d0a159-68a2-4587-9375-5046beddc20c][Continue]] incremental syncs until delta is small (e.g., < 1 minute of data).
- *Read Testing:* Client optionally reads from new PDS to verify accessibility.

***** Phase 3: Cutover

- *Freeze Writes:* Brief write lock on old PDS (seconds to minutes).
- *Final Delta:* Transfer last remaining changes.
- *DID Update:* Publish new DID Document with new PDS service endpoint.
- *Relay Broadcast:* Announce endpoint change to all subscribed Relays.
- *New PDS Becomes Authoritative:* Write traffic now routes to new PDS.

***** Phase 4: Stabilization

- *Monitor:* Observe new PDS for errors, dropped messages, or performance issues.
- *Verification:* Confirm all personas can reach new PDS, all content accessible.
- *Grace Period:* 24-48 hour buffer where old PDS remains available as hot standby.
- *Rollback Window:* If issues detected, can revert to old PDS via DID re-update.

***** Phase 5: Decommissioning

- *Archive:* Old PDS data backed up (user's discretion).
- *Tombstone:* Old PDS endpoint publishes redirect or shutdown notice.
- *Cleanup:* Remove old PDS from user's infrastructure (cancel cloud instance, retire hardware).
- *Final Verification:* Confirm no traffic routing to old PDS.

**** Technical Considerations

***** Concurrent Access During Migration

- *Read Replication:* Old PDS can serve reads while new PDS receives writes (after cutover) to reduce downtime to near-zero.
- *Message Queueing:* Relays queue messages during the brief cutover window; messages forwarded once new PDS confirms readiness.
- *Conflict Avoidance:* Strict sequencing ensures no split-brain scenarios (only one PDS accepts writes at any time).

***** Verification Protocol

- *CID Audit:* Iterate through all CIDs in Persona's content graph; verify retrievable from new PDS.
- *Signature Verification:* Spot-check Content Object signatures against Persona's public keys.
- *Access Control Verification:* Test decryption of sample encrypted content using Persona's keys.
- *Subscription Testing:* Verify Relays successfully forward new CIDs from new PDS.

***** Rollback Procedures

- *Trigger:* Migration fails verification or new PDS experiences critical failure.
- *DID Reversion:* Re-publish previous DID Document with old PDS endpoint.
- *Relay Re-announce:* Broadcast reversion to all Relays.
- *Data Reconciliation:* If any writes occurred on new PDS before failure, sync them back to old PDS before re-activating.
- *Graceful Degradation:* If both PDS fail, Persona can restore from backup and re-establish with same or new infrastructure.

**** Developer Implementation Example

#+begin_src typescript
// Initiate PDS migration sequence
interface PDSMigrationPlan {
  sourcePDS: string;           // Old PDS endpoint
  targetPDS: string;           // New PDS endpoint
  personaDID: string;
  phases: MigrationPhase[];
  estimatedDuration: number;     // Estimated seconds for full migration
  rollbackDeadline: number;      // Timestamp until rollback is possible
}

interface MigrationPhase {
  name: 'preparation' | 'full-sync' | 'incremental' | 'cutover' | 'verification' | 'completed';
  status: 'pending' | 'in-progress' | 'completed' | 'failed';
  startedAt?: number;
  completedAt?: number;
  progressPercent: number;
}

// Phase 1: Full replication
async function replicateContentObjects(
  sourcePDS: string,
  targetPDS: string,
  personaDID: string,
  authToken: string
): Promise<ReplicationResult> {
  const sourceClient = new PDSClient(sourcePDS, personaDID);
  const targetClient = new PDSClient(targetPDS, personaDID);
  
  // Fetch all CIDs from source
  const allCIDs = await sourceClient.listAllCIDs();
  
  // Batch transfer with verification
  const results = await batchTransfer(allCIDs, {
    source: sourceClient,
    target: targetClient,
    verifyCID: true,             // Verify hash after transfer
    batchSize: 100,
    concurrency: 5
  });
  
  return {
    transferred: results.successful.length,
    failed: results.failed,
    duration: results.elapsedTime
  };
}

// Phase 3: Cutover - Update DID Document
async function executeCutover(
  persona: Persona,
  newPDSEndpoint: string,
  oldPDSEndpoint: string
): Promise<CutoverResult> {
  // 1. Freeze writes on old PDS
  await freezeOldPDS(oldPDSEndpoint, persona.did);
  
  // 2. Final incremental sync
  await finalIncrementalSync(oldPDSEndpoint, newPDSEndpoint);
  
  // 3. Update DID Document with new service endpoint
  const updatedDoc = await updateDIDDocument(persona.did, {
    service: [{
      type: 'PDS',
      serviceEndpoint: newPDSEndpoint,
      // ... other service properties
    }]
  });
  
  // 4. Sign and publish new DID Document
  const signedDoc = await persona.sign(updatedDoc);
  await didResolver.publish(signedDoc);
  
  // 5. Notify all subscribed Relays
  await notifyRelays(persona.did, {
    type: 'PDS_ENDPOINT_CHANGE',
    oldEndpoint: oldPDSEndpoint,
    newEndpoint: newPDSEndpoint,
    signature: signedDoc.proof
  });
  
  return { status: 'success', newDocumentCID: signedDoc.cid };
}

// Verification: Confirm all content accessible
async function verifyMigration(
  newPDS: string,
  personaDID: string,
  expectedCIDCount: number
): Promise<VerificationResult> {
  const client = new PDSClient(newPDS, personaDID);
  
  // Verify CID count matches
  const reachableCIDs = await client.listAllCIDs();
  if (reachableCIDs.length !== expectedCIDCount) {
    throw new Error(`CID mismatch: expected ${expectedCIDCount}, found ${reachableCIDs.length}`);
  }
  
  // Spot-check: verify random sample of CIDs
  const sample = selectRandomSample(reachableCIDs, 100);
  const verificationResults = await Promise.all(
    sample.map(cid => verifyContentObject(client, cid))
  );
  
  const failures = verificationResults.filter(r => !r.valid);
  if (failures.length > 0) {
    throw new Error(`Verification failed for ${failures.length} objects`);
  }
  
  return { status: 'verified', sampleSize: sample.length, failures: 0 };
}
#+end_src

**** User Experience Considerations

- *Progress Dashboard:* Real-time view of migration progress with estimated time remaining.
- *Notification:* Alerts to user's clients about upcoming maintenance window (for cutover).
- *Zero-Click Resume:* If migration interrupted, resumes from last checkpoint automatically.
- *Cost Transparency:* Estimate bandwidth/storage costs before starting (especially for cloud-to-cloud).
- *Support Contact:* During migration, provide direct line to new PDS operator for issues.

**** Security During Migration

- *Authenticated Transfer:* All replication traffic encrypted and authenticated (mTLS or Noise).
- *No Plaintext Exposure:* Ciphertext transferred as-is; no decryption during migration.
- *Audit Trail:* All migration events logged as tamper-evident Content Objects.
- *Guardian Notification:* Optional: notify social recovery guardians of major infrastructure change.
- *Rate Limiting:* Prevent migration from overwhelming either PDS; throttle if needed.

*** PDS-to-PDS Synchronization: Redundancy and Resilience

In a truly sovereign digital ecosystem, users should never be constrained to a single point of failure. the protocol's PDS-to-PDS Synchronization protocol empowers users to run multiple Personal Data Stores simultaneously—for redundancy, load balancing, or geographic distribution. This protocol enables bidirectional synchronization of encrypted Content Objects between a user's PDS nodes, maintaining CID integrity, conflict resolution, and data consistency across the distributed infrastructure. It ensures that your digital presence remains resilient, available, and performant, regardless of individual infrastructure limitations.

**** Concept

The PDS-to-PDS Synchronization Protocol allows users to maintain multiple, synchronized copies of their encrypted data across different PDS instances. This capability transforms the PDS from a single point of failure into a distributed, fault-tolerant system that can withstand outages, attacks, or infrastructure changes. By synchronizing data across multiple nodes, users achieve:

-   *High Availability:* If one PDS becomes unreachable, others can seamlessly serve data, ensuring continuous access to your digital identity and content.
-   *Geographic Distribution:* PDS nodes can be strategically located in different regions to minimize latency and comply with local data sovereignty requirements.
-   *Load Distribution:* High-traffic Personas can distribute read operations across multiple synchronized PDS nodes, improving performance and responsiveness.
-   *Disaster Recovery:* Synchronized PDS nodes provide inherent backup capabilities, ensuring data preservation even in catastrophic failures.

**** Sync Protocol Architecture

**** Merkle DAG Synchronization
- Each PDS maintains a Merkle DAG of all stored Content Objects.
- Root hash represents complete state of the PDS.
- Sync compares Merkle roots to efficiently identify differences.

**** Sync Modes

**** Full Sync
- Complete synchronization of all Content Objects.
- Used for initial setup or recovery from desync.
- Sends all CIDs not present in the other PDS.

**** Incremental Sync
- Only synchronizes changes since last sync.
- Maintains sync cursor (last synced CID timestamp).
- More efficient for ongoing synchronization.

**** Selective Sync
- Synchronizes only specific content types or time ranges.
- User-defined filters (e.g., "only public posts", "last 30 days").
- Useful for bandwidth-constrained scenarios.

**** Sync Process

**** Phase 1: Handshake
- PDS nodes authenticate using Persona's DID.
- Exchange authentication proofs (signatures).
- Verify both nodes are authorized for this Persona's data.
- Exchange capabilities (sync modes supported, bandwidth limits).

**** Phase 2: Discovery
- PDS A computes Merkle root of current Content Object set.
- PDS B does the same.
- Compare roots: if equal, sync complete; if different, continue.
- Exchange Merkle proofs to identify divergent branches.

**** Phase 3: Delta Calculation
- Based on Merkle proof comparison, identify missing CIDs on each side.
- Calculate delta: set of CIDs A has that B doesn't, and vice versa.
- Partition delta into batches for transfer.

**** Phase 4: Transfer
- Request missing CID payloads from peer PDS.
- Receive Content Objects with full metadata.
- Verify CID integrity (content-addressed verification).
- Store in local PDS.

**** Phase 5: Conflict Resolution
- If same CID exists with different content (hash mismatch):
  - Both versions preserved (content-addressed storage).
  - Conflict marked for manual resolution.
  - User interface shows both versions, user selects authoritative.
- If same CID exists with same content: no conflict.

**** Phase 6: Confirmation
- Both PDS nodes recompute Merkle roots.
- Verify roots match post-sync.
- Update sync cursor for incremental future syncs.

**** Sync Conflicts

**** Conflict Types

**** Write-Write Conflict
- Same CID modified differently on two PDS nodes simultaneously.
- Resolution: Keep both, mark secondary as "alternate version", user resolves.

**** Tombstone Conflict
- CID deleted on PDS A, modified on PDS B.
- Resolution: Configurable policy ("last write wins" vs. "preserve all").

**** Schema Conflict
- Content Object valid on PDS A but rejected by PDS B (schema mismatch).
- Resolution: Log error, quarantine object, notify user.

**** Periodic Synchronization

- *Frequency:* User-configurable (real-time, hourly, daily).
- *Real-time Sync:* WebSocket connection for immediate propagation.
- *Scheduled Sync:* Cron-like jobs for periodic full or incremental syncs.
- *On-Demand Sync:* Manual trigger by user or administrator.

**** Security Considerations

- *Authentication:* Both PDS nodes MUST authenticate as authorized for Persona's data.
- *Encryption:* Sync traffic SHOULD be encrypted (TLS 1.3 or Noise Protocol).
- *Authorization:* PDS nodes MAY implement additional access controls (IP allowlists).
- *Audit:* All sync events logged as Content Objects for audit trail.

**** Performance Optimization

- *Delta Encoding:* Only transfer missing CIDs and metadata.
- *Compression:* Payload compression for large Content Objects.
- *Parallel Transfer:* Multiple concurrent transfers for large datasets.
- *Resume:* Partial transfers resume from interruption point.

**** Implementation (TypeScript)

#+begin_src typescript
interface PDSSyncSession {
  sessionId: string;
  participantPDS: string[];       // PDS DIDs
  personaDID: string;
  mode: 'full' | 'incremental' | 'selective';
  status: 'handshake' | 'discovery' | 'transfer' | 'complete' | 'error';
  startedAt: number;
  completedAt?: number;
}

interface MerkleNode {
  cid: string;
  children: MerkleNode[];
  hash: string;                   // Blake3 hash of concatenated child hashes
}

interface SyncDelta {
  fromPDS: string;
  toPDS: string;
  missingCIDs: string[];
  conflictCIDs: string[];
  estimatedSize: number;          // Bytes
}

interface SyncConfig {
  mode: 'full' | 'incremental' | 'selective';
  filter?: {
    contentTypes?: string[];
    afterTimestamp?: number;
    beforeTimestamp?: number;
    publicOnly?: boolean;
  };
  frequency?: 'realtime' | number; // number = seconds between syncs
  priority?: 'low' | 'normal' | 'high';
}
#+end_src

*** Distributed Mirroring & Social Resilience

**** Following: Default "Feed Gossip" & The Phoenix Effect
The protocol ensures baseline content resilience by leveraging a gossip-based mirroring architecture triggered by every "Follow" event.
- *Following = Essential Replicating:* When a user "follows" another Persona, their device or PDS automatically joins the gossip swarm for that target's most critical low-bandwidth data.
- *Feed Gossip Scope:* To balance network resilience with device resources, default gossip is restricted to the *Identity Log (KEL)* and a rolling window of *recent text-based Notes* (e.g., the last 1,000 posts).
- *The Phoenix Effect:* This level of mirroring ensures the "Phoenix Effect" remains viable. If a user's PDS is destroyed, they can "shout" to their followers: "I am the owner of DID:123. Please send me everything you have signed by my key." The essential history and social graph flow back from the community.
- *Censorship Resistance:* By making essential gossip a default behavior, the social graph and latest news stay alive through the "cracks" of the internet automatically.

**** Supporting: Opt-in "Supporter-Mirroring" & Decentralized CDN
For high-bandwidth content and deep archives, the protocol transitions from simple gossip to an explicit infrastructure donation model.
- *Persistent Mirroring:* When a user clicks "Support," they opt-in to a deeper technical commitment. The supporter's PDS archives the *entire historical feed* of the creator, not just the recent window.
- *High-Bandwidth "Pinning":* Supporters provide the backbone for the *"Follower-as-CDN"* model. A supporter can allocate specific storage (e.g., "Pin 5GB of latest video") to ensure large payloads (audio, video, high-res images) remain highly available.
- *WebRTC Peering & Seeding:* Supporters act as active "Seeds" in a BitTorrent-style swarm. When a new viewer watches a video, the app prioritizes streaming via WebRTC from online supporters rather than the creator's PDS, ensuring viral content has $0 infrastructure cost for the creator.

**** "In-Kind" vs. "Monetary" Support
This tiered model transforms the relationship between organizations and their communities:
- *Scalable In-Kind Support:* A "Poor but Loyal" follower contributes at the Gossip tier (bandwidth for text), while a "Dedicated Patron" contributes at the Mirroring tier (storage for video).
- *Resilience against De-platforming:* Even if a government blocks a creator's main server, the "Swarm" of followers and supporters continues to host and circulate the content through the P2P network.

**** Encrypted Peer-Backups (The "Friend-Box")
While the social swarm recovers public history, users ensure the recovery of private data (drafts, DMs) via formal peer-to-peer backup agreements.
- *The "Friend-Box" Logic:* Users can establish reciprocal "Friend-Box" agreements where they swap encrypted storage space (e.g., swapping 10GB of space with three trusted friends).
- *Mechanism:* The user's PDS automatically generates and sends an encrypted, compressed "State Snapshot" (a Merkle DAG of the entire PDS state) to these friends' servers periodically (e.g., nightly).
- *Privacy Guarantee:* Because the backup is encrypted with the user's keys, the friends cannot read the private drafts or DMs; they only host the raw ciphertext blobs.
- *Restoration:* In the event of a catastrophic local failure (e.g., fire, server loss), the user can download their latest snapshot from a friend and instantly restore their entire digital life to a new PDS node using their recovered Identity Keys.

** Relay Network: The Circulatory System of the Protocol

The Relay Network serves as the protocol's intelligent, adaptive message routing layer—ephemeral, user-chosen pathways that efficiently route encrypted Notes via a pub/sub model. Unlike centralized servers that store and monitor your data, Relays are transient routers that respect your privacy, delivering your messages without ever holding them long-term. They are the circulatory system of the protocol network, ensuring vital communication flows freely and securely.

*** Requirements

- Relays MUST route ciphertext based on CID and persona subscriptions.
- Relays MUST NOT store data long-term (unless paid to).
- The system MUST incentivize Relays to route high-traffic content or provide specific delivery guarantees.
- The system MUST allow users to publish their CIDs to multiple relays to ensure availability and bypass censorship.
- Relays MUST support subscription filters for content discovery.

*** Technical Logic
*** Relay Routing & Prioritization: Pay-to-Prioritize (P2P)

To ensure high-performance and sustainability without central control, the protocol's Relays utilize a *Pay-to-Prioritize (P2P)* routing strategy. Crucially, Relays are *Logic-Blind*. They do not inspect the Note's payload or contract terms (which may be encrypted). Instead, they prioritize traffic based on explicit, unencrypted metadata.

**** Explicit Priority Fee (The "Fast-Lane")
If a Note requires instant routing (e.g., a time-sensitive financial transaction or live chat), the sender can attach a Lightning micropayment directly to the routing request.
- *`priority_fee`:* A metadata field indicating the attached fee. Relays automatically move Notes with sufficient priority fees into the highest-speed queue.
- *Proof of Priority:* The fee *is* the proof. The Relay doesn't need to know *why* the Note is important, only that the sender is willing to pay for bandwidth.

**** Economic Density & Wire-Size Billing
Relays manage their resources using an *Economic Density* metric:
- *Sender Reputation (Zero-Fee Routing):* Small Notes from highly staked or reputable DIDs are often routed with zero additional fees to foster network liquidity and social interaction.
- *Low-Density (Large/Static):* Large Notes (e.g., binary payloads, media) are routed via *Bulk Pipes*. Billing for these Notes is proportional strictly to their raw payload size on the wire.

**** Incentivization
- Relays charge for routing high-traffic content or for specific delivery guarantees based on wire-size and explicit priority fees.

*** Relay Discovery

*** Relay Economics and Network Resilience

**** Relay Discovery

***** Bootstrap Problem

New clients need to find Relay nodes without hardcoded lists (centralization risk).

***** Discovery Mechanisms

****** DNS TXT Records
- Domain: `_agora-relay._tcp.example.com`
- Returns: List of relay endpoints (WebSocket URLs)
- Update: DNS propagation handles relay churn

****** Well-Known DID
- DID: `did:agora:bootstrap`
- Service Endpoint: "RelayDirectory" with list of known high-reputation relays
- Maintained: By Protocol Trust, updated quarterly

****** DHT (Future)
- Distributed hash table for relay discovery
- Similar to BitTorrent trackerless torrents
- Resistant to censorship

****** Social Bootstrap
- Friend's relay list shared on connection
- "You follow Alice → Oh, Alice uses Relay X, try that"
- Gossip protocol for relay reputation

**** Relay Subscription

***** Subscription Types

- *CID Filters:* Subscribe to new CIDs from specific DIDs
- *Topic Filters:* Subscribe to content with specific tags
- *Content-Type Filters:* Only contracts, only posts, etc.

***** Subscription Management

- *WebSocket:* Persistent connection for real-time updates
- *REST Poll:* HTTP polling for clients without WebSocket
- *Webhook:* Push notifications for server-side clients

***** Subscription Pricing
- *Basic:* Free (up to 100 subscriptions)
- *Premium:* 100 satoshis/month (unlimited)
- *Enterprise:* Negotiated (dedicated relay capacity)

**** Relay Operator Profiles
1.  *"Backbone" Providers (Big Tech/NGOs):* Large organizations (e.g., Bluesky Social PBC or the "Free Our Feeds" collective) run "Global Relays," ingesting entire network activity for platform-wide search and indexing.
2.  *"Neighborhood" Operators (NGOs & Communities):* NGOs, professional guilds, or town councils run community-specific relays, indexing only the members relevant to their specific domain.
3.  *"Specialists" (Commercial Startups):* Companies (e.g., Primal, River) run highly-optimized relays to power specific apps, prioritizing speed and specialized feature sets for their target market.

**** Relay Economic Models
To ensure sustainability without compromising user data (avoiding "Surveillance Capitalism"), operators utilize diverse revenue models:
-   *The "Anti-Spam" Entrance Fee:* One-time or monthly payments (e.g., $1–$5) via Bitcoin Lightning to discourage bot-farms and cover hardware costs.
-   *The "Boutique" Model (Freemium):* Free "Read" access with a paid subscription required to "Write" (post data), often offering guarantees for data persistence and indexing quality.
-   *Institutional "Public Good" Funding:* Government or NGO-funded "Public Interest Relays" provided as a digital utility to ensure censorship-resistant communication.
-   *"Zaps" & Micro-tips:* Direct user-to-operator micro-tips via integrated Lightning Keys, rewarding relays for high-quality filters or specialized indexes.

**** Relay Pricing

***** Standard Price Announcement
- Relay publishes `price_per_kb` in Lightning millisats
- WebSocket endpoint: `/pricing` returns current rates
- Update frequency: Hourly, cached by clients

***** Pricing Tiers

- *Basic:* 1 millisat/KB (default routing)
- *Priority:* 10 millisats/KB (fast lane)
- *Bulk:* 0.5 millisats/KB (>100KB messages)
- *Free:* 0 millisats/KB (below 1KB, within rate limits)

***** Fee Structure

- *Relay:* Keeps 70% of fees (operating costs)
- *Validator Oracles:* 20% (fraud detection)
- *Protocol Protocol:* 10% (development fund)

**** Network Resilience: Global Firehose vs. Fragmented Relays
The social protocol's design ensures that the relay network is inherently replaceable and resilient:
-   *Replaceable Relays:* Users can instantly switch to competitor relays if a provider becomes greedy or censorious by simply re-pointing their PDS.
-   *"Multi-homed" Data (Firehose Protection):* Users push posts to multiple relays simultaneously. If any relay fails, history remains accessible via others, ensuring followers can always maintain connectivity.

*** Privacy Considerations: The "Honeypot Relay" Risk

Because a relay is fundamentally a server that routes traffic, it is technically possible for an operator to offer a "free" service while secretly harvesting metadata to sell to advertisers. This creates the risk of "Honeypot Relays" during the network's early bootstrap phase.

**** The Metadata Harvesting Trap
Even if messages are End-to-End Encrypted (E2EE), a malicious relay can observe highly valuable metadata for surveillance capitalism:
- *IP Addresses:* Revealing the user's physical location and device profile.
- *The Social Graph:* Observing who a DID communicates with, how often, and who constitutes their "inner circle."
- *Activity Patterns:* Tracking when a user is active, sleeping, and which tags/topics they frequently query.
- *Unencrypted Content:* Building interest profiles based on public posts and read-only data.

**** Defense Against Decentralized Surveillance
While Honeypot Relays pose a risk, their power is fundamentally weaker than legacy Web 2.0 walled gardens:
1. *No Walled Garden (Instant Migration):* If a relay is discovered to be harvesting data, users simply uncheck a box in their PDS settings. They migrate to a new relay instantly, and their followers find them immediately because their identity (DID) remains constant.
2. *Fragmented Data:* Because users multi-home their data across several relays simultaneously, no single relay possesses the "whole picture" of a user's digital life.
3. *The Tor/VPN Option:* Advanced users and organizations can run their PDS traffic through Tor or a VPN, stripping away the IP address—the most valuable piece of surveillance data.

**** Organizational Defense: The Tiered Relay Strategy
For Collectives, NGOs, or LLCs managing sensitive operations, relying on "free" public relays is an unacceptable security risk. These entities MUST utilize a Tiered Relay Strategy:
- *Tier 1 (Internal Relay):* The NGO runs its own private, isolated relay exclusively for internal board and team communications. This relay is "dark" to the public internet and collects zero metadata for third parties.
- *Tier 2 (Public Gateway):* The organization uses high-traffic "Surveillance" or public relays solely for PR, marketing, and public announcements. They treat these relays like digital billboards—expected to be public, but never used for private business.

**** Relay Reputation

- *Uptime:* % availability over last 30 days
- *Latency:* Message propagation time
- *Censorship:* Has relay blocked legal content?
- *Fees:* Reasonable pricing?
- *Users:* Number of active personas (network effect)

** Search & Indexing: The Firehose Indexers

In a [[id:1d074690-a279-59cb-b91d-e9a22ae104ad][decentralized network]], finding historical content or discovering new Personas requires specialized indexing infrastructure. Indexing Nodes are high-performance servers that ingest the public Relay firehose to provide full-text search and complex discovery services.

*** Requirements
- Indexing Nodes MUST ingest public Content Objects from the Relay firehose.
- Indexing Nodes MUST support full-text search across public metadata and decrypted public content.
- The system MUST provide a standardized Search API for clients to query indexes.
- *The Aggregator API:* The standard endpoint MUST support an open querying format (e.g., `GET /search/query?q=keyword`) returning a structured schema of DIDs, Handles, and Profile snippets.
- *Ranking Transparency:* The provider MUST publish its Ranking Logic (e.g., "We prioritize accounts with 3+ Web-of-Trust endorsements") so users understand the algorithmic biases of the index.
- Indexing Nodes MUST respect content flags (e.g., `indexable: false` or `ephemeral`).
- The system MUST support "Private Indexing," where a user's PDS builds a local search index for their own encrypted history and DMs.

*** Technical Logic
- *Public Indexing:* Backbone providers run global indexers using technologies like Meilisearch or Elasticsearch to enable "Google-like" search for the whole platform.
- *Private Indexing:* Thin clients delegate private search tasks to the user's PDS, which maintains a local, encrypted index of all authorized Content Objects.
- *Economics:* High-speed indexing services MAY utilize micro-payments (Lightning) or subscriptions for advanced query capabilities or higher rate limits.

*** Persona Discovery & The Global Directory
To solve the UX problem of finding friends in a decentralized namespace, Indexers act as a Global Directory, constantly cataloging Handle <-> DID mappings broadcast over the network.

**** Multi-Format Handle Indexing
When a user searches for "@alice," the Indexer searches across all supported namespace formats simultaneously:
- *Subdomains:* `alice.aletheia.social`
- *Custom Domains:* `alice.com`
- *Web3 Names:* `alice.eth` or `alice.p2p`

**** Verified Search Results (Anti-Squatting)
Because anyone can theoretically publish a Note claiming to be "Alice," the Search UI relies on DIDs to guarantee authenticity.
- *Cryptographic Back-Links:* The Search engine ONLY displays a "Verified" checkmark if the human-readable handle (e.g., `alice.com`) has a valid cryptographic DNS/TXT record pointing back to the Persona's DID, *and* the DID has published a signed Note claiming that handle.
- *Unverified Flagging:* If a user squats on a username without owning the corresponding domain or blockchain record, the Indexer explicitly flags the search result as "Unverified" or excludes it.

**** "Privacy-First" Search
Because the social protocol supports multiple isolated Personas per user, global search is strictly opt-in:
- *Public Personas:* (e.g., a "Work" or "Creator" Persona) publish a "Directory Opt-In" Note. Indexers catalog them, making them searchable by anyone.
- *Private Personas:* (e.g., an "Anonymous" or "Family" Persona) do not publish this Note. They are entirely hidden from global Indexers. To find a Private Persona, another user must possess their exact DID string or be invited via a secure DIDComm routing channel.

** Social Protocol-to-Web Gateways: The Bridge to the Open Web

*** Concept
To make decentralized, P2P content accessible to users on the "Open Web" (traditional browsers like Chrome or Safari without special plugins), the social protocol must bridge the gap between Content-Addressed Data (CIDs) and Location-Addressed URLs. 

Gateways act as "translators" sitting on the edge of the decentralized network, talking HTTP to the legacy web while speaking P2P protocols internally. Every PDS or dedicated "Public Relay" can act as a web gateway.

*** 1. The HTTP Gateway (The Bridge)

**** Public Gateway API & URL Mapping
A piece of content identified by its hash (CID), such as `bafy...123`, can be viewed by anyone at a standard HTTP URL.
- *Pathing:* Gateways MUST support standard pathing `/ipfs/{cid}` and `/at/{did}/{collection}/{rkey}`.
- *CORS Policy:* Gateways MUST implement a permissive CORS policy to allow decentralized applications (dApps) to fetch media directly across origins.
- *MIME-Type Sniffing:* The gateway MUST read the Universal Event metadata and serve correct HTTP headers (e.g., `Content-Type: video/mp4`) so browsers handle the media natively.

**** The Translation Process
When a browser hits that link, the Gateway performs the following automated steps:
1. *Fetch:* Retrieves the data from the P2P swarm using the social protocol's native protocols.
2. *Verify:* Cryptographically verifies the Note's signature against the creator's Persona DID to ensure authenticity.
3. *Wrap:* Wraps the raw content (Markdown, JSON) in standard HTML/CSS templates so it renders correctly in a standard web browser.

*** 2. Human-Readable Handles (DNS & ENS)

Most users will not share long cryptographic hashes. To make content web-friendly, Gateways automate domain routing.

**** DNSLink (Traditional Domains)
Users can point their own domains (e.g., `alice.com`) directly to their Persona. 
- *Automatic Resolution:* When someone visits `alice.com`, the Gateway reads a DNS TXT record that says: "Go find content hash XYZ on the social protocol network."
- *Zero-Configuration SSL:* Gateways automatically provision and renew HTTPS certificates (via Let's Encrypt) for any domain linked to a Persona DID.
- *Well-Known Verification:* Gateways automatically serve the user's DID document at `https://[custom-domain]/.well-known/atproto-did` to prove ownership.

**** Automated Subdomain Issuance (Registrar Service)
To onboard users quickly without forcing them to buy a domain, PDS providers act as Registrars.
- *Availability & Routing:* The PDS performs an automated availability check. If a handle is free, it updates its Virtual Host configuration and internal DNS to instantly route traffic for `newuser.provider.org`.

**** Web3 Domains (.eth, .p2p)
For users utilizing blockchain-based naming services, the social protocol integrates with specialized gateways (e.g., Eth.limo). A user types `yourname.eth.limo` into a standard browser, and the gateway does the heavy lifting of resolving the blockchain record and serving the underlying P2P data.

*** 3. Social Mirroring for Search Engines (SEO)

To ensure social protocol content is discoverable on legacy search engines like Google, the network utilizes automated rendering pipelines.

**** The Firehose
Social protocol Relays emit a continuous "Firehose" of every public Note created on the network.

**** SEO Rendering (App Views)
Specialized indexers or "App Views" (functioning like web-frontends) consume this firehose. They automatically generate static, crawlable HTML pages for every public profile, post, and thread. This ensures that decentralized content is aggressively indexed by Google's web crawlers, matching or exceeding the discoverability of traditional centralized blogs.

*** 4. Persona-as-Host (Native Web Hosting)
Because of this robust Gateway architecture, publishing a website becomes a native feature of the network.
- *Static Asset Resolver (SAR):* The PDS/Gateway can interpret a directory CID as a web root. If a request hits a folder CID without a filename, the SAR automatically serves `index.html`. It resolves all internal assets (images, CSS) using Relative Pathing to ensure the site works regardless of the gateway domain.
- *Automated Deployment (Push-to-Publish):* Developers can use Git integration. When code is pushed, a CI/CD action builds the site, signs the resulting root CID with the Persona Key, and broadcasts the Publish Event to the PDS.
- *Instant Rollbacks:* Every Publish Event is logged in the Persona's immutable history. Reverting a site is simply signing a new Note pointing back to a previous CID.

*** Monetized Static Sites (Split-State Encryption)
Gateways integrate with the Exchange layer. Owners can host static sites where certain paths are encrypted. The Gateway serves the public storefront, but the actual asset is only "unwrapped" locally if the user's browser provides a Lightning Preimage (LSAT) proving payment.

*** Requirements

- Gateways MUST take CID-based social protocol content and render it as HTML for legacy browsers.
- Gateways MUST support SEO-friendly rendering for public content.
- The system MUST allow anyone to run a Gateway (not restricted to Relay operators).
- Gateways MUST NOT require authentication for public content.
- Gateways SHOULD cache content to reduce load on PDS/Relay networks.
- The system MUST support Gateway discovery (similar to Relay discovery).
- Gateways MUST respect content flags (e.g., `indexable`, `ephemeral`).
- Gateways MUST NOT expose private/direct content.

*** Relationship to Relays

- *Relays* serve protocol-native clients via WebSocket/pub-sub protocols.
- *Gateways* serve legacy browsers via HTTP.
- They are *separate infrastructure* - a Gateway may use Relays as a backend, but they're distinct services.

*** Gateway Discovery

**** Concept
Gateways bridge social protocol content to the legacy web via HTTP. Discovery mechanisms are needed for clients to find reliable gateways to generate shareable HTTP links for their public content.

**** Discovery Mechanisms

***** Public Registry
- A well-known DID (e.g., `did:agora:gateways`) maintains a list of verified, active gateways.
- Clients can query this registry to get a randomized or latency-sorted list of active gateways.

***** DNS TXT Records
- Similar to Relay discovery, domains can publish `_agora-gateway._tcp` TXT records pointing to HTTP endpoints.

***** User Preference
- Users can manually configure a preferred gateway in their client settings (e.g., `agora.example.com`).
- Clients use this preferred gateway when generating "Share Link" URLs.

** Infrastructure Discovery: DID Document Bindings

For a Persona to function within the network, its Decentralized Identifier (DID) must "bind" to specific infrastructure endpoints. This is achieved via the `service` section of the social protocol DID Document.

*** The Service Schema
Every social protocol DID Document SHOULD include a list of service endpoints that allow other Personas and clients to locate the user's data and communication channels.

#+begin_src json
{
  "id": "did:agora:123...",
  "service": [
    {
      "id": "#pds",
      "type": "AgoraPDS",
      "serviceEndpoint": "https://pds.example.org"
    },
    {
      "id": "#relay",
      "type": "AgoraRelay",
      "serviceEndpoint": "wss://relay.aletheia.social"
    },
    {
      "id": "#search",
      "type": "AgoraSearch",
      "serviceEndpoint": "https://search.agora-backbone.net"
    }
  ]
}
#+end_src

*** Resolution & Routing Logic
1. *Discovery:* When a client wants to interact with a Persona, it first resolves the DID to its latest DID Document (via the KEL or a resolver).
2. *Binding:* The client extracts the `AgoraPDS` endpoint to fetch content and the `AgoraRelay` endpoint to subscribe to real-time updates.
3. *Failover:* If a primary PDS is unreachable, the client attempts to resolve alternative endpoints if provided in the service list (supporting the multi-homed data model).

** Client Architecture: PDS-Proximate / Thin Client Model

*** Concept

The social protocol's architectural strategy for client applications aims to balance user sovereignty with broad accessibility and app store compliance. Instead of relying solely on "sovereign clients" (full-featured applications running entirely on edge devices), a hybrid approach will be adopted: core client logic will reside closer to the Personal Data Store (PDS), with only a "thin client" deployed on edge devices (e.g., mobile apps, web browsers). This allows for greater flexibility in distribution and development.

*** Motivation: App Store Compliance & Broad Reach

Traditional "sovereign client" models, where full application logic, data processing, and cryptographic operations occur entirely on the user's edge device, can encounter significant hurdles with mainstream app stores (e.g., Apple App Store, Google Play Store). These platforms often impose restrictions on:

-   Custom cryptographic implementations
-   Direct access to underlying network protocols
-   Data storage and handling outside platform-defined sandboxes
-   Features deemed to circumvent platform monetization or control

The PDS-proximate / thin client model is a pragmatic solution to navigate these limitations, enabling the social protocol to reach a wider user base through conventional app distribution channels without compromising core protocol principles.

*** Strategic Advantages

1.  *Enhanced App Store Compliance:* A thin client, functioning primarily as a user interface and communication layer with the PDS, is less likely to violate app store guidelines, increasing the likelihood of approval and sustained availability.
2.  *Reduced Edge Device Footprint:* Lower computational, memory, and storage requirements on mobile phones and other edge devices. This translates to better performance, lower battery consumption, and broader compatibility across a range of hardware.
3.  *Streamlined Updates & Maintenance:* Core application logic and feature updates can be deployed directly on the PDS or associated infrastructure, minimizing the need for frequent client-side app store updates and accelerating development cycles.
4.  *Enriched PDS Functionality:* The PDS evolves from a passive data archive into a more active, "smart" personal server capable of hosting and executing significant portions of the client application logic. This allows for more sophisticated features (e.g., advanced indexing, content processing, AI integration) to run efficiently on behalf of the user.
5.  *Greater Platform Portability:* A thin client model naturally facilitates deployment across diverse platforms, including web browsers (via WebAssembly or JavaScript), minimal native mobile wrappers, and potentially embedded systems, ensuring a consistent user experience.

*** Architectural Considerations & Challenges

1.  *PDS Reliability and Performance:* The user experience becomes intrinsically linked to the performance, uptime, and responsiveness of the PDS. Robust PDS implementations and reliable hosting options are paramount.
2.  *Privacy and Trust Model:* While the PDS is personal to the user, moving client logic there means processing occurs "off-device." End-to-end encryption must remain a fundamental guarantee, ensuring the PDS only handles encrypted data where sensitive information is concerned. User control over the PDS becomes the primary locus of sovereignty.
3.  *Offline Capabilities:* Thin clients will inherently have limited or no offline functionality, as they rely on real-time communication with the PDS. Strategies for graceful degradation and cached read-only access for essential data will be necessary.
4.  *Definition of "Thinness":* A clear architectural specification is required to define the boundary between client logic executed on the PDS and the minimal essential logic (e.g., basic key management, UI rendering) that must reside on the edge device for security and usability.

*** Conclusion

The adoption of a PDS-proximate / thin client architecture is a strategic imperative for the social protocol to achieve mass adoption and navigate the complexities of modern app distribution, while simultaneously enhancing the capabilities of the Personal Data Store as a dynamic and powerful extension of the user's digital self.

** Related Documents

- [[id:f6cfc54b-919b-4311-bcbf-65e976755d40][The Primitive]] - Content Object structure and core encryption primitives.
- [[id:0f949f6c-4cf1-49eb-b9a4-ebcac27ee548][Social]] - Messaging models, social publishing, and the attention marketplace.
- [[id:6fe67db6-25bd-4d11-bd1d-b44ec809e858][Identity]] - Master Key (Psyche) and Persona governance.
- [[id:90484f4a-5b70-4001-93d6-e610e54ed573][Exchange and Contracts]] - Economic primitives, fee structures, and the SCAL layer.

** Gaps

- *None.* All identified gaps in the infrastructure layer have been resolved.